Abstract

Large scale neuro-epidemiology studies using brain imaging need tissue segmentation methods to determine lesion or normal tissue volumes. These techniques are often validated statistically using analyses intended for one-dimensional not 2D or 3D data. We highlight drawbacks of this approach and propose an alternative. We measured intracranial (ICV), cerebrospinal fluid (CSF) and white matter lesion (WML) volumes on structural MR brain images of individuals from the Lothian Birth Cohort 1936(www.disconnectedmind.ed.ac.uk) with a reference standard (RS) and two automated methods (M1 and M2). We used 18 subjects representing a range of CSF and WML volumes. We compared agreement with Bland-Altman1 and similarity using the Jaccard Index.2 The Bland-Altman method suggested different agreement between the automated measures and RS than was apparent on visual inspection of the segmented volumes or the similarity index. For example, the difference between the ICV RS and M1 was larger (1.44%) than between RS and M2 (0.71%), but the similarity indices were 0.96 and 0.97 respectively. For CSF, the M2 volume had sixfold worse agreement with RS than M1 (mean difference M2=131.2 cm³ vs M1=20 cm³) but the mean similarity index was 0.54 for both methods. Apparently good agreement for WML volumes mirrored a high similarity index, but it was not always an indicator of good segmentation assessed visually. The validation of tissue/lesion segmentation methods on medical images for epidemiological studies should include spatial information by plotting similarity indices across the sample.