Background Limited psychometric research has examined the reliability of self-reported measures of neighbourhood conditions, the effect of measurement error on associations between neighbourhood conditions and health, and potential differences in the reliabilities between neighbourhood strata (urban vs rural and low vs high poverty). We assessed overall and stratified reliability of self-reported perceived neighbourhood conditions using five scales (social and physical disorder, social control, social cohesion, fear) and four single items (multidimensional neighbouring). We also assessed measurement error-corrected associations of these conditions with self-rated health.

Methods Using random-digit dialling, 367 women without breast cancer (matched controls from a larger study) were interviewed twice, 2–3 weeks apart. Test–retest (intraclass correlation coefficients (ICC)/weighted κ) and internal consistency reliability (Cronbach's α) were assessed. Differences in reliability across neighbourhood strata were tested using bootstrap methods. Regression calibration corrected estimates for measurement error.

Results All measures demonstrated satisfactory internal consistency (α≥0.70) and either moderate (ICC/κ=0.41–0.60) or substantial (ICC/κ=0.61–0.80) test–retest reliability in the full sample. Internal consistency did not differ by neighbourhood strata. Test–retest reliability was significantly lower among rural (vs urban) residents for two scales (social control, physical disorder) and two multidimensional neighbouring items; test–retest reliability was higher for physical disorder and lower for one multidimensional neighbouring item among the high (vs low) poverty strata. After measurement error correction, the magnitude of associations between neighbourhood conditions and self-rated health were larger, particularly in the rural population.

Conclusion Research is needed to develop and test reliable measures of perceived neighbourhood conditions relevant to the health of rural populations.