Background Inference on population health is commonly derived from health survey data, based on the assumption that they are representative of the target communities. Departure from representativeness may weaken external validity leading to biased estimates, with important implications for public health evidence, particularly in relation to prevalence and quantity estimates such as population alcohol consumption. A key aspect determining the extent to which surveys are representative is the level of participant response. Inverse probability weights based on a limited range of demographic variables are usually applied in an attempt to correct for non-response. We aimed to investigate whether weighted estimates of all-cause mortality and mortality from alcohol-related conditions derived from the 2003 Scottish Health Survey (SHeS) – with household response of 67% – reflect those in the population of Scotland.

Methods Baseline observations from SHeS were individually record-linked to mortality data for the 91% of respondents consenting to linkage, and directly age-standardised survey-weighted mortality rates were calculated for the 3117 men and 3980 women aged 20 years and older at interview. Equivalent mortality rates were calculated for the whole of Scotland in the same age group using population estimates and mortality data contemporaneous with the linked survey data.

Results There were 201 (6.4%) male deaths and 215 (5.4%) female deaths in the 2003 SHeS by the end of 2008. Among men, all-cause mortality was markedly lower in the SHeS sample (918 per 100,000 person-years [95% CI:850–987]) than in the Scottish population (1361 [95% CI:1357–1365]). Figures for women were also highly significantly different (739 [95% CI:682–795] for the SHeS and 928 [95% CI:925–931] for the Scottish population). Alcohol-related mortality was lower in the SHeS sample (38 [95% CI:18–57] in men and 11 [95% CI:0–22] in women) relative to the Scottish population (57 [95% CI:56–58] in men [non-significant] and 25 [95% CI:24–25] in women [significant]).

Conclusion Respondents to the 2003 SHeS differ from the population they are intended to represent, with much lower than expected all-cause mortality in both sexes; alcohol-related mortality rates were somewhat lower than expected suggesting lower alcohol consumption among survey respondents which, if genuine, would lead to inherent underestimation of population consumption levels. Importantly, differences existed despite the application of conventional weighting and age-standardisation methods. Consideration should be given to the levels of resource allocated for increasing survey response and the further development of survey methodology to address the resultant systematic bias in health survey data arising from non-response.