Natural experiments: an underused tool for public health?
Section snippets
Non-comparability of intervention and control groups
As with any observational study, there are many potential sources of bias. Chief among these is the likelihood that the control and intervention groups are dissimilar at baseline in ways that are related to health outcomes. In randomized controlled trials, both known and unknown factors are distributed at random, and therefore any difference in outcome can only be due to the intervention.14 By contrast, in natural experiments, health outcomes may differ amongst groups because of baseline
Natural experiments: the benefits
One potential benefit of natural experiments is their external validity, as they provide assessments of effectiveness rather than efficacy. However, this may be compromised by low internal validity.23 Their generalizability needs consideration, as the intervention may be so locally specific that study results cannot be generalized to other areas, even within the same country. This may necessitate multisite evaluations to explore effects in different settings. For example, the SHARP study is
Conclusion
Natural experiments are common in real life but are not in common use in public health research. Randomized controlled trials do represent ‘best available evidence’, but in many public health settings, they unfortunately represent best unavailable evidence because they can be problematic to achieve, or simply because of the unwillingness of governments to consider implementing interventions in a manner that would facilitate robust outcome assessment.2 A commitment to the use of natural
Acknowledgements
The work on the hypermarket study was undertaken by MP, SC, LS and AF, who received funding from the Department of Health. The views expressed in the publication are those of the authors and not necessarily those of the Department of Health. The SHARP study (conducted by MP, CH and AK) is funded by the Chief Scientist Office of the Scottish Executive Department of Health, and Communities Scotland. Mark Petticrew is funded by the Chief Scientist Office of the Scottish Executive Department of
References (32)
How do we make health impact assessment fit for purpose?
Public Health
(2003)Levels and kinds of evidence for public-health nutrition
Lancet
(2001)Tackling inequalities in health: the need for building a systematic evidence base
J Epidemiol Community Health
(2003)Evidence based policy making
BMJ
(2003)Securing good health for the whole population: final report
(2004)Putting the picture together: prosperity, redistribution, health and welfare
The fall of the wall and gender as ‘natural’ experiments
J Epidemiol Community Health
(2000)- et al.
What is the lag time between income inequality and health status?
J Epidemiol Community Health
(2000) - et al.
Glossary: unemployment, job insecurity, and health
J Epidemiol Community Health
(2001) - et al.
Public health intervention research: the evidence
(2001)
Why we need observational studies to evaluate the effectiveness of health care
BMJ
A strategy for tackling health inequalities in the Netherlands
BMJ
Health impact assessment
BMJ
Criteria for evaluating evidence on public health interventions
J Epidemiol Community Health
Randomisation methods in controlled trials
BMJ
Effects of the Heartbeat Wales programme over five years on behavioural risks for cardiovascular disease: quasi-experimental comparison of results from Wales and a matched reference area
BMJ
Cited by (237)
The bricks and mortar of collaborative ecosystem-based restoration and management
2022, Journal of Great Lakes ResearchNurturing urban innovation and knowledge in the ongoing COVID-19 world
2022, Journal of Innovation and KnowledgeExplaining spatial accessibility to high-quality nursing home care in the US using machine learning
2022, Spatial and Spatio-temporal EpidemiologyFunding for preventative Children's Services and rates of children becoming looked after: A natural experiment using longitudinal area-level data in England
2021, Children and Youth Services Review