Elsevier

Public Health

Volume 119, Issue 9, September 2005, Pages 751-757
Public Health

Natural experiments: an underused tool for public health?

https://doi.org/10.1016/j.puhe.2004.11.008Get rights and content

Summary

Policymakers and public health researchers alike have demanded better evidence of the effects of interventions on health inequalities. These calls have been repeated most recently in the UK in the final Wanless report, which spoke of the “almost complete lack of an evidence base on the cost-effectiveness of public health interventions”, and pointed more generally to the limited evidence base for public health policy and practice. Wanless and others have suggested that the gaps may be partially filled by exploiting the opportunities offered by “natural experiments”, such as changes in employment opportunities, housing provision, or cigarette pricing. Natural experiments have an important contributions to make within the health inequalities agenda. First, they can play an important role in investigating the determinants of health inequalities. Second, they can assist in the identification of effective interventions, an area where it is widely acknowledged that the evidence-base is currently sparsely populated. This paper discusses some of the benefits and limitations of using this type of evidence, drawing on two ongoing quasi-experimental studies as examples.

Section snippets

Non-comparability of intervention and control groups

As with any observational study, there are many potential sources of bias. Chief among these is the likelihood that the control and intervention groups are dissimilar at baseline in ways that are related to health outcomes. In randomized controlled trials, both known and unknown factors are distributed at random, and therefore any difference in outcome can only be due to the intervention.14 By contrast, in natural experiments, health outcomes may differ amongst groups because of baseline

Natural experiments: the benefits

One potential benefit of natural experiments is their external validity, as they provide assessments of effectiveness rather than efficacy. However, this may be compromised by low internal validity.23 Their generalizability needs consideration, as the intervention may be so locally specific that study results cannot be generalized to other areas, even within the same country. This may necessitate multisite evaluations to explore effects in different settings. For example, the SHARP study is

Conclusion

Natural experiments are common in real life but are not in common use in public health research. Randomized controlled trials do represent ‘best available evidence’, but in many public health settings, they unfortunately represent best unavailable evidence because they can be problematic to achieve, or simply because of the unwillingness of governments to consider implementing interventions in a manner that would facilitate robust outcome assessment.2 A commitment to the use of natural

Acknowledgements

The work on the hypermarket study was undertaken by MP, SC, LS and AF, who received funding from the Department of Health. The views expressed in the publication are those of the authors and not necessarily those of the Department of Health. The SHARP study (conducted by MP, CH and AK) is funded by the Chief Scientist Office of the Scottish Executive Department of Health, and Communities Scotland. Mark Petticrew is funded by the Chief Scientist Office of the Scottish Executive Department of

References (32)

  • M. Joffe

    How do we make health impact assessment fit for purpose?

    Public Health

    (2003)
  • A. Truswell

    Levels and kinds of evidence for public-health nutrition

    Lancet

    (2001)
  • J. Mackenbach

    Tackling inequalities in health: the need for building a systematic evidence base

    J Epidemiol Community Health

    (2003)
  • S. Macintyre

    Evidence based policy making

    BMJ

    (2003)
  • D. Wanless

    Securing good health for the whole population: final report

    (2004)
  • R. Wilkinson

    Putting the picture together: prosperity, redistribution, health and welfare

  • C. Alvarez-Dardet

    The fall of the wall and gender as ‘natural’ experiments

    J Epidemiol Community Health

    (2000)
  • T. Blakely et al.

    What is the lag time between income inequality and health status?

    J Epidemiol Community Health

    (2000)
  • M. Bartley et al.

    Glossary: unemployment, job insecurity, and health

    J Epidemiol Community Health

    (2001)
  • L. Millward et al.

    Public health intervention research: the evidence

    (2001)
  • N. Black

    Why we need observational studies to evaluate the effectiveness of health care

    BMJ

    (1996)
  • J. Mackenbach et al.

    A strategy for tackling health inequalities in the Netherlands

    BMJ

    (2002)
  • K. Lock

    Health impact assessment

    BMJ

    (2000)
  • L. Rychetnik et al.

    Criteria for evaluating evidence on public health interventions

    J Epidemiol Community Health

    (2002)
  • C. Roberts et al.

    Randomisation methods in controlled trials

    BMJ

    (1998)
  • C. Tudor-Smith et al.

    Effects of the Heartbeat Wales programme over five years on behavioural risks for cardiovascular disease: quasi-experimental comparison of results from Wales and a matched reference area

    BMJ

    (1998)
  • Cited by (0)

    View full text