Article Text

Download PDFPDF

Synthetic control methodology as a tool for evaluating population-level health interventions
  1. Janet Bouttell1,
  2. Peter Craig2,
  3. James Lewsey1,
  4. Mark Robinson3,
  5. Frank Popham2
  1. 1 Health Economics and Health Technology Assessment, Institute of Health and Wellbeing, University of Glasgow, Glasgow, UK
  2. 2 MRC/CSO Social and Public Health Sciences Unit, Institute of Health and Wellbeing, University of Glasgow, Glasgow, UK
  3. 3 Public Health Observatory, NHS Health Scotland, Glasgow, UK
  1. Correspondence to Janet Bouttell, Health Economics and Health Technology Assessment, Institute of Health and Wellbeing, University of Glasgow, Glasgow, G12 8RZ, UK; Janet.Bouttell{at}glasgow.ac.uk

Abstract

Background Many public health interventions cannot be evaluated using randomised controlled trials so they rely on the assessment of observational data. Techniques for evaluating public health interventions using observational data include interrupted time series analysis, panel data regression-based approaches, regression discontinuity and instrumental variable approaches. The inclusion of a counterfactual improves causal inference for approaches based on time series analysis, but the selection of a suitable counterfactual or control area can be problematic. The synthetic control method builds a counterfactual using a weighted combination of potential control units.

Methods We explain the synthetic control method, summarise its use in health research to date, set out its advantages, assumptions and limitations and describe its implementation through a case study of life expectancy following German reunification.

Results Advantages of the synthetic control method are that it offers an approach suitable when there is a small number of treated units and control units and it does not rely on parallel preimplementation trends like difference in difference methods. The credibility of the result relies on achieving a good preimplementation fit for the outcome of interest between treated unit and synthetic control. If a good preimplementation fit is established over an extended period of time, a discrepancy in the outcome variable following the intervention can be interpreted as an intervention effect. It is critical that the synthetic control is built from a pool of potential controls that are similar to the treated unit. There is currently no consensus on what constitutes a ‘good fit’ or how to judge similarity. Traditional statistical inference is not appropriate with this approach, although alternatives are available. From our review, we noted that the synthetic control method has been underused in public health.

Conclusions Synthetic control methods are a valuable addition to the range of approaches for evaluating public health interventions when randomisation is impractical. They deserve to be more widely applied, ideally in combination with other methods so that the dependence of findings on particular assumptions can be assessed.

  • public health
  • health impact assessment
  • methodology
  • research methods

This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Footnotes

  • Contributors FP and JB drafted the paper. All authors were involved in the initial conception and design of the study, reviewed and commented on drafts of the article and approved the final article.

  • Funding PC and FP are funded by the Medical Research Council under grants MC_UU_12017/13 and MC_UU_12017/15 and the Scottish Government Chief Scientists Office under grants SPHSU13 and SPHSU15.

  • Competing interests None declared.

  • Patient consent Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.