You are here

Article Text

Article menu

Download PDFPDF

Review
Return on investment of public health interventions: a systematic review
  1. Rebecca Masters1,2,
  2. Elspeth Anwar2,3,4,
  3. Brendan Collins2,4,
  4. Richard Cookson5,
  5. Simon Capewell2
  1. 1 North Wales Local Public Health Team, Public Health Wales, Mold, Flintshire, UK
  2. 2 Department of Public Health and Policy, University of Liverpool, UK
  3. 3 Department of Public Health, Halton Borough Council, Cheshire, UK
  4. 4 Department of Public Health, Wirral Metropolitan Borough Council, Merseyside, UK
  5. 5 Centre for Health Economics, University of York, UK
  1. Correspondence to Rebecca Masters, North Wales Local Public Health Team, Public Health Wales, Mold, Flintshire CH7 1PZ, UK; Rebecca.Mason{at}nhs.net

Abstract

Background Public sector austerity measures in many high-income countries mean that public health budgets are reducing year on year. To help inform the potential impact of these proposed disinvestments in public health, we set out to determine the return on investment (ROI) from a range of existing public health interventions.

Methods We conducted systematic searches on all relevant databases (including MEDLINE; EMBASE; CINAHL; AMED; PubMed, Cochrane and Scopus) to identify studies that calculated a ROI or cost-benefit ratio (CBR) for public health interventions in high-income countries.

Results We identified 2957 titles, and included 52 studies. The median ROI for public health interventions was 14.3 to 1, and median CBR was 8.3. The median ROI for all 29 local public health interventions was 4.1 to 1, and median CBR was 10.3. Even larger benefits were reported in 28 studies analysing nationwide public health interventions; the median ROI was 27.2, and median CBR was 17.5.

Conclusions This systematic review suggests that local and national public health interventions are highly cost-saving. Cuts to public health budgets in high income countries therefore represent a false economy, and are likely to generate billions of pounds of additional costs to health services and the wider economy.

  • PUBLIC HEALTH
  • ECONOMICS
  • Health inequalities
  • PUBLIC HEALTH POLICY
  • Economic evaluation

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

https://doi.org/10.1136/jech-2016-208141

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Benjamin Franklin once famously stated that “an ounce of prevention is worth a pound of cure”. Long-term pressures on public sector costs due to demographic and technological changes and cost inflation in the caring professions have intensified following the 2008 global financial crisis. Public health is often considered a politically soft target for budget cuts, as recently demonstrated by major budget reductions in the UK.1–3

    The benefits of population-level public health expenditure—unlike those of personal healthcare and social care expenditure—tend to be long term, mostly accruing after the current politicians and policymakers have moved on. Though large and certain at the population level, benefits are also seen as small and uncertain for individual voters. It is therefore important to take a hard look at the cost-effective evidence, and move towards more rational decision-making in this politically charged area.

    Return on investment (ROI) and cost-benefit ratio (CBR) are two forms of economic evaluation that value the financial return, or benefits, of an intervention against the total costs of its delivery. The CBR is the benefit divided by the cost, and the ROI is the benefit minus the cost expressed as a proportion of the cost, that is, the CBR−1. To help inform the discussion of proposed cuts to public health budgets, we set out to determine the ROI and opportunity cost for a range of public health interventions at the local and national levels. The theory underpinning this review is that, because political backing for public health intervention is often lacking, many interventions with a high ROI are not funded. This is because public health interventions are often opposed by powerful commercial interests, and the health gains for individuals are often perceived as too small to sway their voting intentions, despite adding up to large gains at the population level.4

    Methods

    We conducted a systematic review to examine the ROI of public health interventions delivered in high-income countries with universal healthcare. These included the UK, Western Europe, the USA, Canada, Japan, Australia and New Zealand.

    Search strategy

    The authors used Acheson's definition of public health when considering our search strategy: “The science and art of promoting and protecting health and well-being, preventing ill-health and prolonging life through the organised efforts of society”.5 This definition is purposefully broad and the authors felt that it would incorporate the various fields of public health. We searched the PubMed, MEDLINE, Scopus, CINAHL, Cochrane, PsycInfo and AMED databases using the following search terms: ‘public health’ (all fields) AND ‘return on investment’ OR ‘cost benefit analysis’ (title or abstract). We also hand searched the references of the included analyses to identify any further studies. A grey literature search was completed using Google, yielding three additional results. Limits were set to publications in the English language, and to interventions targeted at humans (where applicable). Studies with poor generalisability to the UK were excluded, including a number from the USA that may poorly reflect UK healthcare systems, structure and demographics.

    Study selection and inclusion criteria

    We included studies of any design that reported a ROI of public health interventions delivered in industrialised countries providing universal healthcare.

    Selection of articles and extraction of data

    One investigator (RM) performed the initial screening of the titles. A second reviewer (EA) independently reviewed the titles and potentially relevant abstracts. The results were cross-referenced and any disagreements were discussed with a third reviewer (BC).

    One investigator (RM) led the data extraction and quality assessment, which was then independently duplicated by EA. A third reviewer (BC) adjudicated on any disagreements regarding result details or quality assessment. RM contacted authors for additional data in three cases, with two responses.

    Assessment of methodological quality in included studies

    The methodological quality of each included study was assessed independently by two reviewers (RM and EA) using the National Institute for Health and Care Excellence (NICE) quality appraisal checklist for economic evaluations to assess the quality and external validity of each study.6 Disagreements in methodological quality assessments for all the included studies were resolved by consensus or by recourse to a third member of the review team (BC).

    Results

    We identified 2957 potentially relevant titles, after excluding 2559 duplicates. A further 2816 papers were excluded following title or abstract review. We finally included 52 relevant titles published over four decades (see online supplementary figure S1).

    Results were stratified by public health specialty (table 1), and by interventions at a local level (table 2) or national level (table 3). Results were reported in five different currencies, as detailed in tables 2 and 3.

    View this table:
    Table 1

    ROI of public health programmes overall, and stratified by level and specialism

    View this table:
    Table 2

    Return on investment of local public health programmes: specific studies

    View this table:
    Table 3

    Return on investment of national public health programmes: specific studies

    The median ROI for all public health interventions was 14.3, and the median CBR was 8.3.

    The reported ROI and CBRs ranged widely. The ROIs ranged from –21.27 (influenza vaccination of healthy workers7) to 221 (lead paint control8). The CBRs reported ranged from 0.66 (20 mph zones in low-impact areas9) to 167 (single measles vaccinations10). Studies reporting ROIs at the extreme end of the spectrum tended to be of poorer quality. Studies reporting a CBR tended to be higher quality.

    ROI of public health programmes stratified by specialism

    Analysis by specialism revealed that health protection and legislative interventions generally yielded high returns on investment, often being delivered on a national basis and only requiring a one-off intervention (such as a vaccination or a new tax). In contrast, interventions for healthcare public health, health promotion or wider determinants typically had lower returns, being often more complex, resource intensive and sustained. Figure 1 provides overviews of the median, maximum and minimum ROI by specialism, and figure 2 provides an overview of the median, maximum and minimum and CBR values stratified by specialism.

    Figure 1
    Figure 1

    Median, maximum and minimum return on investment values stratified by specialism.

    Figure 2
    Figure 2

    Median, maximum and minimum cost:benefit ratios stratified by specialism.

    Health protection interventions

    Eighteen studies reported a large ROI in relation to health protection. The ROI median was 34.2, and ranged from −21.37 to 2218, and the CBR median was 41.8 (range from 1.211 to 16710).

    Health promotion interventions

    Fifteen health promotion interventions were reported, 12 ROI studies and 3 CBR studies. Returns on investment were variable. The median ROI was 2.2 (range 0.712 to 6.213). The median CBR was much higher at 14.4 (range 210 to 29.414).

    Legislative interventions

    Four studies reported on legislative interventions, with substantial returns. The median ROI was 46.5 (range 3815 to 5515). The median CBR was 5.8 (range 310 to 8.616).

    Healthcare public health interventions

    Six studies reported ROIs in relation to healthcare public health interventions. The ROI median was 5.1, and ranged from 1.1517 to 19.35.18 No studies reported a CBR.

    Wider determinants interventions

    Twelve studies reported a return on wider determinants interventions (for instance, targeted at children or juvenile offenders). The median ROI was 5.6 (range 1.119 to 10.820) with a median CBR of 7.1 (range 0.667 to 23.621).

    ROI of public health interventions by level

    A total of 29 studies reported returns on investment or CBRs in relation to local public health interventions. The median ROI was 4.1, and ranged from −0.922 to 19.3.18 The median CBR was 10.3 (range −0.77 to 23.619).

    There were 28 studies reporting on national public health interventions. The median ROI was 27.2 and ranged from −21.37 to 221.8 The median CBR was 17.5 (range 1.211 to 1678).

    Discussion

    Our systematic review offers several potentially important observations

    First, even with the most rudimentary economic evaluations, it was clear that most public health interventions are substantially cost saving. This confirms our theory that public health interventions generally offer a considerable ROI. Median ROI was generally higher than median CBR in all of our key public health expenditure categories. This was because most studies only report one of these two measures, and studies that report ROI tend to have higher estimates. A direct comparison is possible, by converting between ROI and CBR at the study level using the simple formula ROI=CBR−1.

    Second, we demonstrated a public health ‘effectiveness hierarchy’. Public health interventions at a local level averaged an impressive ROI of 4, meaning that every pound invested yields a return of £4 plus the original investment back. However, ‘upstream’ interventions delivered on a national scale generally achieve even greater returns on investment, particularly legislation (a 10-fold higher ROI averaging 46).

    Third, Benjamin Franklin's belief that “an ounce of prevention is worth a pound of cure” is thus borne out by the costs-savings demonstrated, particularly when compared with recent returns for investment in healthcare.23 It has been estimated that investing an additional £13 000 in the English National Health Service (NHS) can achieve health benefits of one additional quality-adjusted life year (QALY).23 When this health benefit is valued in monetary terms at the UK Department of Health's current rate of £60 000 per QALY,24 this represents a ROI of 3.16 (£60 000−£13 000/£13 000).

    Fourth, this systematic review was partly prompted by recent government cuts to public health budgets in England. We therefore focused on public health interventions delivered in other high-income countries in order to maximise UK relevance. We can therefore now better estimate the likely opportunity costs of the proposed cuts in local and national public health budgets. The median ROI for all public health interventions was 14.3, and the median CBR was 8.3. An ROI of 14.3 implying a cash return of 1430% would sound too good to be true in the financial world. However, public health is different, because decision-making is governed by politics rather than markets. Our systematic review clearly demonstrates that there are big public health investment opportunities out there—they just need some political will to implement them. If we take the lower, conservative CBR figure of 8.3, this would suggest that the opportunity cost of the recent £200 million cuts to public health funding in England is likely to be eightfold higher, in the region of £1.6 billion. The UK government's ‘efficiency savings’ thus represent a false economy which will generate many billions of additional future costs to the ailing NHS and wider UK economy. The recent UK increases in (avoidable) teen pregnancies, sexually transmitted infections, homelessness and suicides are thus predictable and worrying. Do they represent harbingers of worse to come? Although this study draws on the experience of the UK public health system, there are implications for public health systems outside of the UK, which may be guided towards areas of potential underinvestment, and avoid harmful cuts in public health budgets.

    Previous reviews

    Ours may be the first comprehensive systematic review of ROI and CBR to include the broad spectrum of public health interventions. Furthermore, it extends and strengthens earlier, narrower reviews which consistently highlighted the cost-effectiveness of selected public health interventions. These included the Australian Assessing Cost Effectiveness (ACE) Prevention Study25 and the Health England Leading Priorities (HELP) Tool, which ranked several public health interventions against a set of criteria.26 NICE appraises and recommends public health programmes and interventions in England. In 2012, they reviewed 200 cost-effectiveness estimates used in their guidance. Many interventions (particularly around smoking cessation) produced a net cost-saving for the NHS, that is, the intervention was more effective and cheaper than the comparator.27 Most interventions were highly cost-effective with a very low cost per QALY: 85% were cost-effective at a threshold of £20 000 per QALY, and 89% at the higher £30 000 threshold, 5% exceeded £30 000 per QALY and only the final 5% were dominated (ie, more costly and less effective than the comparator).27

    Health protection interventions

    Eighteen studies reported ROI or CBR figures in relation to health protection interventions.7 ,8 ,10 ,11 ,28–40The median ROI for health protection interventions was very high at 34.2. The Australian single measles vaccination programme in the 1980s and 1990s reported the highest CBR, with a CBR of 167:1.10 The UK now uses the combined Measles, Mumps and Rubella vaccination that has an excellent ROI of 14:1.28

    Seven studies assessed the prevention, notification, follow-up and treatment of infectious diseases such as hepatitis B and HIV. Overall, they demonstrated a consistently high ROI, reflecting the high disease burden of infectious diseases and the huge benefits of prevention.29

    Calculating the ROI of influenza vaccination of healthy working adults is challenging, as it is highly sensitive to the efficacy of the seasonal vaccine. Thus, most such studies7 ,30 have reported a modest twofold ROI overall, but with extreme ROI values ranging from −21 to +174.30

    Legislative interventions

    One paper reported ROI in relation to legislative interventions, which offered substantial returns on investment, with a median ROI of 46.5. Furthermore, they are relatively low cost and target behaviour at a national level. Introducing a sugar sweetened beverage tax could save $55 for every single dollar invested15 in the USA.

    Health promotion interventions

    The 15 studies analysing health promotion interventions reported an overall twofold ROI with a more impressive median CBR of 14.4.10 ,13 ,14 ,22 ,41–51 Interventions aimed at reducing rates of falls are able to show one of the swiftest returns on investment of any of the public health interventions identified within this study, with a CBR of 20.6 returned within 18 months.41 Falls prevention interventions by their nature are relatively low cost (structured exercise programmes for those at risk of falls), and yet their potential impact on demand management for hospital services is clearly demonstrated. Shifting investment from secondary care for the treatment of falls to primary prevention would show significant and swift returns on investment.

    Tobacco control interventions10 ,42 ,43 overall reported a twofold ROIs, which increased when targeted at high-risk clients such as pregnant women.42 Such contrasting results perhaps highlight the complexity of public health interventions.

    Healthcare public health interventions

    Six studies17 ,18 ,52–55 reported healthcare public health results, offering a substantial median ROI of 5.14. The majority focused on disease management or medication adherence for high-risk patients, such as home blood pressure monitoring for hypertension diagnosis and treatment.18

    Wider determinants interventions

    Twelve studies reported results for wider determinants interventions.9 ,10 ,12 ,19–21 ,56–61 Public health interventions addressing wider determinants also averaged a fivefold ROI. Several studies assessed effectiveness of early years interventions, particularly those targeted at juvenile offenders, or those deemed to be at risk of future offending. Although much of this literature is from the USA20 ,21 ,56 ,57 emerging UK evidence demonstrates similar returns to society and the wider economy.58 The benefits of early years interventions thus extend far beyond health, with participants reporting improvements in literacy, job prospects and earnings (hence savings to the criminal justice system, increased taxation of higher earnings, etc).

    This also highlights the ‘cross-sector flow problem’: cost-effective public health programmes may not be commissioned if decision-makers are only looking through a narrow health lens.

    Strengths

    We describe a carefully conducted systematic review. Although the precision of application of ROI calculations varies widely, even the most rudimentary analyses consistently suggest that most public health interventions are substantially cost-saving.

    Limitations

    Several limitations should be considered. First, the difficulty of defining what constitutes a ‘public health intervention’, particularly those focused on wider determinants. We purposefully cast the net wide to achieve a broad systematic review. Further analysis of particular topic areas might now be beneficial.

    Second, publication bias appears likely, and even some published studies may have been missed. Such studies are inevitably scattered across a wide field of journals and some economic studies may only be available via organisational websites. However, we did search the grey literature and we did identify almost 3000 total studies—a reassuringly high number.

    Third, we did not conduct a formal meta-analysis because of the very inconsistent manner in which ROI was calculated, with differing cost perspectives, time horizons and discount rates. Discount rates ranged from 0% to 10%. A high discount rate disadvantages public health interventions that have a long payback time.62 Conversely, a 1 year time horizon may offer too short a time frame.

    Fourth, the generalisability of the interventions conducted from one country to the next will vary. Participants in US studies may poorly reflect UK demographics and vice versa. Furthermore, some studies focused on vaccination practices that are no longer employed in a number of countries (eg, single measles and haemophilus influenzae type b vaccinations). Similarly, the majority of workplace health promotion initiatives come from the USA, where employers who pay for employees' healthcare will have an additional financial incentive to promote the health of their workforce.

    Fifth, the quality of the economic evaluations varied considerably. Practice has clearly improved substantially since the 1970s, with recent evaluations employing more sophisticated modelling techniques. Designing such studies can be challenging as public health interventions are often complex and multifactorial, and it can be difficult to isolate an effect size even within a randomised controlled trial. Some of the published literature may therefore systematically overestimate or underestimate the ROI of interventions, and hence the need for more research.

    Unanswered questions and future research

    There is a clear need for further high-quality economic evaluations of public health interventions, which include a range of discount rates and robust sensitivity analyses.

    Implications for clinicians and policymakers

    Overall, the results of our systematic review clearly demonstrate that public health interventions are cost-saving, both to health services as well as the wider economy. Furthermore, some are very rapid: falls prevention interventions reported substantial returns within 6–12 months.41 One might reasonably expect equally rapid returns for preventive interventions such as immunisation, healthcare, smoking cessation and nutrition.63 Although attempting to quantify returns within a short timescale can be challenging, even larger returns on investment were seen over a 10–20 years time horizon.10 ,15 ,17 ,32 ,58 This has significant implications for policymakers, who often work to a much shorter time horizon (typically 3–5 years). We suggest that Public Health England, NICE and other advisory bodies therefore need to routinely emphasise that public health interventions can offer surprisingly rapid returns, which may increase further over the longer term.

    Conclusions

    This systematic review suggests that local public health interventions are cost-saving, and offer substantial returns on investment, nationwide programmes even more so.

    The cuts to public health budgets therefore represent a false economy. They are likely to generate billions of pounds of additional costs to the health services and wider economy.

    What is already known on this subject

    • It is well known that it is financially preferable for healthcare systems to aim to prevent ill health rather than to subsequently treat it. A number of studies have calculated the return on investment for individual prevention interventions; however, no systematic review has spanned the breadth of public health.

    What this study adds

    • This systematic review demonstrates a median return on investment of public health interventions of ∼14:1. Thus, for every £1 invested in public health, £14 will subsequently be returned to the wider health and social care economy. Furthermore, this review categorises the return on investment according to the public health specialty and local versus national levels of intervention. It suggests that cuts to public health services are short sighted and represent a false economy, with substantial opportunity costs.

    Acknowledgments

    The authors would like to thank Knowsley Metropolitan Borough Council Public Health Team, and Tracy Owen from the North West Commissioning Support Unit FADE Evidence Knowledge Centre, for their support in the development of this review.

    References

    1. ↵
      British Medical Association. Local authorities plunder ring fenced public health funds. 2015. (cited 15 October 2015). http://www.bma.org.uk/news-views-analysis/news/2015/january/local-authoritiesplunder-ringfenced-public-health-funds
    2. ↵
      The Kings Fund. Cuts to public health spending: the falsest of false economies. 2015 (cited 12 February 2016). http://www.kingsfund.org.uk/blog/2015/08/cuts-public-health-spending-falsestfalse-economies
    3. ↵
      British Medical Journal. Raiding the public health budget. 2014. (cited October 2015). http://www.bmj.com/content/348/bmj.g2274
    4. ↵
      2. Hauck K ,
      3. Smith P
      . Public choice analysis of public health priority setting. In: Cuyer A, ed . Encyclopaedia of health economics. Oxford, UK: Elsevier, 2014:184–93 .
    5. ↵
      Faculty of Public Health. What is Public Health [internet]. (cited 20 Jan 2017). http://www.fph.org.uk/what_is_public_health
    6. ↵
      National Institute for Clinical Excellence. Methods for the development of NICE public health guidance (third edition). Appendix I Quality appraisal checklist—economic evaluations. 2012. (cited 2 December 2015). https://www.nice.org.uk/article/pmg4/chapter/Appendix-I-Quality-appraisalchecklist-economic-evaluations
    7. ↵
      2. Nichol KL
      . Cost-benefit analysis of a strategy to vaccinate healthy working adults against influenza. Arch Intern Med 2001;161:749. doi:10.1001/archinte.161.5.749
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      2. Gould E
      . Childhood lead poisoning: conservative estimates of the social and economic benefits of lead hazard control. Environ Health Perspect 2009;117:1162–7. doi:10.1289/ehp.0800408
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      2. Peters JL ,
      3. Anderson R
      . The cost-effectiveness of mandatory 20mph zones for the prevention of injuries. J Public Health (Oxf) 2013;35:40–8. doi:10.1093/pubmed/fds067
      OpenUrlAbstract/FREE Full Text
    10. ↵
      2. Abelson P ,
      3. Taylor R ,
      4. Butler J , et al
      . Returns on Investment in Public Health. Canberra: Department of Health and Ageing, 2003.
    11. ↵
      2. Drummond M
      . Return on investment in needle and syringe programs in Australia: a summary report for the Commonwealth Department of Health and Ageing. Perth: Commonwealth of Australia, 2002.
    12. ↵
      2. Evans-Lacko S ,
      3. Henderson C ,
      4. Thornicroft G , et al
      . Economic evaluation of the anti-sigma social marketing campaign in England 2009-2011. Br J Psychiatry 2013;55:95–101. doi:10.1192/bjp.bp.112.113746
      OpenUrl
    13. ↵
      Trust for America's Health. Prevention for a healthier America: investments in disease prevention yield substantial savings, stronger communities. Washington DC: Trust for America, 2008.
    14. ↵
      2. McGuire A ,
      3. Hughes D
      . The Economics of Family Planning Services: a report prepared for the Contraceptive Alliance. 1995. London: Family Planning Association.
    15. ↵
      2. Gortmaker SL ,
      3. Long MW ,
      4. Resch SC , et al
      . Cost effectiveness of childhood obesity interventions: evidence and methods for CHOICES. Am J Prevent Med 2015;49:102–11. doi:10.1016/j.amepre.2015.03.032
      OpenUrlCrossRefPubMed
    16. ↵
      2. Miller TR ,
      3. Zaloshnja E ,
      4. Hendrie D
      . Cost-outcome analysis of booster seats for auto occupants aged 4 to 7 years. Pediatrics 2006;118:1994. doi:10.1542/peds.2006-1328
      OpenUrlAbstract/FREE Full Text
    17. ↵
      2. Van Vonno CJ ,
      3. Ozminkowski RJ ,
      4. Smith MW , et al
      . What can a pilot congestive heart failure disease management program tell us about likely return on investment? A case study from a program offered to federal employees. Dis Manag 2005;8:346–60.
      OpenUrlPubMed
    18. ↵
      2. Arrieta A ,
      3. Woods JR ,
      4. Qiao N , et al
      . Cost-benefit analysis of home blood pressure monitoring in hypertension diagnosis and treatment: an insurer perspective. Hypertension 2014;64:891–6. doi:10.1161/HYPERTENSIONAHA.114.03780
      OpenUrlCrossRef
    19. ↵
      2. Steinbach R ,
      3. Cairns J ,
      4. Grundy C , et al
      . Cost benefit analysis of 20mph zones in London. 2012. Inj Prev 2013;19:211–13. doi:10.1136/injuryprev-2012-040347
      OpenUrlAbstract/FREE Full Text
    20. ↵
      2. Reynolds AJ ,
      3. Temple JA ,
      4. White BAB , et al
      . Age 26 cost-benefit analysis of the child-parent center early education program. Child Dev 2011;82:397404. doi:10.1111/j.1467-8624.2010.01563.x
      OpenUrl
    21. ↵
      2. Kleitz SJ ,
      3. Borduin CM
      . Cost-benefit analysis of multisystematic therapy with serious and violent juvenile offenders. J Fam Psychol 2010;24:657–66. doi:10.1037/a0020838
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      2. Nyman JA ,
      3. Barleen NA ,
      4. Abraham JM
      . The effectiveness of health promotion at the University of Minnesota: expenditures, absenteeism, and participation in specific programs. Am Coll Occup Environ Med 2010;52:269–80. doi:10.1097/JOM.0b013e3181cf3aed
      OpenUrl
    23. ↵
      2. Claxton K ,
      3. Martin S ,
      4. Soares M , et al
      . Methods for the estimation of the NICE cost effectiveness threshold. Health Technol Assess 2015;19:1–203, iii–iv(cited 20 January 2017).
      OpenUrlCrossRefPubMed
    24. ↵
      2. Glover D ,
      3. Henderson J
      . Quantifying health impacts of government policies. London: Department of Health, 2010. (cited 20 January 2017) https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/216003/dh_120108.pdf .
    25. ↵
      2. Vos T ,
      3. Carter R ,
      4. Barendregt J , et al
      . Assessing Cost-Effectiveness in Prevention (ACE-Prevention): final report. Melbourne: University of Queensland, Brisbane and Deakin University, 2010.
    26. ↵
      Matrix Evidence. Prioritising investments in public health. 2009. (cited 1 March 2016) http://help.matrixknowledge.com/
    27. ↵
      2. Owen L ,
      3. Morgan A ,
      4. Fischer A , et al
      . The costeffectiveness of public health interventions. J Public Health 2012;34:37–45. doi:10.1093/pubmed/fdr075
      OpenUrlAbstract/FREE Full Text
    28. ↵
      2. White CC ,
      3. Koplan JP ,
      4. Orenstein WA
      . Benefits, risks and costs of immunization for measles, mumps and rubella. Am J Public Health 1985;75:739–44. doi:10.2105/AJPH.75.7.739
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      2. Creese A ,
      3. Floyd K ,
      4. Alban A , et al
      . Cost-effectiveness of HIV/AIDS interventions in Africa: a systematic review of the evidence. Lancet 2002;359:1635–42. doi:10.1016/S0140-6736(02)08595-1
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      2. Ding Y ,
      3. Zangwill KM ,
      4. Hay JW , et al
      . A cost-benefit analysis of in-hospital influenza vaccination of postpartum women. Obstet Gynecol 2012;119:306–14. doi:10.1097/AOG.0b013e318242af27
      OpenUrlPubMed
    31. ↵
      2. Andresen MA ,
      3. Boyd N
      . A cost-benefit and cost-effectiveness analysis of Vancouver's supervised injection facility. Int J Drug Policy 2010;21:70–6. doi:10.1016/j.drugpo.2009.03.004
      OpenUrlCrossRefPubMed
    32. ↵
      2. Boccalini S ,
      3. Taddei C ,
      4. Ceccherini V , et al
      . Economic analysis of the first 20 years of universal hepatitis vaccination program in Italy: an a posteriori evaluation and forecast of future benefits. Hum Vaccines Immunother 2013;9:51119–128. doi:10.4161/hv.23827
      OpenUrl
    33. ↵
      2. Garpenholt Ö ,
      3. Silfverdal S-A ,
      4. Levin L-Å
      . Economic evaluation of general childhood vaccination against haemophilus influenza type b in Sweden. Scand J Infect Dis 1998;30:5–10. doi:10.1080/003655498750002222
      OpenUrlPubMed
    34. ↵
      2. Holtgrave DR ,
      3. Valdiserri RO ,
      4. Gerber AR , et al
      . Human immunodeficiency virus counseling, testing, referral and partner notification services: a cost-benefit analysis. Arch Intern Med 1993;153:1225–30. doi:10.1001/archinte.1993.00410100057008
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      2. Hutchinson AB ,
      3. Farnham PG ,
      4. Duffy N , et al
      . Return on public health investment: CDC's expanded HIV testing initiative. J Acquir Immune Defic Syndr 2012;59:281–6. doi:10.1097/QAI.0b013e31823e5bee
      OpenUrlCrossRefPubMed
    36. ↵
      2. Kwon JA ,
      3. Anderson J ,
      4. Kerr CC , et al
      . Estimating the cost-effectiveness of needle syringe programs in Australia. AIDS 2012;26:2201–10. doi:10.1097/QAD.0b013e3283578b5d
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      2. Nguyen TQ ,
      3. Weir BW ,
      4. Des Jarlais DC , et al
      . Syringe exchange in the United States: a national level economic evaluation of hypothetical increases in investment. AIDS Behav 2014;18:2144–55. doi:10.1007/s10461-014-0789-9
      OpenUrl
    38. ↵
      2. Zhou F ,
      3. Bisgard KM ,
      4. Yusuf HR , et al
      . Impact of universal haemophilus influenzae type B vaccination starting at 2 months of age in the United States: an economic analysis. Pediatrics 2002;110:653–61. doi:10.1542/peds.110.4.653
      OpenUrlAbstract/FREE Full Text
    39. ↵
      2. Nelson W ,
      3. Swint JM
      . Cost-benefit analysis of flouridation in Houston, Texas. J Public Health Dent 1976;36:88–95. doi:10.1111/j.1752-7325.1976.tb02854.x
      OpenUrlPubMed
    40. ↵
      2. Romano PS ,
      3. Waitzman NJ ,
      4. Scheffler RM , et al
      . Folic acid fortification of grain: an economic analysis. Am J Public Health 1995;85:667–76. doi:10.2105/AJPH.85.5.667
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      2. Beard J ,
      3. Rowell D ,
      4. Scott D , et al
      . Economic analysis of a community-based falls prevention program. J R Inst Public Health 2006;120:742–51.
      OpenUrl
    42. ↵
      2. Collins B
      . Cost effectiveness of stop smoking services in Wirral: 2015 update. UK: Wirral, 2015.
    43. ↵
      2. Richard P ,
      3. West K ,
      4. Ku L
      . The return on investment of a Medicaid Tobacco Cessation Programme in Massachusetts. PLoS ONE 2012;7:e29665. doi:10.1371/journal.pone.0029665
      OpenUrlCrossRefPubMed
    44. ↵
      2. Baker KM ,
      3. Goetzel RZ ,
      4. Pei X , et al
      . Using a return-on-investment estimation model to evaluate outcomes from an obesity management worksite health promotion program. J Occup Environ Med 2008;50:981–90. doi:10.1097/JOM.0b013e318184a489
      OpenUrlCrossRefPubMed
    45. ↵
      2. Goetzel RZ ,
      3. Tabrizi M ,
      4. Henke RM , et al
      . Estimating the return on investment from a health risk management program offered to small Colorado-based employers. J Occup Environ Med 2014;56:554–60. doi:10.1097/JOM.0000000000000152
      OpenUrlCrossRefPubMed
    46. ↵
      2. Kuehl KS ,
      3. Elliot DL ,
      4. Goldberg L , et al
      . Economic benefit of the PHLAME wellness programme on firefighter injury. Occup Med (Lond) 2013;63:203–9. doi:10.1093/occmed/kqs232
      OpenUrlAbstract/FREE Full Text
    47. ↵
      2. Long DA ,
      3. Sheehan P
      . A case study of population health improvement at a Midwest Regional Hospital Employer. Popul Health Manag 2010;13:72–7. doi:10.1089/pop.2008.0034
    48. ↵
      2. Schwartz SM ,
      3. Mason ST ,
      4. Wang C , et al
      . Sustained economic value of a wellness and disease prevention program: an 8-year longitudinal evaluation. Popul Health Manag 2014;17:90–9. doi:10.1089/pop.2013.0042
      OpenUrl
    49. ↵
      2. Windsor RA ,
      3. Lowe JB ,
      4. Perkins LL , et al
      . Health education for pregnant smokers: Its behavioural impact and cost benefit. Am J Public Health 1993; 83:201–6. doi:10.2105/AJPH.83.2.201
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      2. Wang LY ,
      3. Vernon-Smiley M ,
      4. Gapinski MA , et al
      . Cost benefit study of school nursing services. JAMA Pediatr 2013;5441:642–8.
    51. ↵
      2. Ozminkowski RJ ,
      3. Dunn RL ,
      4. Goetzel RZ , et al
      . A return on investment evaluation of the Citibank, N.A, Health Management Program. Am J Health Promot 1999;14:31–43. doi:10.4278/0890-1171-14.1.31
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      2. Moore JM ,
      3. Shartle D ,
      4. Faudskar L , et al
      . Impact of a patient-centred pharmacy program and intervention in a high-risk group. J Manag Care Pharm 2013;19:228–36. doi:10.18553/jmcp.2013.19.3.228
      OpenUrl
    53. ↵
      2. Rundell OH ,
      3. Paredes A
      . Benefit-cost methodology in the evaluation of therapeutic services for alcoholism. Alcohol Clin Exp Res 1979;3:324–33. doi:10.1111/j.1530-0277.1979.tb05331.x
      OpenUrlPubMedWeb of Science
    54. ↵
      2. Spence MM ,
      3. Makerem AF ,
      4. Reyes SL , et al
      . Evaluation of an outpatient pharmacy clinical services program on adherence and clinical outcomes among patients with diabetes and/or coronary artery disease. J Manag Care Speciality Pharm 2014;20:1036–45. doi:10.18553/jmcp.2014.20.10.1036
      OpenUrl
    55. ↵
      2. Lokkerbol J ,
      3. Adema D ,
      4. Cuijpers P , et al
      . Improving the cost-effectiveness of a healthcare system for depressive disorders by implementing telemedicine: a health economic modelling study. Am J Geriatr Psychiatry 2014;22:253–62. doi:10.1016/j.jagp.2013.01.058
      OpenUrlCrossRefPubMed
    56. ↵
      2. Schweinhart LJ ,
      3. Montie J ,
      4. Xiang Z , et al
      . Lifetime effects: the HighScope Perry Preschool study through age 40. (Monographs of the HighScope Educational Research Foundation, 14). Ypsilanti, MI: HighScope Press, 2005.
    57. ↵
      2. Dopp AR ,
      3. Borduin CM ,
      4. Wagner DV , et al
      . The economic impact of multisystemic therapy through midlife: a cost-benefit analysis with serious juvenile offenders and their siblings. J Couns Clin Psychol 2014;82:694–705. doi:10.1037/a0036415
      OpenUrl
    58. ↵
      2. Bonin EM ,
      3. Stevens M ,
      4. Beecham J , et al
      . Costs and longer-term savings of parenting programmes for the prevention of persistent conduct disorder: a modelling study. BMC Public Health 2011;11:803. doi:10.1186/1471-2458-11-803
      OpenUrlCrossRefPubMed
    59. ↵
      2. Guo JY ,
      3. Gandavarapu S
      . An economic evaluation of health-promotive built environment changes. Prev Med 2010;50:44–9. doi:10.1016/j.ypmed.2009.08.019
      OpenUrl
    60. ↵
      2. Medivil J ,
      3. Garcia-Altes A ,
      4. Perez K , et al
      . Speed cameras in an urban setting: a cost-benefit analysis. Inj Prev 2012;18:75–80. doi:10.1136/ip.2010.030882
      OpenUrlAbstract/FREE Full Text
    61. ↵
      2. Wang G ,
      3. Macera CA ,
      4. Scudder-Soucie B , et al
      . A cost benefit analysis of physical activity using bike/pedestrian trails. Health Promot Pract 2005;6:174. doi:10.1177/1524839903260687
      OpenUrlCrossRefPubMed
    62. ↵
      2. Drummond M ,
      3. Brandt A ,
      4. Luce B , et al
      . Standardizing methodologies for economic evaluation in health care: practice, problems, and potential. Int J Technol Assess Health Care 1993;9:26–36.
      OpenUrlPubMed
    63. ↵
      2. Capewell S ,
      3. O'Flaherty M
      . Mortality falls can rapidly follow population-wide risk factor changes. Lancet 2011;378:752–3. doi:10.1016/S0140-6736(10)62302-1
      OpenUrlCrossRefPubMedWeb of Science
    View Abstract

    Supplementary materials

    Footnotes

    • Contributors RM drafted the paper, conducted the literature search, assessed the methodological quality of included studies and conducted the analysis. EA independently conducted the systematic review and assessed the methodological quality of the included studies. BC adjudicated in any disagreements in the methodological quality assessments, provided health economics advice and assisted with the data analysis. RC provided expert health economics advice and support throughout, and made significant contributions to the analysis and manuscript. SC devised the concept, supervised the project, provided expert public health advice and support throughout, and made significant contributions to the analysis and manuscript. All authors made substantial contributions to the analysis and manuscript, including final approval.

    • Funding RC is supported by the National Institute for Health Research (Senior Research Fellowship, SRF-2013-06-015).

    • Disclaimer The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.