Article Text

Download PDFPDF
OP10 Developing approaches for handling complexity in evidence from systematic reviews and meta-analyses of public health interventions
  1. Dylan Kneale1,
  2. Alison O’Mara-Eves1,
  3. Bridget Candy2,
  4. Katy Sutcliffe1,
  5. Lizze Cain3,
  6. Sandy Oliver1,
  7. Niccola Hutchinson-Pascal3,
  8. James Thomas1
  1. 1EPPI-Centre, UCL Social Research Institute, University College London, London, UK
  2. 2School of Life and Medical Sciences, University College London, London, UK
  3. 3Co-Production Collective, University College London, London, UK


Background The development of evidence-based strategies to tackle complex public health issues has been widely recommended. Nevertheless, methodologically robust sources of evidence may not necessarily be perceived as useful to decision-making in local settings. Despite their high regard, the ability to utilise evidence from meta-analyses and systematic reviews is hampered by the lack of explicit connection between the contexts in which interventions were evaluated and the context in which the evidence is to be applied. Here we present the results of work that set out to develop new approaches for exploring and enhancing the generalisability of meta-analysis through additional synthesis.

Methods We focused on children’s health as a case example and reanalyzed a meta-analysis of school-based interventions to reduce fat intake and a meta-analysis of school-based interventions to improve positive mental health. We assessed how using existing observational data and employing statistical approaches (namely reweighting of effect sizes to reflect the similarity to areas of interest (recalibration), regression analysis, and cluster analysis) in novel ways can help to create an overall measure of effect from meta-analysis that is more applicable to a defined population and/or more interpretable for decision-making.

Results Using a recalibration approach we found that, while overall the evidence suggests that the intervention under consideration was not effective in reducing fat intake, this interpretation changes when we place greater emphasis on the similarity of studies to particular settings. Namely, for both a fixed effect and random effects specification, studies that are more similar to Local Authorities of interest show greater effects and consequently contribute more towards the pooled effect size. In turn, the recalibrated effect size generated indicates a larger effect with a narrower confidence interval; in these areas we have greater confidence that school-based interventions will have an impact on reducing fat intake, and while the anticipated effect size remains small it may be substantial at a population level. Similarly, applying regression approaches to a meta-analysis of school-based interventions for children’s mental health resulted in a clearer message of the influence of context and population factors to guide decision-making.

Discussion The methodological advancements developed here for examining context in meta-analysis provide useful adjunct evidence to decision-makers, alongside existing meta-analytic evidence. This research also highlights the gulf between the deep and nuanced way in which diverse groups of stakeholders understand context, and the sparse treatment of context by researchers within trials and systematic reviews.

  • generalisability
  • meta-analysis
  • logic model

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.