Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis
- Correspondence to Dr Eva Rehfuess, Institute for Medical Informatics, Biometry and Epidemiology, University of Munich, Marchioninistrasse 15, 81377 Munich, Germany; rehfuess{at}ibe.med.uni-muenchen.de
-
Contributors EAR had the original idea for this research; PV and EAR contributed actively to the design, analysis and interpretation, drafting the article and revising it critically. Both the authors read and approved the final manuscript.
- Accepted 8 May 2012
- Published Online First 31 July 2012
Abstract
Introduction Systematic reviews are a cornerstone of evidence-based public health. The method of appraising the quality of different intervention and observational study designs in such reviews remains an important challenge. This article examines the applicability of selected quality appraisal tools (QATs) and the impact of choice of tool on the meta-analysis of a published systematic review.
Methods The authors selected a systematic review on the effectiveness of hand washing with soap in preventing diarrhoea, covering a range of epidemiological study designs. 6 QATs were used to assess 13 studies meeting their inclusion criteria; component sections/questions were coded numerically to derive a summary score between −1 (low quality) and +1 (high quality) for each QAT and study. Heterogeneity in study quality was evaluated graphically using traffic light schemes and spider charts. Random effects meta-analysis was undertaken for all studies; sensitivity analyses for each QAT included only those studies with a score of 0 or above.
Results The authors found substantial heterogeneity in summary scores for a given study. Their main meta-analysis yielded an OR of 0.60 (95% CI 0.47 to 0.77) with most sensitivity analyses giving similar pooled effect sizes with wider CIs.
Discussion The six QATs differ greatly in applicability across study designs, approach to quality appraisal (ie, scale vs checklist, presence/absence of summary score), coverage of domains and quality of component questions and answers. Learning from advantages and disadvantages of each QAT, we recommend research into the development of a reliable QAT with a broad applicability across study designs.
- Public health
- systematic reviews
- diarrhoea
- methodology
- meta-analysis
- air pollution
- child health
- epidemiology
Footnotes
-
Funding Eva Rehfuess gratefully acknowledges financial support from the Munich Center of Health Sciences.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.
