Background and Objectives As well as guidance on clinical interventions, the National Institute for Health and Clinical Effectiveness (NICE) produces evidence-based public health guidance. The aim of this study was to explore the gap between the requirement for robust research evidence to support guidance development and the actual availability of evidence on a range of topics.

Design Quantitative data were extracted from the 14 reviews (for 7 projects) carried out by the School of Health and Related Research (ScHARR) Public Health Collaborating Centres since May 2008 to assist and inform NICE public health guidance. Literature searches were carried out using best current practice and used iterative methods for maximum efficiency. Qualitative information was obtained from review of notes of both formal and informal meetings of participants from ScHARR, NICE, and NICE advisory committees.

Main Outcome Measures The two types of outcome measures were: the proportion of actually relevant papers, by type and quality, that were identified from the potentially relevant papers retrieved by electronic literature searches; and the views of those taking part in this process.

Results There was only a very low yield of relevant research studies that addressed the questions posed by the scope of the reviews. The average for the 14 reviews examined was 2.2% (range 0.23–11.8%) of around 3000 searched articles per review. In spite of search strategies that are designed for high sensitivity, there were frequent comments from NICE and its committee members about the lack of research material available, and requests for additional searches which rarely provided additional relevant material. Specific problems identified include: poor development of the public health intervention; inadequate description of the intervention; insufficient suitably controlled trials; study duration too short; use of process or intermediate outcome measures rather than hard outcomes such as survival or quality of life; lack of health economic studies to inform the cost-effectiveness of the public health interventions.

Conclusion Overall it is found that there is a low return from the existing published evidence – deficiencies are found in both the quality and quantity of papers that evaluate public health interventions so that the NICE committees producing guidance feel dissatisfied with the evidence available. This emphasises the need for better research in public health which addresses the question of what interventions are effective and cost-effective in practice.