Evidence from research can be used to direct or deny investment in health promoting and preventive public health interventions and thus can be a contentious issue. Evidence takes many forms and is only one component of the decision making process that determines priorities for health promotion and public health investment. Decisions in public health also rely on economic, social, and political considerations.¹ Scientific research (including studies using quantitative and qualitative methodologies) can form one component of the evidence picture. Systematically summarising and synthesising the evidence has therefore provided a useful and important contribution to evidence based decision making.

Systematic reviews use explicit and well established methods to synthesise the results of separate but similar studies and are designed to reduce the potential biases associated with the appraisal of individual studies, and have been used to examine an array of contemporary and often contentious “real world” issues.² Cochrane reviews are conducted with specific editorial and peer review processes built in to ensure the production and publication of high quality reviews. There is also a commitment for (electronically) published reviews on the Cochrane Library (http://www.update-software.com/Cochrane/) to be updated at least every two years and to be responsive to comments and criticism as they arise.

The Cochrane Health Promotion and Public Health Field (the HPPH Field) is an entity of the Cochrane Collaboration with a remit to promote the interests and needs of those in health promotion and public health and also to raise the awareness of Cochrane reviews among this audience. Overall, the aim of the HPPH Field is to increase the number, quality, and utility of Cochrane reviews of the effectiveness of health promotion and public health interventions. The Cochrane Collaboration has historically been focused on the production of systematic reviews of the effectiveness of medical, surgical, and pharmacological interventions. However, the Cochrane Library has published a range of public health related reviews of interventions directed at the population or designed to promote the health of individuals and target groups. In January 2003, we reviewed the number and content of reviews published in issue 1 of the Cochrane Library 2003. Altogether 142 reviews and 105 protocols of health promotion or public health interventions were listed; a further, 66 relevant titles (pre-Cochrane protocol stage) had been registered with 25 Cochrane Collaborative Review Groups (the editorial working groups of the Collaboration). Most of these completed and planned reviews will have been researcher driven reviews, or research driven (based on the availability of existing trials).

While there is information on how to do, find, and implement the results of systematic reviews, there is little documentation on how to prioritise topics for systematic reviews. This paper is concerned with the potential for consultative processes to be used to prioritise public health topics for future Cochrane reviews. The approach has been used to determine review topics by the needs of potential end users, as compared with researcher driven topic determination. Lomas et al recently commented on the need for greater involvement of research users in setting priorities for health research and described two broad approaches to setting such priorities; technical assessments that rely heavily on data, and interpretive assessments, which seek consensus from a set of relevant participants. The authors argue that using an interpretive approach that involves the research’s potential users should increase the likelihood that the research will address important practical problems and that the results will actually be applied thereafter.³ It has also been contended that the participation of consumers and policymakers in the design, conduct, and reporting of systematic reviews can help to produce reviews that are relevant and understandable to target audiences.⁴ Defining the “consumer” is difficult to do in public health as all individuals, whether by consent or default, are often consumers of health promotion and public health interventions. In this paper we considered consumers to be the users and commissioners of systematic reviews, who should also by default, in their public advocacy role, represent lay consumers of public health interventions.

AIMS OF THE PROJECT

The purpose of this project was to identify a list of prioritised topics for Cochrane reviews. We also sought to develop a consultative process to engage global public health agencies to identify those which would be most useful to policy decision making, including topics for developing countries. This project is part of an overall strategy to ensure that the work of the HPPH Field is relevant to users of the reviews of public health evidence.

METHODS

The specific strategies were:

1. Development and use of a taskforce of individuals to identify and nominate global research needs

2. Identification of gaps in the current systematic review literature

3. Production of a list of useful review topics for decision making within public health agencies globally

4. Prioritisation of the nominated review topics, using an a priori set of criteria

5. Communication and dissemination of the list of prioritised review topics to achieve completed reviews.

(1) Development and use of an international taskforce of research advisors to identify and nominate global research needs

The terms of reference of the taskforce were “to identify and prioritise health promotion and public health review topics that are most needed for inclusion on the Cochrane Database of Systematic Reviews and to identify opportunities to promote and support the production of these reviews thereafter”. People representing organisations (including regional departments where appropriate) responsible for improving population health, representing the health promotion and public health professionals, with knowledge of decision makers for health policies, and/or with the ability to direct or influence research, in and for developing countries, were invited to participate. Eleven representatives from the World Health Organisation (from central and regional officers), International Union of Health Promotion and Education, Global Forum for Health Research, US Centers for Disease Control and Prevention, MacFarlane Burnet Institute (International Health Unit), the Medical Research Council of South Africa, and the Cochrane Health Promotion and Public Health Field were included in the taskforce of research advisors for this project. Teleconferences and email communication were used to achieve programme aims and objectives.

(2) Identification and analysis of gaps in the current systematic review literature

We searched the electronic literature to develop a list of existing systematic reviews of effectiveness (including Cochrane) of health promotion and public health interventions. Systematic reviews that had been conducted within the scope of large, or well funded, projects that aimed to conduct systematic reviews of the health promotion and public health literature using explicit methods and criteria were included. A project was concurrently being conducted in Canada seeking to identify appraise and make available all methodologically sound reviews in health promotion and public health conducted since 1985 (see: http://health-evidence.ca). We therefore restricted the scope of the current task in order for the results to be timely and to minimise duplication. The systematic review sources that were included were:

1. The Cochrane Library (http://www.cochrane.org),

2. Effective Public Health Practice Project (Canada) (http://www.city.hamilton.on.ca/sphs/EPHPP),

3. The Community Guide to Preventive Services (United States) (http://www.thecommunityguide.org),

4. the Health Development Agency Effectiveness reviews (United Kingdom) (http://www.hda-online.org.uk/evidence),

5. the Evidence for Policy and Practice Information and Coordinating (EPPI-) Centre (UK) (http://eppi.ioe.ac.uk/)

6. the National Health Service Research & Development Health Technology Programme (UK) (http://www.hta.nhsweb.nhs.uk).

The characteristics of reviews produced by each of these sources, including how review topics are chosen, when they are updated, the review process, study exclusion policies, how the reviews are funded, and availability/accessibility of the reviews, were collated.

Only reviews that were within the scope of the Cochrane HPPH Field (see: http://www.vichealth.vic.gov.au/cochrane/about/scope.htm) were included in the list of existing reviews. The scope of the Cochrane HPPH Field is narrower than most public health definitions (because of the need to minimise overlap with other Cochrane entities), however the effectiveness of vaccines was the one exception that was included as it is a particular area of priority for public health in low-middle income countries.

The WHO World Health Report 2002⁵ was used to further categorise, under the risk factor subject areas, the listed existing systematic reviews. The WHO report was selected as the most appropriate current globally endorsed document by which to start the mapping process. The report’s concluding recommendations informed the emphasis within the project—that is, (a) governments play a stronger part in formulating risk prevention policies, including more support for scientific research, (b) priority is given to prevention of global risks to health, and (c) highest priority be given to those interventions that are cost effective and affordable.

(3) Production of a list of useful review topics for decision making within public health agencies globally

Several models were identified as illustrations of how research topics have been prioritised by various organisations, to identify appropriate, evidence based and strategic criteria for setting priorities. Common themes were identified and discussed within the group, with eventual consensus that each member would broadly consider four factors when prioritising their chosen topics for prioritisation:

1. Burden of disease, magnitude of problem, urgency

2. Importance to developing countries

3. Avoidance of duplication

4. Opportunity for action

Taskforce members were given the opportunity to nominate topics to be considered, from which a list of topics for further prioritisation would be developed.

(4) Prioritisation of the nominated review topics, using an a priori set of criteria

Each member was asked to identify their prioritised 10 topics by considering the agreed to criteria, drawing upon their own knowledge and judgment about global research needs, and in consultation with colleagues, as appropriate. A final top 15 priority list was produced by aggregating the topic choices of the responding members and listing in order of the most chosen topics.

(5) Communication and dissemination of the list of prioritised review topics

Strategic communication strategies, within and external to the Cochrane Collaboration, are being developed. We are communicating and engaging with global organisations that are likely to be in a position to fund review and for whom the results are of use.

RESULTS

Recording the evidence base and gap analysis

Using the defined scope of the Cochrane HPPH Field, 381 systematic reviews were identified from the selected projects as at September 2002; 186 were Cochrane reviews or protocols. Most of these addressed infectious diseases (mainly on the effectiveness of vaccinations), tobacco control, and sexual health (including the prevention of HIV). Reviews covered in The Community Guide were also well represented under these three topic areas.

The systematic review topics were categorised by the risk factors identified in the World Health Report 2002. Nineteen per cent (n = 75: Cochrane 35) reviewed interventions related to addictive substances, particularly tobacco control, but only one addressing prevention of illicit drug use. “Other addictive drug topics” were being planned as a topic for the Community Guide at the time. Sixteen per cent covered infectious diseases (primarily vaccine effectiveness); and 16% covered diet related risks and physical activity. Thirteen per cent addressed sexual health interventions including HIV prevention while 4% addressed child and maternal under-nutrition. Mental health (11%) and injury prevention and safety promotion (7%) were also represented, although not included as a category by the World Health report. Areas highlighted by the World Health report as important health risks, but not addressed to any great extent by the identified systematic reviews (less than 3% of total number of reviews) included prevention of health risks in areas of oral health, respiratory, environment, occupation, and selected risks to health: unsafe health care and childhood sexual abuse.

List of prioritised topics for review

Taskforce members were asked to put up potential review topics for further prioritisation and 31 topics were suggested and then modified to reflect an appropriate review question (that is, an intervention or set of interventions to review for effectiveness). Eight broad topic areas were included in the this first master list: (1) working with the private sector; (2) mental and social health; (3) settings based health promotion; (4) housing; (5) environmental health; (6) maternal and newborn, and child health; (7) nutrition; and (8) tobacco control. The topics invariably reflected the area of expertise and interest of the members. This was expected and not unwanted. The final list is broad and covers many dimensions of contemporary health promotion and public health practice. Many of the titles reflect complex interventions that will require sophisticated critical appraisal and systematic reviewing skills.

The box shows the top 15 review topics identified as major priorities. The further 11 topics making up the 26 identified by the taskforce can be found on the Cochrane HPPH Field’s web site (http://www.vichealth.vic.gov.au/cochrane/activities/priorities.htm).

Disseminating review topics

Each of the relevant Cochrane Collaborative Review Groups has been notified of topics falling within their scope. The Cochrane HPPH Field will actively seek resources for the production of these reviews, in collaboration with the appropriate Review Groups. All systematic review topics are now displayed on the Cochrane HPPH Field’s web site (http://www.vichealth.vic.gov.au/cochrane/activities/priorities.htm) and we have begun the process of matching review topics to potential reviewers seeking topics and those that have expressed interest on the Cochrane HPPH Field’s contact database. We will continue to advocate for funding for these reviews in global health forums and with specific organisations and funding bodies. We believe the dissemination process will be critical to achieving this, recognising that incentive to act can be stimulated by public recognition of gaps in the research and an identified need to act by significant stakeholders.

DISCUSSION

This study was the first priority setting exercise of the Cochrane HPPH Field. Rather than use a researcher driven approach it sought to consult with the users of public health systematic reviews, scope the existing reviews against global health burden priorities and using explicit evidence based criteria, make recommendations for a future work programme. The completion of these priority reviews will make a significant contribution to the extent to which policy makers can be certain that reviews of the research literature are available electronically and regularly updated.

There are limitations to the study that need to be noted and considered. The process of identifying priorities in global research is difficult and necessarily combined with research based, political, and pragmatic decisions. Various criteria for setting priorities to improve the effectiveness of medical care have been devised in response to the recognised need to close the gap between evidence and practice⁶ and for setting priorities for clinical practice guidelines.⁷ This project aimed to use an explicit consultative method, guided by a broad analysis of the gaps in the evidence base to produce a useful outcome. The process used was not able to be completely exhaustive and therefore did not set out to emulate more structurally formal, resource intensive projects such as The US Preventive Services Task Force, an expert panel established by the US federal government in 1984 to develop evidence based practice guidelines on screening tests and other preventive services.⁸ The outcome, a set of prioritised systematic reviews of public health interventions, will inform the debate and can be used in a variety of forums to highlight other areas of public health where a review of the evidence base is warranted.

Stage 2 of this project will recruit a wider range of people, from a larger pool of organisations (from local to interventional levels), representing all aspects of public health, and including consumers of preventive interventions. Such an undertaking was not possible for the first stage of the project described here, however, given the limited existing resources the overall conclusion by those involved was to make a start.

The priorities list may seem weighted to interventions less relevant to developing countries, however we believe this is not the case. WHO 2002 and 2003 World Health Reports have highlighted the importance of defining the scope of global health as broader then infectious diseases, undernutrition, and maternal deaths.^9,10 Currently, cardiovascular disease is a major killer in developed and developing countries; the current prevalence of tobacco use, alcohol misuse, and unhealthy diets in children in many low-middle income countries are rising especially in fast growing urban areas. Already the diseases they cause—cardiovascular disease, cancers, diabetes, and chronic respiratory diseases—are resulting in significant negative health and economic impacts in those countries. No longer are “lifestyle” diseases—diseases of affluence. However, the global donor and research community has not responded to this new reality.^11–13 Research by developing countries is critical not only to emphasise the importance of chronic diseases and their associated risks, but more importantly to start developing new science based solutions adapted to developing county realities.

This process has highlighted a number of topics that are of imminent relevance and priority to global public health organisations. Participants in the project were selected based on their background and organisational responsibility, and their willingness to participate. They brought a range of knowledge and expertise in research and users’ evidence needs. As a result, the group was able to identify a broad range of useful and often complex interventions for review that will make an extremely valuable contribution to the evidence base. A challenge for reviewers of these topics will be to reflect on the evidence from developing and industrialised countries and to synthesise these in a way that is useful to both contexts. The broad and complex topics included in the list will call for thoughtful and innovative review methodology. The Cochrane HPPH Field has also convened an international taskforce of researchers with expertise in systematic reviews to produce recommendations as to how to conduct reviews of complex interventions to assist reviewers.

Many of the interventions in the priority list have been adopted by practitioners and organisations based on their theoretical underpinnings and/or models or frameworks that have produced degrees of success in the past. Well conducted, regularly updated Cochrane reviews of these interventions will add to the current effectiveness literature; in some cases the review may highlight questions that the primary research has not yet answered and will act to signal the need to direct funding to support primary research in areas of global importance. For others it may illuminate contextual and process factors crucial to the success of an intervention, or set of interventions.

CONCLUSION

The rigour and publication standards of Cochrane systematic reviews have the potential to strengthen the evidence base for health promotion and public health. Far from being a threat to the art and science of health promotion and public health, reviews of the relevant evidence in these areas identified as priorities will provide a foundation for evidence based decision making in these global areas of health concern. Cochrane reviews are only one component of the decision making process, and not all interventions lend themselves to the processes and format of a Cochrane review, therefore the Cochrane HPPH Field is committed to advancing projects that seek to increase the utility of reviews to public health workforce and consumers.

For more information on Cochrane reviews in health promotion and public health, to review recently completed guidelines for Cochrane public health systematic reviews, or to suggest interventions for future Cochrane reviews, please visit the HPPH Field’s web site (http://www.vichealth.vic.gov.au/cochrane).