Health impact assessment (HIA) is a decision support tool that is supposed to add value to the process of decision making by providing an analysis of the potential effects of a planned policy, programme, or project (PPP) on dimensions of health.^1–3 Specifically, HIA seeks to provide decision makers with information to mitigate the negative and maximise the positive impacts on health and health inequalities. A successful HIA is one where its findings are considered by decision makers to inform the development and implementation of a PPP^4,5

The past decade has witnessed a substantial growth in HIA activity, evidenced by the publication of academic articles and textbooks, the development of training courses and conferences, and the production of guidance at municipal, national, and supranational government levels. Somewhat surprisingly however, given that HIA explicitly seeks to influence decision making, there have been few approaches to identify the factors associated with its success in doing so.

The aim of this project is to identify from a range of sources the factors associated with an HIA’s success in integrating health considerations into the final decision and implementation of a PPP. This study was undertaken as part of the wider Health and Environmental Linkages Initiative (HELI) programme sponsored by the World Health Organisation (http://www.who.int/phe/health_topics/heli/en/). The HELI programme is a global effort to promote and facilitate action in developing countries to redress environmental threats to human health.⁶ A full report of the progress on the programme to date can be found at http://www.who.int/entity/phe/publications/helibrochureintro.pdf

METHODS

To address the study question three complementary approaches were undertaken to gather relevant information.

1. A review of HIA case studies.

2. A review of commentaries, reviews, and discussion papers relating to HIA and decision making, (commentaries).

3. An email survey of a purposive sample of HIA academics, HIA practitioners, and policymakers to explore perceptions of the characteristics of HIAs and any other factors that participants believed to have influenced the decision making process.

Identification of case studies and commentaries

A systematic review of the literature was not possible within the time allocated for this work within the HELI programme. We thus adopted a search strategy supplemented by a “checking process” by people familiar with the HIA literature.

Our search strategy targeted published and unpublished case studies and commentaries using the following methods:

• Interrogation of the UK Health Development Agency’s health impact assessment gateway web site (http://www.hiagateway.org.uk—this has now moved to http://www.publichealth.nice.org.uk)

• Interrogation of the WHO health impact assessment web site (http://www.who.int/hia/en)

• Interrogation of electronic databases: Medline1966-week 4 2004 and EMBASE 1980-week 5 2004 using the free text terms “health impact assessment” and “HIA”. The search was undertaken on both keyword and title.

No language restrictions were used and where possible translations of foreign articles were undertaken.

The list of references generated by the search was sent to members of an international HIA electronic discussion group (HIAnet) and to the Steering Group of the HELI Work Programme. Members of both groups were asked to respond if they felt any key papers or reports were missing.

After the exclusion of literature other than that concerned with HIA, the list of references were then subdivided into two groups, case studies and commentaries, on the basis of the paper’s title and abstract (where available). This process was undertaken independently by two authors, JM and JP, and any discrepancies were resolved through access to the full paper. Hard copies of papers were then obtained via electronic download, contact with authors, or from the British Library.

Review of case studies

The case studies were reviewed by four people: two authors (CD and JM) and two research associates based in the authors’ research unit. Each case study was reviewed by one researcher only because of pressures of time. Data were abstracted onto a standardised, paper form. The form was revised after a pilot review on a small sample of case studies undertaken by each reviewer to ensure consistency of abstraction. All reviewers followed written abstraction guidelines, resolving any outstanding queries during the review process through discussions with JM. Data from the paper record were transferred to an electronic database constructed in Microsoft access.

The form was designed to capture the following information from HIA case studies:

• HIA characteristics (as recorded by the case study authors): information on the year undertaken; geopolitical level; setting; sector; HIA type (project, programme, policy); methods and techniques used (including consideration of the published literature and public participation); identification of assessors (academic, service, community).

• Authors’ perceptions (commentary by the authors) linking characteristics of the HIA case study to the integration of findings into the decision making process.

After the form review and where there were missing data supplementary information was requested from authors via email.

Review of commentaries

Commentaries were reviewed by one researcher (JP) with regard to comments on the characteristics of HIAs encouraging integration of HIA findings in the decision making process.

Email survey of purposive sample of HIA academics, HIA practitioners, and policymakers

An electronic questionnaire was developed that sought awareness of factors that act as “enablers” or “barriers” to the integration of HIA findings and health considerations into decisions concerning PPPs. The questionnaire was sent to a purposive sample of 31 HIA academics, practitioners and policymakers working in 11 countries, primarily European.

RESULTS

Identified papers

Figure 1 details the literature identified during the review process, both for commentaries and for case studies. In total 550 references were identified. Of these 265 were of potential relevance, the remainder being obviously unrelated to HIA on review of the title and/or abstract.

Of the 265 references of potential relevance, 258 papers were captured by the electronic search strategy and 7 were suggested by the WHO HELI Steering Group. Members of HIAnet did not suggest any additional papers to those identified by the electronic search. Six of the seven papers suggested by the WHO HELI steering group had been recently published in a special HIA edition of the WHO Bulletin but HIA or health impact assessment had not been used in their title. The seventh paper had not been captured in the initial search because it was published just after the cut off for our search period. All were included and classed as commentaries for the purpose of our review.

One hundred and thirteen of the 265 references of potential relevance were commentaries and 152 were case studies. Two commentaries could not be obtained for review. After review 79 commentaries were excluded as they contained no guidance on HIA characteristics that would encourage their use by decision makers. Twenty five case studies could not be obtained for review and 18 case studies were excluded before review as they were duplicates. After review 22 case study reports were excluded as the HIA was not explicitly linked to a decision or set of decisions. The final sample thus comprised 32 commentaries and 88 case studies.

Table 1 shows the characteristics of the 88 case studies.

Enablers and barriers to HIAs having influence in the decision making process

Table 2 summarises the key messages arising from the review of case studies, review of commentaries, and email survey with regard to factors acting as either enablers or barriers to the ability of an HIA to influence the decision making process. Ten (32%) people responded to the email survey.

The enablers and barriers have been classified into two broad categories, the first relating to the decision makers and the policy process and environment, and the second to the conduct and reporting of an HIA. Of the former, factors cited as enablers included the involvement of decision makers in the conduct and planning of the HIA, input from people outside the decision making process, and the importance of striking a balance between decision maker ownership and HIA credibility. A clear organisational commitment to HIA (including manpower, infrastructure, policy, and statutory framework support), the subject of the HIA not being a controversial issue, and the provision of realistic recommendations that concur with other political drivers were also cited as important enablers. Barriers included a lack of awareness of health by other sectors and a lack of knowledge relating to the policymaking environment on the part of those conducting the HIA.

With regards to the conduct of HIAs, enablers included timing the assessment to fit with the decision making process, the use of a consistent methodological approach, consideration of a broad range of factors that have a potential impact on health, the inclusion of empirical evidence relating the positive or negative effects of a PPP on health, and the quantification of impacts where possible. The conduct of HIAs by expert assessors was also cited as an enabler by bringing credibility to the results. Barriers cited were the lack of an established standard method for the conduct of HIAs and resource limitations (including staff and time).

In relation to reporting, the tailored presentation of findings and the shaping of recommendations to reflect organisational concerns were reported as enablers, and the use of jargon as a barrier.

DISCUSSION

This review represents the first attempt we are aware of to identify factors that are viewed as being associated with the effectiveness of HIA in influencing decision making processes. We specifically set out to identify these factors using the perceptions of people with experience of HIA through a review of case studies and commentaries, and an email survey of experts.

Most of the identified case studies were conducted in Europe from 1999 onwards. This undoubtedly reflects the substantial interest in HIA occurring in Europe since this time, particularly in the UK, Netherlands, and Sweden.⁷ The observed decrease in HIA activity after 2002 in our sample is more likely to be attributable to a delay between HIA conduct and reporting rather than a real decrease in HIA activity. However, the paucity of non-European case studies is probably also a reflection of bias in our search strategy. Specifically, the use of a UK based database of HIAs, the lack of published case studies currently available via medical electronic bibliographies, and the exclusive use of the search terms HIA and health impact assessment may have limited the ascertainment of non-European assessments. For example, we recognise that many appraisals of development projects in non-European countries have included elements of HIA and that these will not have been captured by our search.

Our search identified articles that had and had not been subject to peer review but we did not record this information during data abstraction. It is possible that articles that have been peer reviewed may differ in their reporting of methods compared with unpublished studies. However, the nature of these differences are difficult to speculate upon: it might be presumed that peer review would require authors to be detailed in their description of the HIA and as such these articles would be more likely to include all relevant information than those that have not been published. Conversely, the word limit in many peer reviewed journals may mean that information is lost in the need for brevity and thus unpublished material would provide greater detail on the methods adopted.

Each article was reviewed by only one person. We recognise the benefits of double blind data abstraction but were unable to do this because of constraints of time. We believe that the use of a structured form, abstraction guidelines, and piloting before full review will have minimised any errors.

Most HIAs identified were undertaken at local level. It is reasonable to assume that this reflects actual activity: such assessments are undoubtedly simpler in both conceptual and methodological terms. The fact that few HIAs of national or supranational policies or programmes were identified probably echoes this point. Indeed, there have been recent efforts to develop approaches for complex policy level HIAs.^8,9,10 However, the fact that larger scale policies with greater attendant health impacts have been less likely to be subject to HIA could suggest that they are also more politically sensitive, with proponents less willing to subject policies to assessment. Moreover, policy level HIAs may be subject to a greater degree of uncertainty about possible impacts than project level assessments where the intervention/s can be more clearly defined and described. Indeed, even in countries such as the UK where there have been explicit policy commitments to HIA, little activity exists at a national governmental level.¹¹ In the Netherlands where HIA activity has been more integrated with national policy formulation, experience of implementing HIAs and subsequent recommendations has been frustrating.¹²

Most identified case studies were conducted prospectively with a view to modifying the PPP concerned before implementation. Less than one fifth of HIAs were attempting to inform a decision relating to the adoption of a PPP. This suggests that at present most HIAs are undertaken as a remedial process. It might be argued that, where possible, more common use of HIA to aid the selection of appropriate policy options may have more impact with regards to making healthy public policy.

In relation to the conduct of HIAs, the description provided by most authors was a “rapid” assessment. This may be a reflection of the resources currently dedicated to HIA. However, it should be noted that we were reliant on authors’ descriptions for type and timing of the HIA, and that there are no clear cut definitions of rapid, intermediate, or comprehensive. The lack of consideration of the published literature in over one quarter of assessments and uncertainty in a further 58% is perhaps surprising given the central role advocated for this activity in HIA guidance.^13,14 We would expect some reference to the literature as a means of informing an HIA in all but the briefest of screening assessments. Forty eight (55%) HIAs reported some form of community participation in the assessment process. However, information on the nature and extent of the participation was generally poorly reported.

The response from authors of case studies to the request for additional information was low. This was partly a reflection of people having moved on to work on different projects and/or in different organisations. However, many authors who did respond frequently indicated that follow up and monitoring, especially relating to impacts on decision making, had not been built into the design of the HIA. The need for a more systematic approach to the monitoring and quality assurance of HIAs has been argued for elsewhere, and a proposed framework developed.¹⁵

Alongside the case study review we used two other sources, a review of commentaries and an email survey of HIA experts, to identify factors that may be enablers and/or barriers to the eventual HIA influence on decision making. The commentaries focused mainly on the practicalities of HIA execution (for example, how to gather data, how to model impacts) rather than on the effectiveness of these methods in influencing decision making. Only a small number of reports specifically set out to reflect upon the HIA process and the interaction with decision making, and only four used actual examples as a basis for commentary.^16–19

The response to the email survey of HIA academics and practitioners and policy makers was low. None the less, the triangulation of the findings from all three data sources (review of case studies, review of commentaries, and email survey) does show some agreement, thereby offering some reassurance as to the validity of the findings gleaned. However, it should be acknowledged that these three groups of people with knowledge to contribute about the application of HIA are not always mutually exclusive. It may therefore be useful to gain additional perspective on the findings of this research, for example, from other fields such as policy studies or political science, and from other interested parties, such as those working in organisations that are being asked to commission, or to act upon the findings, of HIAs.

The findings in this review emphasise in particular the importance of considering the politico-administrative environment in which HIA operates. HIAs have “successfully” influenced decision making relating to PPPs when (1) key decision makers are involved in the design and conduct of the HIA; (2) there is organisational commitment to HIA; and/or (3) where statutory frameworks provide legitimacy for HIA within the policy process. They have had influence where HIA recommendations are realistic (read achievable) and echo political drivers. If these findings are correct they produce a paradox for HIA in that its success may demand compromises around core values such as independence and impartiality. That is, to maximise the likelihood of an HIA being considered by decision makers requires those decision makers to be involved in some way with the process, and for the process to “fit” the local politico-administrative environment.²⁰ This conflict may be most apparent when organisations undertake HIAs on their own work programmes.²¹

Some recommendations relating to the conduct of HIAs are of obvious face validity—for example, ensuring that the HIA is undertaken within a time frame concordant with the decision making process. Others—for example, the quantification of impacts—may, in the light of other work, be considered to be more contentious.^22,23 The suggestion that the lack of a standard methodology may be a barrier, and conversely that a consistent approach may facilitate the use of HIAs is particularly interesting however. Some practitioners have pointed out that institutionalising the HIA process within decision making organisations is the most important step if HIA is to be adopted by policy makers.²⁴ This implies that the extent to which HIA fits the requirements of organisations and decision makers may be as important as the technical methods adopted to undertake it. If this is so, then “successful” HIAs of identical proposals might look very different if undertaken in/for different organisations. Practitioners’ focus may in the future need to shift solely from the assessment of impacts in itself to an understanding of the policymaking process and environment. This will determine the type of information and evidence needed, and in turn this determines the characteristics of the HIA.²⁵

Finally, we should reflect on whether it is possible to link the characteristics of HIA to success, if we define “success” in terms of evidence of a change in/influence on a specific policy action or decision. HIA may bring about greater consideration of health into policymaking via less tangible routes—for example, by encouraging intersectoral working, by raising awareness of the wider determinants of health and so forth. As one respondent to the survey noted:

if the HIA is undertaken in a context where it otherwise would not have been then it will have put health on the agenda and therefore cannot have ‘failed’. It may not have been influential in changing a policy in a given situation but will have had some impact in its own right. The policy process is not a fixed linear activity that can be influenced by one facet of an activity undertaken by one group of people but is impacted upon by a range of complex interrelated factors—it is impossible to attribute failure or otherwise to an HIA in this way. The HIA is not something fixed but is a set of processes with multiple outcomes. All that can be attributed is that the findings of an HIA were not adopted in a given decision-making context –that is very different from the HIA being a failure. There is a danger that taken simplistically the whole HIA process can be reduced to a failure simply because the findings fail to show how the outcomes have influenced the policy. One of the main goals of HIA is the building of healthy public policy and it is questionable whether the success or benefit of the entire HIA process can be attributed to these kinds of outcomes in the way that the questions in this survey imply.