rss
J Epidemiol Community Health 56:132-138 doi:10.1136/jech.56.2.132
  • Public health policy and practice

A framework for the evidence base to support Health Impact Assessment

  1. M Joffe,
  2. J Mindell
  1. Department of Epidemiology and Public Health, Imperial College School of Medicine, London
  1. Correspondence:
 Dr M Joffe, Department of Epidemiology and Public Health, Imperial College School of Medicine, St Mary's Campus, Norfolk Place, London W2 1PG, UK;
 m.joffe{at}ic.ac.uk
  • Accepted 3 July 2001

Abstract

Objective: To introduce a conceptual structure that can be used to organise the evidence base for Health Impact Assessment (HIA).

Background: HIA can be used to judge the potential health effects of a policy, programme or project on a population, and the distribution of those effects. Progress has been made in incorporating HIA into routine practice, especially (in the UK) at local level. However, these advances have mainly been restricted to process issues, including policy engagement and community involvement, while the evidence base has been relatively neglected.

Relating policies to their impact on health: The key distinctive feature of HIA is that determinants of health are not taken as given, but rather as factors that themselves have determinants. Nine ways are distinguished in which evidence on health and its determinants can be related to policy, and examples are given from the literature. The most complete of these is an analysis of health effects in the context of a comparison of options. A simple model, the policy/risk assessment model (PRAM), is introduced as a framework that relates changes in levels of exposures or other risk factors to changes in health status. This approach allows a distinction to be made between the technical process of HIA and the political process of decision making, which involves lines of accountability. Extension of the PRAM model to complex policy areas and its adaptation to non-quantitative examples are discussed.

Issues for the future: A sound evidence base is essential to the long term reputation of HIA. Research gaps are discussed, especially the need for evidence connecting policy options with changes in determinants of health. It is proposed that policy options could be considered as “exposure” variables in research. The methodology needs to be developed in the course of work on specific issues, concentrated in policy areas that are relatively tractable.

Conclusions: A system of coordination needs to be established, at national or supranational level, building on existing initiatives. The framework suggested in this paper can be used to collate and evaluate what is already known, both to identify gaps where research is required and to enable an informed judgement to be made about the potential health impacts of policy options. These judgements should be made widely available for policy makers and for those undertaking health impact assessment.

Health has become prominent in recent years as a focus for public debate, not only in relation to personal risk behaviour and medical care, but also as an outcome of a range of types of policy. In particular, air pollution and food “scares” including BSE have become major political issues, and other aspects of health are frequently in the news. More broadly, the need for Health Impact Assessment (HIA) has been acknowledged by a succession of official documents, including the white paper on public health in England,1 the Acheson report on social inequalities in health2 and the London Health Strategy.3 At the supranational level, the Amsterdam Treaty of the European Union states that “A high level of human health protection shall be ensured in the definition and implementation of all Community policies and activities.”

HIA has been defined as “a combination of procedures, methods and tools by which a policy, program or project may be judged as to its potential effects on the health of a population, and the distribution of those effects within the population”.4 HIA is usually suggested for policy areas other than health services, for example transport, housing or social inequalities, because health has tended to be neglected in policy development. It is applicable to policies in these areas whether they are motivated by the desire to improve or protect health, such as traffic calming schemes, or for other purposes. HIA can also be applied in the context of health services, to assess the result of policies or the contribution of different components of care to a change in health status. Although HIA has largely been applied to policies that primarily affect the public sector, the same approach is equally applicable to private sector activity.5 In all cases, a broad range of effects needs to be examined, including undesirable and/or unintended consequences.

The idea has much in common with social or environmental impact assessment; in the latter case, it is more similar to Strategic Environmental Assessment, which examines policies, programmes and plans, than to project-based Environmental Impact Assessment, which is longer established.

Progress in implementing HIA has been made in the past few years, and a number of guidelines6–9 and reviews10–12 have become available. Much of the recent activity has been aimed at ensuring the incorporation of HIA into policy making at the local level. Some success has been achieved in influencing projects to mitigate their potential harm, for instance the additional runway at Manchester Airport13 and numerous examples in the developing world.14 In the Netherlands, screening of new policies for possible effects on health has been introduced at the national level.15

These advances have primarily focused on improving policy engagement and involvement of both the community covered by the proposal and of other key stakeholders—that is, on the process, using existing knowledge. However, it is generally acknowledged that there are serious gaps in the evidence base required to carry out a rigorous HIA. Already, the lack of good and complete information is a serious limitation on HIAs. In the long run, if HIAs are to be effective, they will need reliable evidence that covers all aspects of the work. At present, such information is patchy.

This paper explores the different ways of relating policies to their health impacts. We then consider the relatively neglected question, what types of evidence are required for carrying out a rigorous HIA? Other issues that relate to the quality and effectiveness of HIA are not discussed in this paper. We place particular emphasis on the need for a conceptual structure that spans the gap between policy options and changes in health status, so as to identify the classes of information that are typically lacking. It is essential to be proactive in specifying what types of data are needed, and then explore the adequacy and quality of what is available, rather than to adopt the passive approach of simply carrying out literature searches.

RELATING POLICIES TO THEIR IMPACTS ON HEALTH

The key feature of HIA that differentiates it from pure epidemiology is that risk factors, exposures or determinants are not just taken as given, but are considered in the context of their own underlying causes. These could be called the determinants of the determinants. This simple two stage model is illustrated in figure 1A. More complex frameworks have been suggested, such as the DPSEEA model (in the context of environmental health), which distinguishes three stages of antecedents of exposure: driving forces, pressure, and state.16

Figure 1

(A) The determinants of determinants; (B) altering the determinants of determinants.

Epidemiologists commonly distinguish fixed risk factors, such as age, sex and genetic inheritance, from those that are alterable. The same idea can be applied to the determinants of determinants, namely changes in the socioeconomic and cultural factors that influence the risk factors. Different policy options have different effects on these socioeconomic and cultural factors. This is illustrated in figure 1B, which can be thought of as the “difference” version of figure 1A.

There are many ways in which evidence can be used when engaging with policy making. Figure 1B can be used to situate several distinct types of work on health and its determinants in relation to policy (table 1). It is not a question of “which of these are really HIA”, but rather, to realise that each has a place in piecing together the chain of causation between policy and health outcomes. They may therefore include useful evidence for HIA, as long as the information is based on good research.

Table 1

Examples of work on health and its determinants in relation to policy

Located entirely in the top half of figure 1, (a) is analysis of policy, whereas HIA sets out to perform analysis for policy. For it to form part of HIA, explicit links would need to be made with the health consequences. Similarly, needs assessment (b) and calculation of the burden of disease (c) are located entirely in the bottom half of figure 1, and lack explicit links with the determinants of the determinants.

Much information already exists within epidemiology on the risks attributable to specific exposures. In particular, (c) is an implicit comparison with the no-exposure situation, and may be an important component in an HIA, as it goes beyond descriptive epidemiology to map out the scale of an existing health problem, sometimes referred to as the “burden of disease.”21,39 However, the total health damage attributable to a risk factor is different from the health gain achievable by a change in the level of the risk factor resulting from a policy intervention (Mindell and Joffe, submitted for publication), unless the proposal is to remove the exposure altogether.

Social context (d) and advocacy (e) do not involve explicit analysis covering the whole of the diagram. The extended epidemiology or social context model (d) could connect with the perspective of figure 1A or figure 1B, depending on whether or not it focuses primarily on alterable factors, but it lacks an explicit link back to the process of alteration that is the policy process. The advocacy approach (e) involves an analysis in the bottom half, and then looks towards the upper half but without an explicit consideration of the range of consequences of policy options other than on the original topic that was analysed. It may take a completely apolitical form, as in the attempt to ensure that the findings of research are translated into practice.

In contrast, (f) through (i) all encompass both health and policy but differ in the way that this is done. (f) Does not deal directly with policy options but with the health effects of achieving a given target; without considering how the target is to be accomplished, its link to policy is incomplete. Both (g) and (h) are limited to assessing the health effects of a single option, the first passively, the second with the deliberate intent to influence implementation so as to protect health. (i) Compares the health effects of a number of options, giving them the status of antecedent variables in a research study.

Type (i) has several advantages. By comparing options, it gives decision makers the most explicit information on the health consequences of their actions. The approach is routinely used in economic evaluation, in which different scenarios are modelled and their effects compared. Furthermore, it is simple to integrate with the analysis of policy outcomes other than health. This is important, as policies have multiple effects. Whereas advocacy (e) can be limited by the difficulty of systematically integrating health advocacy with the possible non-health outcomes of policy (intentional and unintentional), in the case of (i) the non-health outcomes can be subjected to a parallel assessment.

In addition, type (i) lends itself readily to an appropriate division of labour between the technical work of HIA and the political processes of policy development and decision making. It is important for HIA not to blur the line between the technical information on the probable consequences of particular decisions, and the policy process that entails taking into account underlying values, many different types of outcome, trade offs between positive and negative aspects, and lines of accountability.

WHAT TYPES OF EVIDENCE ARE REQUIRED?

A useful approach is based on the standard risk assessment model (fig 2A): three elements are combined to generate the assessment of the existing risk. Firstly, the types of health effect that a particular exposure can cause (hazards) are identified. Secondly, the “dose-response” relation quantifies this: for a given level of exposure, a certain effect (or probability of an effect) will result. In practice, this is seldom a dose in the familiar sense, which applies to an individual, but is an ambient level to which the population is exposed. These are derived from the scientific literature in the fields of epidemiology and/or toxicology. The third element is a description of current levels of exposure. Together they permit an estimate of the magnitude of the health risk from the current exposure.

Figure 2

(A) The four elements of risk assessment; (B) PRAM model.

Comparison of policy options

The risk assessment model can readily be extended back to policy. Its first two elements, hazard identification and dose-response relation, are usually considered to be constant for a given population, although they may vary between individuals in relation to age, sex, disease status, genetic differences or nutritional status. In contrast, the actual exposure level varies by time and place. Crucially, it is subject to alteration in response to different policy options. This suggests the “Policy/Risk Assessment Model” (PRAM, fig 2B).31 Itsimplications are discussed further below. Both the possibility of differential susceptibility and changes in exposure of subpopulations at higher risk and/or with lower resources are important factors to include in HIA, to assess impacts on health inequalities.

Operationalising the PRAM model

An example

It is possible to model the health gain consequent on reducing ambient air pollution. A large body of evidence now exists from time series studies, showing that acute changes in particulate matter concentration are followed by changes in total mortality and by changes in mortality and morbidity from respiratory and cardiovascular conditions (hazard identification).40 The next step is to make a judgement whether the association is causal, which is a necessary condition for proceeding further. In this instance, the key uncertainty concerns so called “harvesting”, that while particulate air pollution may bring deaths forward, it is uncertain whether it shortens life by a matter of days, weeks or longer41 or whether there is an effect on annual mortality.42–46

To put the PRAM model into practice, the size and age structure of the population is combined with estimates of the exposure-response relation, derived from regression estimates from the relevant studies, and exposure levels before and after a policy intervention. The simplest method is to compare existing PM10 concentrations, derived from monitoring data, with the targets required by the UK National Air Quality Strategy,47 which is an example of category (f), health effects of attaining targets, in the above schema. It is also possible to estimate the effects on ambient concentrations of policies that reduce emissions, by modelling the dispersion of the particulate matter. By linking the two types of models a complete model can be constructed, an example of category (i), from comparing policy options through to a range of consequent health outcomes.

Extension of the PRAM concept beyond chemical exposures

The PRAM model is intended to provide a more general structure than its origin in risk assessment may suggest. The term “exposure” can be extended beyond the simple example of a chemical agent. For example, traffic calming measures are intended to impact on health, and it is possible to compare the injury rates with and without them, and between different proposals. Similarly, unemployment has consequences for health and could be regarded as an “exposure” for the purposes of an HIA. Other examples include the price of tobacco, which influences its consumption, the price and availability of fruit and vegetables, and the use of tamoxifen to treat breast cancer (table 2).

Table 2

Examples of determinants of determinants

A key issue is how uncertainty is handled. This is a major topic in its own right, which has had entire books devoted to it (for example, Morgan and Henrion48—see table 3). Here we deal with only two related aspects that are closely connected with the theme of the paper. Firstly, it is essential to be explicit about assumptions, and to identify missing or incomplete information. Secondly, where information is available, a typical situation is that no single consensus view is available. For example, there are several estimates of the association between ambient PM10 concentration and mortality from time series studies.40,49–51 The clearest way to present such uncertainty is through sensitivity analyses, exploring the effects of the different possible assumptions and estimates. At the policy level, the Precautionary Principle may be appropriate.52 In the longer term, well targeted research is necessary to generate the necessary data.

Table 3

Ten “commandments” for good policy analysis (taken from Morgan and Henrion, 199048)

For broader policy areas, such as the health implications of road transport or of social exclusion, a series of chains of events needs to be considered. It is useful to develop a “map” linking the possible causes and effects within this overall policy area (fig 3). Each of the component routes corresponding to each type of “exposure” then needs to be investigated, using the PRAM framework. The organising principle is to identify the possible policy interventions at one end, analogous to the independent variables in aetiological research, and the health outcome variables at the other.

Figure 3

Diagram of pathways from transport policy to health outcomes.

Such a diagram is useful as a reminder that policies typically have a range of effects, on health53 and also on the economy, the environment, etc. In drawing one up, it is important that each pathway is independent of the others to avoid double counting. There are two types of pitfall. The easier problem occurs when one exposure results in two or more outcomes. For example, Ostro and Chestnut are careful to exclude from “the number of people admitted to hospital” those already counted as “deaths” avoided by reducing particulate levels.29 More difficult is the issue of two or more concurrent exposures leading to the same outcome, as with particulate matter and nitrogen dioxide. In such a case the number of people affected by each exposure should not be summed. For example, Künzli et al use PM10 as the sole indicator of traffic related air pollution, and their estimate is therefore a lower bound.54

The foregoing discussion has been in terms of quantification. However, HIA should not be distorted by exclusion of important but unquantifiable determinants. For example, the impact of traffic policy on the whole pattern of everyday life is far reaching. To take just one age group, children now predominantly spend their time indoors and are far more dependent on their parents for mobility compared with a generation ago,55,56 partly because of dangers from traffic.

The same conceptual structure is equally applicable for pathways where measurement is impossible. In such cases quantitative analysis could be replaced by rigorous descriptive evaluation based on qualitative research. The aim would be to describe the effects of change, in an analogous way to the PRAM model in figure 2B. A descriptive HIA based on robust qualitative evidence is preferable to unsound quantification.

ISSUES FOR THE FUTURE

The introduction of HIA has had a positive effect, in that potential impacts on health are beginning to be considered systematically in a variety of policy areas and by decision makers and professionals who may not previously have connected the effects of their work with health. However, the quality of the evidence base currently available for HIA may limit the soundness and completeness of the conclusions, and consequently the capacity to achieve health gain. Thus, the strong policy drive to encourage the performance of HIA at both a national and a local level may have unintended consequences that are counterproductive: if HIA comes to be perceived as ineffective, and therefore wasteful of resources, it will gain a poor reputation and in the longer term will no longer command support.

It is vital, therefore, that the aim should be to conduct HIA to a high standard in order to obtain the maximum possible level of health gain. This aim cannot be realised without a robust evidence base. It is clear from the foregoing discussion that an HIA of the necessary scope and quality requires the information gaps to be filled. Without this, a substantial reorientation of policy in the interests of health will not be feasible.

In particular, we typically lack evidence on links between policy options and health determinants. There are notable exceptions: for example, it has been possible to combine an analysis of the relation between traffic speed and the risk of accident frequency, death and serious injury (exposure-response relation between risk factor and health outcome) with an examination of the contributions that could be made by changes in infrastructure, legislation on speed limits, technological aids, and publicity and education (policy intervention leading to change in exposure).57 Generally, however, there is a paucity of evidence concerning the links in the top half of figure 1B. For example, large scale traffic reduction schemes are relatively untried and their potential effects on a range of outcomes, while potentially large, are imprecise.

At first sight, the suggestion that policy options could be taken as “exposure” variables in research may seem strange. It is certainly unfamiliar. However, there is no reason why it is less rigorous or feasible to study the health effects of policy makers' decisions than it is to study the behavioural determinants of health at the individual level. Epidemiologists could work with policy analysts just as they work with, for example, nutritionists. Undertaking research along these lines will necessitate the cumulative development of expertise.

Division of labour

It is not suggested that each HIA should involve work of this scope and depth. Indeed, the converse is the case for people undertaking a local level HIA or rapid appraisal: they need to have a body of evidence that they can call on, knowing that it is well founded. We are therefore proposing a division of labour between the decentralised activity of carrying out an HIA and the centralised or generalisable activity of providing the evidence for it. Such uncoupling has recently also been suggested by other authors.58

Those carrying out HIAs would be experts on the populations covered, including for example the age distribution, social inequalities and groups with particular needs or vulnerabilities, as well as on the range of relevant exposures. Consideration would also be given to the key stake holders who would implement the policy and who would be affected by it; the policy options that are likely to be considered; the agencies that should be involved, and the prevailing value systems. In conducting the HIA, they would need to rely on the availability of the whole range of information on technical aspects, generated by research of academic quality. In many cases, the information they require will be similar to HIA related work in other places, and for frequently repeated topics it can be made available “off the shelf”. Among other things, this would have the advantage of speeding up the process, which is important as HIA cannot be done until proposals are concrete enough to be evaluated but the assessment needs to be available in good time so that it can influence decision makers' consideration of different options.

Practical issues

Providing a comprehensive review of the evidence base is a far from trivial exercise. It needs to use the best available evidence, bringing together authoritative reviews (where available) and research papers from a variety of disciplines. It also needs to include qualitative research, and evidence from the specific policy areas such as transport or fiscal policy as well as in the health sciences. While electronic databases and the internet are valuable resources, in practice they are incomplete, and need to be supplemented by expert knowledge and personal contacts. A particular problem is that many types of work, for example, local authority projects, are not widely known, and even if one knows of their existence it is often difficult to access the appropriate document. Judging the quality of evidence of the different types is a further important issue,59,60 which is beyond the scope of this paper.

In developing this generalisable work, it is probably wise to prioritise certain policy areas that are more tractable until the methodology is better developed. The key criteria would be that they are likely to have large health impacts, are scientifically feasible, and that there is the political will to allow the analysis to have a real influence on the outcome. Suitable areas include transport, housing, and nutrition.

There is now growing activity in a number of areas that will contribute to the HIA evidence base. For example, the Campbell Collaboration, on the lines of the Cochrane Collaboration, is collating and evaluating high-quality field trials of social, behavioural and education policies, which will include a great deal that is relevant to health.61 The UK Evidence Based Policy Centre (funded by the Economic and Social Research Council) has a node on Evidence-Based Public Health Policy.62

In assembling the different types of information required to investigate the chain of causation between policies and health outcomes, we suggest that the framework put forward in this paper can be used to collate, evaluate, and organise what is known. This makes clear what gaps exist, where research is required. More practically, it enables an informed judgement to be made about the potential health impacts of policy options. These judgements should be made widely available for policy makers and for those undertaking health impact assessment.

Key points

  • Health impact assessment (HIA) is being encouraged but work has concentrated on process not on the rigour of the evidence base.

  • We describe and illustrate nine ways in which evidence on health and its determinants can engage with policy making.

  • We suggest a framework to analyse changes in health status consequent on each policy option.

  • Constructing a sound evidence base will require considerable research effort, including methodological development.

  • The information will need to be collated, evaluated and disseminated for the use of policy makers and those undertaking HIA.

Acknowledgments

The ideas expressed in these paper were developed in the course of a project funded by the European Commission, on the health impact of internal market legislation. An earlier draft was prepared for a seminar organised by the Department of Health. We would like to thank David Briggs, Rainer Fehr, Anna Hansell, Erica Ison and Mark Nieuwenhuijsen for helpful comments on earlier versions.

REFERENCES