REVIEW OF PIN AND DIN

The NNT describes the impact of a given intervention on the population of patients who are treated with the intervention (using the mean baseline risk for the patients eligible for the intervention).

whereFigure 1 shows the relation between the impact numbers and NNT: those with events prevented by intervention are clustered within the population eligible for intervention, who in turn are clustered within the population with disease, who in turn are clustered within the total population, and the NNT, DIN, and PIN are shown as they relate to each of these populations. The NNT is a fraction of the DIN and the DIN may be expressed aswhere P_e is the proportion of the diseased population eligible for the intervention.

Similarly, the DIN is a fraction of the PIN and the PIN may be expressed aswhere P_d is the proportion of the population with the disease of interest. (See appendix for more detail of derivation of these measures.)

Like the NNT, both DIN and PIN should be qualified by a time frame.

AN EXAMPLE OF THE APPLICATION OF DIN AND PIN TO HEALTH POLICY

A committee has the task of recommending which treatments for congestive heart failure (CHF) should be included in the Pharmaceutical Benefits Scheme (a scheme where government subsidies are provided for effective interventions, as happens in Australia). We illustrate the use of PIN and DIN in assisting this decision by comparing the population impact of use of β blockers, ACE inhibitors, and spironolactone on one year mortality in people with CHF.

GENERAL CONSIDERATIONS IN THE APPROPRIATE USE OF PIN AND DIN

Choice of studies for estimates of risk reduction

The first step in the calculation of the impact numbers is to choose well conducted studies that answer the question of interest. It is preferable to choose good quality randomised controlled trials (RCT) or meta-analyses to obtain estimates of relative risk. We use data from the MERIT trial to provide an estimate of the effect of β blockers in patients with CHF.⁷ Data for ACE inhibitors are taken from the SOLVD trial,⁸ and data for spironolactone are from the RAES trial.⁹

Choice of follow up time

The MERIT trial results were reported for a median follow up of one year, the follow up period for SOLVD was about 41 months, and for RAES was 24 months. If we assume that the proportional hazards model is correct, we can compare the relative risk from the three studies despite the different follow up periods. However, we need to choose the baseline risk that is relevant for the period of interest; for this example, we chose 12 months.

Choice of outcome

Another important consideration is that PIN and DIN should be calculated for the same outcome (endpoint) in the trials. Apart from all cause mortality, the trials for CHF reported other endpoints such as cardiovascular mortality, sudden death, and worsening heart failure. For this example, we are interested in all cause mortality.

Choice of relevant comparison groups

It is also important that the RCTs and meta-analyses have control or comparison groups relevant to the application for PIN and DIN. Although there are other ACE inhibitor trials, the comparison groups differ; for example, in the VHeFT II trial,¹⁰ ACE inhibitors were compared with the combination of hydralazine and nitrates. To make valid comparisons, all three trials of treatments for CHF selected for this example had placebo control groups.

Calculation of the proportion of patients eligible for the intervention

For ease of communication we use percentages in our discussion of trials, but the percentages need to be converted to proportions for the actual calculations.

The proportions of patients with CHF who were eligible for treatment were remarkably similar across all three treatments. To calculate the proportion of patients who are ineligible for treatment with β blockers, we have taken the prevalence of reactive airway disease in the general population as 10%¹¹ and added those who were unable to tolerate treatment in the trial (15%). This gives a total of 25% not eligible for β blockers. For ACE inhibitors, the SOLVD study reported that 11 of those screened had contraindications to ACE inhibitors, and 14% discontinued treatment because of side effects, giving a total of 25% who did not receive the treatment. For spironolactone, a MEDLINE search reveals a cohort study documenting that 17% of CHF patients discharged from hospital have renal insufficiency (defined as creatinine >2 mg/dl)—a major contraindication to spironolactone.¹² In addition, 10% of patients in the RAES trial could not tolerate the treatment, primarily because of gynecomastia, giving a total of 27% who would not be eligible to benefit from this treatment.

Incidence or prevalence

To calculate the PIN, we need to obtain P_d, the proportion of the total population with the disease of interest. For this example we expressed this proportion as a prevalence rather than an incidence proportion, because the three treatments we considered could be applied to both new cases of CHF (incident cases) as well as old (prevalent) cases. If we were calculating the PIN for a treatment that was only applicable in the acute phase of an illness, for example, intravenous corticosteroid for episodes of acute severe asthma, then we would have to express P_d as an incidence proportion. The choice of prevalence versus incidence proportion is important; for example the annual incidence proportion of heart failure in the Framingham study is one fifth of the prevalence (about 1.5 and about 7.5 per 1000 respectively¹³).

To calculate the PIN, we have used the prevalence of CHF in the Hunter Valley of Australia, 42 per 10 000 (obtained from a population based register of hospitalised patients)¹⁴ as an estimate for the Australian population.

With these caveats in mind, on the basis of evidence from the RCTs, it seems that spironolactone has the highest potential population impact (table 1). If spironolactone were systematically prescribed to eligible patients with CHF who were able to tolerate it and had mean baseline risks similar to those in the RCT, one death would be averted per year for every 4700 people in the population at large, compared with 8500 if β blockers or 17 300 if ACE inhibitors were used. However, this could be misleading; as with the NNT, the DIN and PIN depend on the baseline risk. We note that baseline risk is much higher in the RAES trial (23%) than in the other two trials (11%, 11.5%); this is because the MERIT and SOLVD trials enrolled patients with grade II and III heart failure, whereas RAES enrolled patients with more severe heart failure (grade IV). If spironolactone were also appropriate for patients with less severe heart failure, we could halve the baseline risk (from 23% to 11.5%) to reflect the same population risk as in the other trials; this would yield a PIN for spironolactone of 9500, so that β blockers would seem to offer greater benefit.

ISSUES SPECIFICALLY RELEVANT TO HEALTH POLICY DECISIONS

There are many steps from reading the results of a clinical trial to assessing how useful an intervention will be in practice, as pointed out by Smeeth and Ebrahim.¹⁵ In this section we describe the methodological questions that face a policy maker in this context.

Choice of overall target population

To be appropriate, we need to calculate estimates that are specific to the local situation. In many cases, the estimates of relative risk reduction from clinical trials are fairly stable across various categories of baseline risk and are thus more generalisable than estimates of absolute risk.¹⁶ We can use the estimates of relative risk from RCTs or meta-analyses, and combine this with the measures of baseline risk for the local population to calculate impact numbers for that setting.

In this example, we wish to compare treatments for CHF in the Australian population. Although the eligibility criteria for the trials included estimates of systolic ejection fraction, these measurements are not routinely done on Australian patients. Admission to hospital with CHF is the primary means of diagnosis. Thus, it may be appropriate to use estimates based on this population.

Use of local data

Data from the Hunter Area Heart and Stroke Register¹⁴ in Australia show that the risk of death during the next year among patients admitted to hospital with CHF is 29%. Using this figure for baseline risk with estimates of the potential proportion of patients eligible (P_e) and prevalence of the disease CHF (P_d), we obtain the results in table 2.

It would seem that spironolactone and β blockers have potentially higher population impact than ACE inhibitors in CHF in the Australian population.

Total versus incremental impact

In this example, the PIN and DIN reflect the total impact of the intervention. We have not taken into account the proportion of the population already taking β blockers, ACE inhibitors, or spironolactone. Health officials may be more interested in the incremental impact of encouraging one or another treatment. Again using data from the Hunter Area Heart and Stroke Register for the first six months of 2000 (unpublished), the prevalence of β blocker, ACE inhibitor, and spironolactone use in patients with CHF was 24%, 59%, and 20% respectively. Thus, there are about 51%, 16%, and 53% of patients who are additionally eligible for β blockers, ACE inhibitors, and spironolactone respectively who are not currently receiving them (the proportion eligible minus the proportion who are currently using the treatment). If we modify the population eligible to reflect these incremental values, we obtain table 3.

As ACE inhibitors are currently used more frequently than the other two drugs, the incremental impact of either β blockers or spironolactone is much greater than that of ACE inhibitors. Thus increasing the prescription of β blockers or spironolactone in CHF patients would have more impact than increasing ACE inhibitors.

Choice of specific target population

In the same way that age or gender specific rates of a disease may be calculated, one may wish to obtain an age or gender specific impact number. For example, one may wish to calculate the PIN for β blockers as a treatment for CHF in adults aged less than 70 years separately to one for those aged 70 to 84 years. To do this, we need to change the baseline risk, the proportion eligible (P_e) and the proportion with disease (P_d) to reflect the new populations. From the Hunter Area Heart and Stroke Register, the baseline risk of death within one year in CHF patients aged 20–69 years and those aged 70–84 years is 21% and 31% respectively. Additionally, older patients have a higher risk of reactive airway disease, cardiac conduction defects, and severe heart failure that would result in fewer being able to tolerate treatment with β blockers. Thus the approximate proportions eligible for treatment (P_e) are 80% and 65% respectively. The proportion of the populations aged 20–69 years and 70–84 years with CHF in the Hunter area are 19 and 362 per 10 000. The DIN and PIN for β blockers in patients with CHF are shown in table 4. β blockers have much greater potential impact in the elderly population.

Similarly, if one were comparing interventions for childhood diseases, such as vaccinations, one might be interested in determining age specific impact numbers.

Different types of interventions

The PIN and DIN may also be used to compare the impact of widely varying interventions on the same outcome. For example, one may want to compare the impact of screening mammography on mortality from breast cancer in women aged 40–49 years and screening faecal occult blood testing on mortality from colorectal cancer in people aged 40–49 years. Estimates of the relative risk reduction associated with these screening tests were obtained from two recent meta-analyses of randomised clinical trials.^17,18 Australian data on cancer mortality were obtained from the Australian Institute of Health and Welfare.¹⁹ In this case, the population eligible for intervention are all those women aged 40–49 years and everyone aged 40–49 years respectively. The “proportion with disease” are the proportions of the population who are in those categories. The estimates are shown in table 5.

As can be seen, the potential impact of screening mammography on breast cancer mortality in women aged 40–49 years is four times that of faecal occult blood testing on colorectal cancer mortality in all individuals in the age group 40–49 years. The population impact of mammography screening is roughly twice that of faecal occult blood testing, and both screening programmes have much higher impact numbers—that is, lower impact—than the secondary prevention drug treatment for congestive heart failure shown above.

Bias, sampling, and the role of sensitivity analyses

The impact numbers are derived from estimates of relative risk, the baseline risk in the relevant population and the proportions P_e and P_d . Therefore they are subject to bias and sampling error.^15,20 Furthermore, as they are reciprocals of numbers that may be very small, or even negative (if a treatment is harmful), they have rather poor statistical properties and confidence intervals may be difficult to interpret.^21,22 They are also based on many assumptions. To obtain robust numbers to aid health policy decisions, sensitivity analyses may be more useful than estimating confidence intervals.

A range of plausible values of baseline risk, proportions eligible for the interventions, and prevalence of disease should be used to estimate the reciprocals of NNT, PIN, and DIN. If the estimates are positive and not very small they can be inverted to give interpretable impact numbers for a variety of scenarios (if they are very small or negative, caution is needed for the interpretation of impact numbers). These calculations are especially important as local data may not be readily available and when available may be subject to significant error.

CONCLUSIONS

We have given examples of how DINs and PINs may assist health policy decisions, and have outlined some caveats in their use.

There is no doubt that in some cases it will be difficult to find the relevant local data to estimate the DINs and PINs. The data required for these calculations are more likely to be found in government reports, local area health statistics, health resource utilisation reports, etc, rather than in MEDLINE quoted articles. As the information from these sources is often not subject to the same peer review as journal articles it will be more difficult to judge its validity. When using suspect data, or if using these measures when local data are not available, we suggest using sensitivity analysis. Developing an evidence base for these measures to support public health decisions highlights the need to systematically collect local morbidity and mortality data of good quality.²³ The success of this approach will depend on information being systematically collected at a local area level, perhaps using disease registries, such as the Hunter Heart and Stroke Register from which we have quoted both published and unpublished data in this paper.

One may wish to compare the population impact of different outcomes for different therapies, for example, thrombolysis for reducing disability in patients with stroke versus β blockers for reducing mortality in patients with CHF. To do this one would have to include utilities to attach relative values for disability and death. Utilities may also be useful in assessing impact on different subgroups of the population, by allowing considerations of productive years of life lost, or disability adjusted life years (DALYs) to be included in the calculations. Likewise, in the example of the three therapies for heart failure, one may wish to take into account the impact obtained per unit cost for the treatment.

The development of these new measures is still in its early stages but we hope that they will assist health professionals to make more rational and more evidence based decisions in public health.²³

APPENDIX

Algebraically, the NNT is defined as the reciprocal of the absolute risk reduction (ARR):

The ARR is defined as the difference in risk between the treatment and the control group (in the epidemiological literature, this is also known as the attributable risk (AR)):where I_e = incidence of outcome in the exposed (or treated) group, I_u = incidence of the outcome in the control group. However, as the relative risk reduction (RRR) [=1-RR or (1-(I_e/I_u)] (RR = I_e/I_u ) is usually reasonably constant across trials the absolute risk reduction is dependent on the baseline risk, and it is more appropriate to calculate ARR taking baseline risk into account.

In extending the NNT to the population level, we define the new measure to be the reciprocal of the absolute risk reduction (attributable risk) at a population level. A measure of the attributable risk in a population is the population attributable risk (PAR), calculated as the attributable risk times the proportion of the population exposed to that particular exposure or treatment (P_e)* †:This proportion may be calculated in two ways:

1. the proportion of people with the disease who are exposed, or eligible, for the treatment,

2. the proportion of people in the total community who are exposed, or eligible, for the treatment.

Each of these two options gives us a different population perspective on the NNT.

We define the disease impact number (DIN) as “the number of those with the disease in question amongst whom one event will be prevented by the intervention”. This measure takes into account the number of people in the population with the disease, not just those eligible for the intervention according to the entry criteria for the RCT. Algebraically, it is expressed as:where P_e is the proportion of the diseased population exposed to or eligible for the intervention. This reduces to:orWe define the population impact number (PIN) as “the number of those in the whole population amongst whom one event will be prevented by the intervention”. This measure takes into account the whole population from which the patients with disease are drawn. Algebraically, it is expressed as:where P_p is the proportion of the whole population exposed to or eligible for the intervention. We can further define P_p as:where P_e is the previously defined proportion of the diseased population exposed to or eligible for the intervention and P_d is the proportion of the whole population with the disease of interest. Hence PIN can be defined as:which reduces to:or