Article Text
PDF
Abstract
Background Estimation of influenza vaccine effectiveness (V/E) is needed early during influenza outbreaks in order to optimise management of influenza—a need which will be even greater in a pandemic situation.
Objective Examine the potential of routinely collected virological surveillance data to generate estimates of V/E in real-time during winter seasons.
Methods Integrated clinical and virological community influenza surveillance data collected over three winters 2004/5–2006/7 were used. We calculated the odds of vaccination in persons that were influenza-virus-positive and the odds in those that were negative and provided a crude estimate of V/E. Logistic regression was used to obtain V/E estimates adjusted for confounding variables such as age.
Results Multivariable analysis suggested that adjustments to the crude V/E estimate were necessary for patient age and month of sampling. The annual adjusted V/E was 2005/6, 67% (95% CI 41% to 82%); 2006/7 55% (26% to 73%) and 2007/8 67% (41% to 82%). The adjusted V/E in persons <65 years was 70% (57% to 78%) and 65 years and over 46% (−17% to 75%). Estimates differed by small insignificant amounts when calculated separately for influenza A and B; by interval between illness onset and swab sample; by analysis for the period November to January in each year compared with February to April and according to viral load.
Conclusion We have demonstrated the potential of using routine virological and clinical surveillance data to provide estimates of V/E early in season and conclude that it is feasible to introduce this approach to V/E measurement into evaluation of national influenza vaccination programs.
- Influenza virus
- vaccine effectiveness
- surveillance
- vaccination
Statistics from Altmetric.com
Knowledge of the effectiveness of healthcare interventions such as immunisation is essential for evaluating their public health value. For vaccines, effectiveness (V/E) is usually defined as the direct protection afforded to an individual under conditions of routine use. For most vaccines, V/E is not expected to vary from year to year, but this is not the case for influenza vaccination, because of the ever-changing character of influenza viruses and necessity for annual update of the components of the vaccine. V/E calculated in 1 year does not hold good for the next. A routinely applied program is, therefore, needed to evaluate annually the effectiveness of influenza vaccine against that year's circulating strains. Early information on the efficacy of the vaccine is of particular importance when faced with an influenza epidemic caused by a virus showing resistance to anti-viral drug treatment, as was seen in 2007/8.1 2 An ideal program would provide timely data to inform influenza management policy, a need that would be even greater in the event of deployment of novel vaccines in a pandemic situation. The ability to estimate V/E in routine healthcare statistical data has been demonstrated using both laboratory and consultation endpoints.3 To do this in “real-time” has logistic difficulties in ensuring timely data capture of all relevant data including the results of virological investigation and evaluation 2–4 weeks in arrears is more realistic.
The measurement and interpretation of V/E in post-marketing observational studies are complicated by numerous confounding variables including case selection bias.4 In particular, bias towards vaccinating healthy seniors,5 those with high risk morbidity,6 7 those with a poor prognosis,6 or the frail elderly,8 those with high levels of healthcare usage;9 10 or bias against vaccinating the terminally ill,11 those vaccinated relatively late in the season,12 the functionally impaired,13 according to the definition of outcome time period, whether entire winter or restricted influenza-active period.14 Because of the potential for bias, observational studies of V/E using the conventional case control or cohort approach require data on clinical confounders to be systematically collected, but even so, there remain confounders such as the likelihood of presenting for medical advice or the likelihood of the doctor taking a swab, which are difficult to estimate in a routine healthcare setting.7 15–18 The preferred method of analysis has involved the use of logistic regression. The difficulties of controlling for all potential confounders limit the accuracy of estimating V/E from service healthcare.8 Moreover, estimates of V/E are specific to the endpoint considered, in particular whether it includes virological confirmation. Use of the less specific clinical case definition of influenza-like illness (ILI) will necessarily produce lower V/E estimates since not all ILI cases will be attributable to influenza infection.
Since 1992, an integrated program of clinical and virological surveillance of influenza has run in England.19 General practitioners (GPs) participating in the clinical surveillance network known as the Royal College of General Practitioners (RCGP) Weekly Returns Service (WRS)20 take swabs from patients presenting with ILI during the influenza season. Over 1000 virological swab samples are submitted by post each season to the Health Protection Agency (HPA) Centre for Infections, London, together with a request form giving vaccination status and relevant demographic and clinical information. The samples are investigated for influenza viruses (including subtype and viral load) and other respiratory viruses.21 22 We aimed to estimate V/E using these virological results and employing a method designed to avoid confounding by indication for vaccination and appropriate to intra-seasonal estimation of V/E.
Methods
Study population
The study population comprised patients presenting in participating practices with an acute febrile respiratory illness considered by the GP to be an ILI over three influenza seasons October to March 2005/6 to 2007/8. There were 41, 44 and 46 practices involved in the three winter seasons, respectively: the registered populations increased over the period from 364 375 to 482 806. Patients presenting (whether consulting at the practice or visited at home) were asked if they were willing to provide swabs from the throat and nose. Clinical information from the consenting patients was summarised on the investigation request form and included information on age, sex, date of swab, duration of illness, whether the patient had received that season's vaccine, dominant symptom group and respiratory syndrome diagnosis. Investigation requests were limited to a maximum of five per week per GP in accordance with laboratory capacity. Since the purpose of the swabbing was to investigate the virological cause of the patient's illness, ethical approval was not required.
Virological analysis
Nasopharyngeal swabs were stored in 2 ml of virus transport medium and posted to HPA; the median interval between taking the swab and its arrival in the laboratory was 2 days. Virological swabs were analysed according to previously published methods for influenza (type and subtype) and RSV using real time RT PCR.2 21 Briefly, multiplex quantitative real-time RT-PCR (qRT-PCR) was performed on nucleic acid extracted directly from clinical samples by automated extraction on a MagNaPure (Roche Applied Science) and eluted in a volume of 100 μl. Primers and probes targeting conserved regions of the HA gene of influenza H1, H3 and B viruses were used. qRT-PCR was carried out in a 25-μl reaction mixture with 7.5 μl extracted template, 12.5 μl 2× qRT-PCR Reaction mix (SuperScript III Platinum One-Step Quantitative RT-PCR kit, Invitrogen, Paisley, Scotland), 900 nM each H1 and H3 forward and reverse primers, 600 nM each B primer, 200 nM H3, 50 nM H1 and 250 nM B TaqMan minor groove binder probes, respectively, 0.5 μl SuperScript III RT/Platinum Taq mix and 0.1 μl ROX Reference Dye. Amplification, detection and data analysis were performed in duplicate on an ABI Prism 7500 real-time thermal cycler. Estimates of viral load were initially reported as adjusted cycle threshold (Ct) values. Ct values greater than 38 were considered negative for all winter seasons.
Statistical analysis
Information reported on the specimen form, together with the results of virological investigation, were entered on receipt of specimen into the HPA laboratory information Management system (LIMS MOLIS) database. Data were extracted, and analysis was performed in STATA V. 10.23 Influenza virus detection was the principal outcome measure. For estimating V/E, “positive for influenza detection by PCR” defined cases and “negative” defined controls. For analysis of viral load, PCR positives with a Ct value <30 or ≥30 were regarded as having a high or low load, respectively.
Crude estimates of V/E were made as one minus the odds of vaccination in influenza-positive persons compared to influenza-negative individuals. These data were then analysed as a case-control study using logistic regression. Variables considered in the analysis were year, age, sex, vaccination, interval from illness onset to specimen and month of specimen. These variables were included in a multivariable model, and their interactions and effect modification on vaccination examined to identify which variables should be taken into account when estimating effectiveness. Stratified analyses were also performed according to year, age (<65, ≥65), interval from onset to sample (≤4 , >4 days) and month of sample (Oct–Jan and Feb–April). Influenza-positive samples were also analysed by virus type (A/B) and by Ct value.
Results
Descriptive analysis
The reported clinical incidence of ILI (all ages per 100 000 population) and the numbers of swabs positive for influenza A or B by week in each of the three winters are shown in figure 1. Incidence peaked in weeks 01/2006, 07/2007 and 07/2008. Altogether, 3760 virological swab samples were investigated over this period from the denominator population of 1.3 million person years. There were 254 exclusions from analysis because of missing data; 32 (1%) did not have a laboratory result; 182(5%) had no vaccination status recorded and 40 (1%) no age data, leaving a total of 3506 (93%) for analysis. Some further exclusions were made for specific analyses: sex (72 exclusions); duration of illness when swabbed (417); viral load CT values (283 of the 963 influenza-confirmed cases, mainly in those from 2005/06 when viral load testing was first initiated).
Weekly incidence of influenza-like illness (ILI) per 100 000 and number of influenza-virus-positive cases over winters 2005/06, 2006/07 and 2007/08.
Of the 3506 samples with a virological result and known age and vaccination status (table 1), 22% were aged 0–14 years; 47%, 15–44 years; 24%, 45–64 years and 8%, ≥65 years. Over the three winter seasons and in the entire WRS population (includes population of non-swabbing practices), there were 11 408 patients reported with ILI: the equivalent distribution by age group was 13%, 51%, 27% and 9%, respectively. In the older age groups (45–64 and ≥65 years), which were more often vaccinated, the proportions of swabbed patients and those reported with ILI were similar. Among the total swabs taken, 963 (27%) were positive for influenza and 521 (15%) were from individuals vaccinated that season. Influenza virus detection varied by age, month, vaccination status and interval between onset of illness and swabbing, the detection rate declining after 4 days. The proportion of patients swabbed after 4 days was 26% and 23%, respectively, in 0–4- and 5–14-year-olds, increasing to 35%, 49% and 44%, respectively, in 15–44-, 45–64- and ≥65-year-olds. The proportion vaccinated among patients swabbed each month increased markedly from October to December, reflecting the increase in vaccine coverage at the start of the influenza season. The proportion also increased with age consistent with the universal vaccination recommendation for ≥65-year-olds and the selective recommendation for those with clinical risks in younger age groups.
Number of swabs submitted, number (No) and percentage (%) positive for influenza virus and number (%) vaccinated by year, month of sample, age, sex and interval to swabbing
In the 680 influenza-positive samples with viral quantitation data available, there was no significant difference in proportion of vaccinated individuals in those with a high viral load (Ct <30) compared with those with a low viral load (Ct >30), 8% vs 7%. Viral load was negatively correlated with interval from onset to swabbing, the proportion with a high viral load decreasing from 55% in those swabbed 0–1 days after onset, to 39% at 2–4 days and 22% after 4 days.
Influenza strain type and subtype varied by year (table 2) as did the vaccine strains and their relatedness to the circulating viruses. However, the proportion of influenza-positive individuals remained similar in each year (24–30%).
Influenza virus strains identified by year among patients presenting with influenza-like illness in the three influenza season 2005/6, 2006/7 and 2007/8
Vaccine effectiveness estimation
Using the combined data over the three seasons, the crude V/E estimate was 65% varying between 53% and 73% in the 3 years (table 3).
Percentage of confirmed influenza cases vaccinated, percentage of non-confirmed cases vaccinated and crude vaccine effectiveness (V/E) estimates with 95% CI by year and for 2005/6–2007/8 combined
In the multivariable analysis age, interval from onset to sample, month and vaccination status were all significantly associated with positivity. However, none of these variables had a significant interaction with vaccination. In the multivariable analysis, adjustment was made for age and month; adjustment for interval was not included due to the missing data and the fact that this variable did not modify the effect of vaccination. The overall V/E estimate after adjustment for age and month (64% CI 51% to 74%) was similar to the crude estimate. Stratified analyses of V/E according to circulating vaccine type and viral load showed no statistically significant differences (table 4).
Adjusted vaccine effectiveness in stratified analyses or according to influenza virus characteristics
To investigate the precision of mid-season V/E estimates, the adjusted V/E using data available by the end of January (covering the period October–January) was calculated: 79% (30% to 94%) for 2005/06, 63% (9% to 85%) for 2006/07 and 65% (27% to 87%) for 2007/08.
Discussion
In this study, we derived estimates of V/E for laboratory-confirmed influenza cases presenting with ILI in general practices in England, utilising data obtained from a routine surveillance program. The estimates were obtained with no additional costs for data acquisition and, if analysed at regular intervals during the season, could provide a robust estimate of the effectiveness of that season's vaccine within a few weeks of the start and during the period of influenza activity. The critical factors are the accumulation of enough swab results to provide a meaningful estimate and minimising delays in the completion and reporting of virological investigation.
Our method using the influenza negatives as controls is similar to the Broome method for estimating effectiveness of pneumococcal vaccine in which cases infected with a non-vaccine serotype are used as controls.24 Vaccine effectiveness is estimated as 1−odds ratio. The “rare disease assumption” means that this will closely approximate true effectiveness (1−relative risk) because the incidence of influenza is always likely to be <10% per month, and the PCR assay only detects recent infection. Use as controls of individuals presenting with a similar but non-vaccine preventable illness should control for biases arising from confounding of vaccination status with risk of disease or propensity to consult, which are major problems where vaccination is selectively recommended for target clinical groups. As expected, when percentage vaccinated among the non-influenza-confirmed controls was compared with overall coverage in the RCGP population, the latter was found to be lower. For ≥65-year-olds, coverage in the influenza-negative controls was 78% vs 67% in the population and for 45–64-year-olds was 25% and 14%, respectively. Use of the percentage vaccinated in the overall population as the “control” to derive V/E estimates, as is done with the screening method,25 would, therefore, have shown no protection.
The RCGP/HPA virological surveillance program has been maintained continuously throughout winters since 1992/1993, and the recent results using qPCR demonstrate sustainability over three winter seasons. The overall estimate of V/E during the three winters was 65% and changed little (64%) after adjustment for potential confounders. There was no indication of differences in V/E by year or type of influenza despite vaccine mismatches to influenza B in 2005/06 and 2007/08. This suggests that, as found elsewhere,26 the clinical effectiveness of influenza vaccines may not be entirely predictable from laboratory evaluation of the degree of antigenic match between vaccine and circulation strains. Our adjusted V/E estimate for 2005/6 for any confirmed influenza was 67% (41–82) and was close to the V/E estimate of 63% (15–84) reported in a smaller Canadian study in 2005/6 that also used influenza-negative ILI cases as controls and in which influenza B cases comprised half of the total.27 The estimate of V/E was less in persons aged ≥65 years (46%) than in those <65 years (70%): this difference was not significant probably due to low power as only 38 virologically positive samples were available from ≥65-year-olds. VE estimates made using solely the data available by the end of January were similar to those made at the end of the season, despite the usual winter influenza epidemic not reaching a peak until February in two of the three winters. Derivation of an intra-seasonal estimate of V/E using virologically confirmed cases has recently been reported in the USA using the more labour-intensive conventional case-control methodology.26
Rapid evaluation of V/E will be important in the event of deployment of novel vaccines in a future influenza pandemic in order to assess the likely benefit of vaccination on overall morbidity and guide recommendations on targetting any residual supplies of vaccine or anti-viral drugs. With a sufficiently large sampling frame, the method described here could also provide rapid age-specific and virus type-specific V/E estimates for seasonal influenza vaccines. Provision of V/E estimates against virologically confirmed endpoints in small subgroups defined by clinical risk, or in preventing death, is, however, more difficult. For these sub-analyses, use of clinically diagnosed ILI cases may provide a useful indication, although V/E estimates will necessarily be lower due to the inclusion of cases with infections caused by other pathogens. We are currently exploring in the RCGP surveillance cohort the extent to which useful additional information on V/E can be obtained with clinical endpoints.
The potential limitations of our study are bias in ascertainment of influenza cases in vaccinated individuals, data accuracy and power. Our method assumes that the sensitivity of laboratory confirmation does not vary with vaccination status, which is supported by the lack of association between vaccination status and either viral load or swabbing interval. The reduction in viral load and percentage confirmed virologically after 4 days is consistent with a decline in sensitivity of the diagnostic method in those presenting late and potentially might lead to underestimation of V/E.28 However, we found no difference in V/E estimates for those sampled before or after 4 days, and a 50% drop in sensitivity in late samples would only result in a theoretical 5% underestimation. Differences by vaccination status in propensity to consult (or propensity to swab) would only introduce bias if the decision of a vaccinated patient to consult or of a GP to swab a vaccinated patient is related to the likelihood of the patient having influenza. The similarity of the age distribution of the swabbed population and the total population of ILI-diagnosed persons suggest consistency of sampling in vaccinated and non-vaccinated persons and throughout the epidemic period. Easier recognition of influenza and increased likelihood of sampling might incline our estimate of V/E to the more severe cases, but that would be an advantage even though it may lead to underestimation of V/E against all influenza-attributable illness. Errors in form completion, especially with regard to vaccination status, are unlikely because influenza vaccination was given in the same practice and recorded on the electronic record in which the clinical details of the consultation for ILI were recorded: patient recall of vaccination status was unnecessary. Routine surveillance data yielded approximately 270 positive swab samples per year. This number proved adequate to provide an overall estimate of V/E at the end of the season with 95% CI range ±15% on the estimate of 65% and aggregated over three seasons ±10%. Few experimental studies have yielded 270 laboratory confirmed cases; hence, our estimate of V/E is comparable in precision with most work in this area.
In conclusion, we have demonstrated the feasibility of measuring V/E using routine surveillance data and a virus confirmed laboratory endpoint at no extra cost. We have shown in data extending over three seasons that V/E estimation early in the influenza season provided a good indication of the end of season estimate. We have piloted the way for continuing real-time surveillance and developed a health intervention outcome measure which is embedded in normal surveillance routines, providing added value from routinely collected information. Further work is in hand to enhance this system such that a more complete clinical picture can be attached to the virological data in order to adjust for all potential confounders.
What is already known on this subject
Randomised clinical trials (RCT) have demonstrated the clinical and cost effectiveness of routine annual vaccination against influenza, but there are no established methods available to monitor vaccine effectiveness routinely during influenza epidemic periods in order to optimise the management of seasonal influenza, as well as for management of a pandemic.
What does this study add
We demonstrate how information gathered as part of a routine surveillance program can be used to measure vaccine effectiveness against laboratory confirmed influenza. In three winters (2005/06, 2006/07, 2007/08), protective efficacy against virologically confirmed illness was established with estimates of influenza vaccine effectiveness (all ages) of 67%, 55% and 67%, respectively.
Acknowledgments
We are pleased to acknowledge the vital contribution made by participating practices who obtain the swabs which are the essential ingredient for this work. We also express our appreciation to the laboratory staff who undertook the virology. We thank Carol Sadler, Richard Allan and Paola Barbero for excellent technical assistance and Richard Pebody for comments on earlier drafts of the paper.
References
Supplementary materials
Web Only Data jech.2009.093450
Files in this Data Supplement:
Footnotes
Funding This study has been undertaken within the routine budgets of the collaborating participants.
Competing interests DMF has received funding to attend influenza-related meetings and has received consultancy fees from influenza vaccine manufacturers. HPA receives funding from a variety of vaccine manufacturers for work in Maria Zambon's laboratory.
Provenance and peer review Not commissioned; externally peer reviewed.