Confounding in evaluating the effectiveness of influenza vaccine

doi:10.1016/j.vaccine.2008.06.040

Vaccine

Volume 26, Issue 50, 25 November 2008, Pages 6459-6461

https://doi.org/10.1016/j.vaccine.2008.06.040 Get rights and content

Abstract

Confounding is a kind of bias which occurs in a research. Confounding is less frequent in randomized controlled trials (RCT) for evaluation of influenza vaccines. However, there are obstacles or difficulties in conducting RCT for evaluation of influenza vaccines, particularly, in the elderly people. Therefore, a retrospective or prospective cohort study has been primarily performed to evaluate effectiveness of influenza vaccine in elderly people. Confounding by indication or other confounding exist in most observational studies. Accordingly, at the stage of designing or analyzing a study, confounding should be controlled with a restriction, matching, stratified or multivariate analysis technique.

Introduction

According to the definition in ‘A Dictionary of Epidemiology’ edited by Last [1], confounding is a situation in which a measure of the effect of an exposure on risk is distorted because of the association of exposure with other factor(s) that influence the outcome under study. As this pertains to evaluating the effectiveness of an influenza vaccine, a potential confounding factor may not only be related to vaccination, but also to the outcome, such as exposure to influenza viruses and manifestation of influenza-like illnesses. A confounding factor would be associated with the outcome not only in the stratum of the vaccinated group, but also in the stratum of the non-vaccinated group. A confounding factor may or may not carry some causal relationship to the outcome. If a confounding factor is not controlled or adjusted, it inevitably induces a bias in the results.

Section snippets

Confounding factors in evaluation of vaccine

Age and sex, past medical history, present health status, and previous history of vaccination, education, marital status, smoking habits, drinking habits, exercise regime, and so on are potential confounding factors which may affect evaluation of influenza vaccine [2]. Among them, relevant to present health status, confounding by indication is the most important confounding factor affecting evaluation of influenza vaccine. According to Rothman [3], confounding by indication arises from the fact

Methods for controlling a confounding factor

There are 2 stages for controlling a confounding factor. The first stage is at the time of designing a study plan, and the second stage is at the time of data analysis. Restriction of study subjects, matching a confounding factor with the comparative groups, and randomization of the study subjects are methods for controlling a confounding factor at the stage of study design [4]. Restriction is a procedure that limits participation in the study to people who are similar in relation to the

Strengths and limitations of RCT

Because a well-conducted RCT would minimize effect of confounding, valid finding can be obtained from such a trial. However, as previously pointed out by Hak et al. [5], researchers face obstacles when planning a RCT for evaluation of influenza vaccine. Firstly, as the incidence of influenza-like illness or adverse effects of influenza vaccine are low, it would require a large number of study subjects. Secondly, several influenza seasons may need to be observed as the virulence of circulating

Controlling a confounding factor using method other than RCT

Retrospective or prospective cohort studies have been carried out for evaluation of influenza vaccine as designs other than RCT, and, confounding by indication, and other confounding has been adjusted with a technique of restriction, matching, stratified analysis, or multivariate analysis as the following articles illustrate.

A retrospective cohort study was conducted in Rotterdam, the Netherlands in 1996 to assess the effectiveness of influenza vaccination among persons aged 65 years or over

Acknowledgment

This study was supported by a Grant-in-Aid for Scientific Research on Emergency/Re-emergency Infectious Diseases from the Ministry of Health, Labours and Welfare of Japan.

References (16)

W.F. Carman et al.
Effects of influenza vaccination of health-care workers on mortality of elderly people in long-term care: a randomised controlled trial
Lancet
(2000)
M.K. Andrew et al.
Rates of influenza vaccination in older adults and factors associated with vaccine use: a secondary analysis of the Canadian Study of Health and Aging
Bio Med Central Public Health
(2004)
K.J. Rothman
Epidemiology. An introduction
(2002)
I. dos Santos Silvia
Cancer epidemiology: principles and methods
(1999)
E. Hak et al.
Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications
J Epidemiol Commun Health
(2002)
S.J. Allsup et al.
Difficulties of recruitment for a randomized controlled trial involving influenza vaccination in healthy older people
Gerontology
(2002)
E.S. Hurwitz et al.
Studies of the 1996–1997 inactivated influenza vaccine among children attending day care: immunologic response, protection against infection, and clinical effectiveness
J Infect Dis
(2000)

There are more references available in the full text version of this article.

Cited by (16)

Is there a difference in the immune response, efficacy, effectiveness and safety of seasonal influenza vaccine in males and females? – A systematic review
2020, Vaccine
Citation Excerpt :
Although multivariate models are mostly used, doing so provides a unique adjusted effect measure from which the effect of sex has been eliminated. Stratification, on the other hand, provides an effect measure for each sex stratum, and thus enables a comparison of the effect between sexes [17]. It has been suggested that the hypothesized higher immunogenicity in females may logically be accompanied by an increase in the incidence of AEFIs, which in turn, may be linked to vaccine hesitancy [14,18].
Seasonal influenza is an important cause of morbidity and mortality, despite being vaccine-preventable. Sex factors (genes and hormones) seem to impact individuals’ susceptibility to infectious diseases and their response to vaccination. However, most vaccine studies do not explicitly assess sex differences in vaccine response, but rather adjust for sex.
We conducted a systematic review to analyze immunogenicity, efficacy, effectiveness and/or safety of seasonal influenza vaccine data stratified by sex. We searched PubMed, EMBASE, CINAHL, Web of Science and clinicaltrials.gov for observational studies and phase III/IV trials from January 1990 to June 2018, published in English or French. Two reviewers independently screened all references, then proceeded to data extraction and quality assessment using the Cochrane tools (RoB and ROBINS-I) on included studies.
Of the 5,745 citations retrieved, 46 studies were included in the SR. Overall, 18 studies assessed immunogenicity, 1 estimated efficacy, 6 measured effectiveness and 25 evaluated safety of seasonal influenza vaccine in females and males (four studies reported on two sex-stratified outcomes concomitantly).
No clear conclusion could be drawn regarding the effect of sex on the immunogenicity and effectiveness of seasonal influenza vaccine, but higher rates of adverse events following immunization (AEFIs) were reported in females. The heterogeneity of data and studies’ low quality prevented us from conducting a meta-analysis. There is a need to emphasize on the appropriate use of the terms sex and gender in biomedical research. Evidence of higher quality is needed to better understand sex differences in response to influenza vaccine.
Vaccination in the elderly: The challenge of immune changes with aging
2018, Seminars in Immunology
Citation Excerpt :
Since randomized placebo-controlled clinical trials in this age class are very uncommon, most of the data arise from observational studies that have used different designs, outcomes, and end points providing a large set of effectiveness estimates [73]. Confounding factors such as comorbidities, health status or previous history of vaccination can alter the estimates and different methods to adjust for these confounding factors have been used [74]. The efficacy findings analysed in younger, healthy seniors may not apply to older and frail seniors because of advanced age and the presence of serious medical conditions associated with immune functions decline.
The unprecedented increase of life expectancy challenges society to protect the elderly from morbidity and mortality making vaccination a crucial mean to safeguard this population. Indeed, infectious diseases, such as influenza and pneumonia, are among the top killers of elderly people in the world. Elderly individuals are more prone to severe infections and less responsive to vaccination prevention, due to immunosenescence combined with the progressive increase of a proinflammatory status characteristic of the aging process (inflammaging). These factors are responsible for most age-related diseases and correlate with poor response to vaccination. Therefore, it is of utmost interest to deepen the knowledge regarding the role of inflammaging in vaccination responsiveness to support the development of effective vaccination strategies designed for elderly.
In this review we analyse the impact of age-associated factors such as inflammaging, immunosenescence and immunobiography on immune response to vaccination in the elderly, and we consider systems biology approaches as a mean for integrating a multitude of data in order to rationally design vaccination approaches specifically tailored for the elderly.
Control selection and confounding factors: A lesson from a Japanese case-control study to examine acellular pertussis vaccine effectiveness
2017, Vaccine
Citation Excerpt :
In addition, when performing observational studies such as case-control studies to evaluate vaccine effectiveness, the presence of confounders is another concern. In the field of vaccine epidemiology, a confounding factor is defined as a variable which relate to vaccination and to the outcome such as infection or infectious disease development, but which is not on the intermediate from vaccination to outcome [2]. For example, age and underlying illness are generally considered to be important potential confounders that may affect the evaluation of vaccine effectiveness.
When using a case-control study design to examine vaccine effectiveness, both the selection of control subjects and the consideration of potential confounders must be the important issues to ensure accurate results. In this report, we described our experience from a case-control study conducted to evaluate the effectiveness of acellular pertussis vaccine combined with diphtheria-tetanus toxoids (DTaP vaccine). Newly diagnosed pertussis cases and age- and sex-matched friend-controls were enrolled, and the history of DTaP vaccination was compared between groups. Logistic regression models were used to calculate odds ratios (ORs) and 95% confidence intervals (CIs) of vaccination for development of pertussis.
After adjustment for potential confounders, four doses of DTaP vaccination showed a lower OR for pediatrician-diagnosed pertussis (OR = 0.11, 95% CI, 0.01–0.99). In addition, the decreasing OR of four doses vaccination was more pronounced for laboratory-confirmed pertussis (OR = 0.07, 95%CI, 0.01–0.82). Besides, positive association with pertussis was observed in subjects with a history of steroid treatment (OR = 5.67) and those with a recent contact with a lasting cough (OR = 4.12).
When using a case-control study to evaluate the effectiveness of vaccines, particularly those for uncommon infectious diseases such as pertussis, the use of friend-controls may be optimal due to the fact that they shared a similar experience for exposure to the pathogen as the cases. In addition, to assess vaccine effectiveness as accurately as possible, the effects of confounding should be adequately controlled with a matching or analysis technique.
Are influenza-associated morbidity and mortality estimates for those ≥65 in statistical databases accurate, and an appropriate test of influenza vaccine effectiveness?
2014, Vaccine
To assess the accuracy of estimates using statistical databases of influenza-associated morbidity and mortality, and precisely measure influenza vaccine effectiveness.
Laboratory testing of influenza is incomplete. Death certificates under-report influenza. Statistical database models are used as an alternative to randomised controlled trials (RCTs) to assess influenza vaccine effectiveness. Evidence of the accuracy of influenza morbidity and mortality estimates was sought from: (1) Studies comparing statistical models. For four studies Poisson and ARIMA models produced higher estimates than Serfling, and Serfling higher than GLM. Which model is more accurate is unknown. (2) Studies controlling confounders. Fourteen studies mostly controlled one confounder (one controlled comorbidities), and limited control of confounders limits accuracy.
(1) Studies of regions with increasing vaccination rates. Of five studies two controlled for confounders and one found a positive vaccination effect. Three studies did not control confounders and two found no effect of vaccination. (2) Studies controlling multiple confounders. Of thirteen studies only two found a positive vaccine effect and no mortality differences between vaccinees and non-vaccinees in non-influenza seasons, showing confounders were controlled.
Key problems are insufficient testing for influenza, using influenza-like illness, heterogeneity of seasonal and pandemic influenza, population aging, and incomplete confounder control (co-morbidities, frailty, vaccination history) and failure to demonstrate control of confounders by proving no mortality differences between vaccinees and non-vaccinees in non-influenza seasons.
Improving model accuracy requires proof of no mortality differences in pre-influenza periods between the vaccinated and non-vaccinated groups, and reduction in influenza morbidity and mortality in seasons with a good vaccine match, more virulent strains, in the younger elderly with less immune senescence, and specific outcomes (laboratory-confirmed outcomes, pneumonia deaths).
Proving influenza vaccine effectiveness requires appropriately powered RCTs, testing participants with RT-PCR tests, and comprehensively monitoring morbidity and mortality.
Real-World Evidence in Cost-Effectiveness Analysis of Enhanced Influenza Vaccines in Adults ≥ 65 Years of Age: Literature Review and Expert Opinion
2023, Vaccines
Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019
2022, Systematic Reviews

View all citing articles on Scopus

View full text

Confounding in evaluating the effectiveness of influenza vaccine

Abstract

Introduction

Section snippets

Confounding factors in evaluation of vaccine

Methods for controlling a confounding factor

Strengths and limitations of RCT

Controlling a confounding factor using method other than RCT

Acknowledgment

Lancet

Rates of influenza vaccination in older adults and factors associated with vaccine use: a secondary analysis of the Canadian Study of Health and Aging

Bio Med Central Public Health

Epidemiology. An introduction

Cancer epidemiology: principles and methods

Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications

J Epidemiol Commun Health

Difficulties of recruitment for a randomized controlled trial involving influenza vaccination in healthy older people

Gerontology

Studies of the 1996–1997 inactivated influenza vaccine among children attending day care: immunologic response, protection against infection, and clinical effectiveness

J Infect Dis