Original article
Are community health interventions evaluated appropriately? A review of six journals☆

https://doi.org/10.1016/S0895-4356(96)00338-1Get rights and content

Abstract

Objectives: To determine if Randomized Controlled Trial (RCT) methodology was used appropriately in community health, we: (1) determined the proportion of non-randomized studies that should have been RCTs, and (2) assessed the quality of the RCTs. Methods: The 1992 issues of six community health journals were manually searched. Intervention studies were analyzed. Studies that did not use randomization were analyzed for feasibility and practicality of RCT methods; RCTs were analyzed for quality using a checklist. RCTs were compared with community health RCTs from The New England Journal of Medicine. The proportion of studies meeting each criterion was determined. Results: Fourteen percent of 603 studies were interventions and 4% were RCTs. Of those not using randomization, 42% should have. Mean RCT scores were significantly lower for the community health journals than for The New England Journal of Medicine. Many criteria important to quality scored poorly. Conclusions: RCTs are underused and lack méthodologic rigor in community health. Conclusions regarding the effectiveness of interventions are therefore suspect.

References (129)

  • JR Gilbert et al.

    A family physician smoking cessation program: An evaluation of the role of follow-up visits

    Am J Prev Med

    (1992)
  • A.L. Cochrane

  • Evidence-based care: 1. Setting priorities: How important is this problem?

    Can Med Assoc J

    (1994)

  • Evidence-based care: 2. Setting guidelines: How should we manage this problem?

    Can Med Assoc J

    (1994)

  • Evidence-based care: 3. Measuring performance: How are we managing this problem?

    Can Med Assoc J

    (1994)
  • J Rafuse

  • Evidence-based medicine: A new approach to teaching the practice of medicine

    JAMA

    (1992)
  • A Robinson

    Research, practice and the Cochrane Collaboration

    Can Med Assoc J

    (1995)
  • P.H. Orr et al.

    Randomized placebo-controlled trials of antibiotics for acute bronchitis: A critical review of the literature

    J Farn Pract

    (1993)
  • A.F. Williams et al.

    Use of seatbelts in cars with automatic belts

    Public Health Rep

    (1992)
  • M.W. Hinds

    Impact of a local ordinance banning tobacco sales to minors

    Public Health Rep

    (1992)
  • L.J. Paulozzl et al.

    An evaluation of the Vermont worksite smoking law

    Public Health Rep

    (1992)
  • D.H. Taylor et al.

    One state's response to the malpractice insurance crisis: North Carolina's Rural Obstetrical Care Incentive Program

    Public Health Rep

    (1992)
  • J.R. DiFranza et al.

    The Tobacco Institute's “It's the Law” campaign: Has it halted illegal sales of tobacco to children?

    Am J Public Health

    (1992)
  • J.L. Forster et al.

    Locking devices on cigarette vending machines: Evaluation of a city ordinance

    Am J Public Health

    (1992)
  • D.E. Peterson et al.

    The effect of state cigarette tax increases on cigarette sales, 1955 to 1988

    Am J Public Health

    (1992)
  • R.L. Flewelling et al.

    First-year impact of the 1989 California cigarette tax increase on cigarette consumption

    Am J Public Health

    (1992)
  • C Poulin et al.

    The hepatitis B immunization strategy in Nova Scotia: Is it effective?

    Can J Public Health

    (1992)
  • S.E. Robertson et al.

    A million dollar measles outbreak: Epidemiology, risk factors, and a selective revaccination strategy

    Public Health Rep

    (1992)
  • J.S. Osberg et al.

    Morbidity among pediatrie motor vehicle crash victims: The effectiveness of seat belts

    Am J Public Health

    (1992)
  • S.A. Esrey et al.

    The complementary effect of latrines and increased water usage on the growth of infants in rural Lesotho

    Am J Epidemiol

    (1992)
  • R Morin et al.

    La promotion du coussin gonflable: Evaluation d'une intervention de sante publique aupres de candidats aux elections federales

    Can J Public Health

    (1992)
  • E Ahmann et al.

    Home apnea monitoring and disruptions in family life: A multidimensional controlled study

    Am J Public Health

    (1992)
  • D.A. Foster et al.

    Influenza vaccine effectiveness in preventing hospitalization for pneumonia in the elderly

    Am J Epidemiol

    (1992)
  • I.J. Higginson et al.

    Effectiveness of two palliative support teams

    J Public Health Med

    (1992)
  • S.J. Manea et al.

    The hepatitis B immunization campaign for children in the Federated States of Micronesia

    Public Health Rep

    (1992)
  • L.M. O'Mara et al.

    Followup of participants in a preschool hearing screening program in child care centres

    Can J Public Health

    (1992)
  • S Ahmed et al.

    Policy and practice —An audit of neonatal BCG Immunization in Avon

    J Public Health Med

    (1992)
  • W.F. Schramm

    Weighing costs and benefits of adequate prenatal care for 12,023 births in Missouri's Medicaid Program, 1988

    Public Health Rep

    (1992)
  • C Hartgers et al.

    Needle sharing and participation in the Amsterdam Syringe Exchange Program among HIV-seronegative injecting drug users

    Public Health Rep

    (1992)
  • C Hartgers et al.

    HIV prevalence and risk behavior among injecting drug users who participate in “Low-Threshold” methadone programs in Amsterdam

    Am J Public Health

    (1992)
  • M Roland et al.

    Boneline: Evaluation of an initiative to improve communication between specialists and general practitioners

    J Public Health Med

    (1992)
  • S Ebrahim et al.

    Assessing the effects of a health promotion programme for elderly people

    J Public Health Med

    (1992)
  • J.N. Payne

    The introduction of a computerized system for notification and improved analysis of infectious diseases in Sheffield

    J Public Health Med

    (1992)
  • B.S. Flynn et al.

    Prevention of cigarette smoking through mass media intervention and school programs

    Am J Public Health

    (1992)
  • N Cotugna et al.

    Development and supermarket field testing of videotaped nutrition messages for cancer risk reduction

    Public Health Rep

    (1992)
  • J.C. Godel et al.

    Iron status and pregnancy in a Northern Canadian population: Relationship to diet and iron supplementation

    Can J Public Health

    (1992)
  • C.L. Perry et al.

    Community-wide smoking prevention: Long-term outcomes of the Minnesota Heart Health program and the class of 1989 study

    Am J Public Health

    (1992)

    • Cited by (30)

    • Level of evidence in four selected rehabilitation journals

      2011, Archives of Physical Medicine and Rehabilitation
      Citation Excerpt :

      The mean kappa value for interrater agreement was .82. Recently there has been a progressive effort by researchers and journal editors to assess and improve the quality of published studies.2-17,21 EBM continues to guide clinical decision-making based on the best available evidence in the literature.13

    • Cardiovascular risk factors and motivation for a healthy life-style in a German community - Results of the GP-based Oestringen study

      2004, Patient Education and Counseling

    • Ensuring the comparability of comparison groups: Is randomization enough?

      2004, Controlled Clinical Trials

    • Randomized trials and public health interventions: Time to end the scientific double standard

      2003, Clinics in Perinatology

    • A community-based program that provides free choice of intervention for older people with chronic pain

      2000, Journal of Pain
      Citation Excerpt :

      Another area of concern in follow-up studies is attrition. Attrition can be problematic because dropout from treatment and attrition from follow-up present problems for generalizing results.51 Of the 71 participants who attended the educational seminar, 18 participants (25%) did not continue with the intervention program; 2 participants (3%) did not provide postintervention data caused by ill health upon completion of the interventions; 3 participants (4%) did not take part in the follow-up measurements; and 3 participants (4%) who attended the multidisciplinary pain management center did not provide follow-up data at the completion of the study because of delay in the commencement and completion of their treatment programs.

    • Social capital and health: Implications for public health and epidemiology

      1998, Social Science and Medicine
    View all citing articles on Scopus
    ☆

    This research was made possible in park by Health Canada through a National Health Research and Development Program research training fellowship to Pamela J. Smith.

    View full text
    Copyright © 1997 Published by Elsevier Inc.