Article Text

The effectiveness of anti-illicit-drug public-service announcements: a systematic review and meta-analysis
1. Dan Werb1,
2. Edward J Mills1,2,
3. Kora DeBeck1,
4. Thomas Kerr1,3,
5. Julio S G Montaner1,3,
6. Evan Wood1,3
1. 1British Columbia Centre for Excellence in HIV/AIDS, Vancouver, Canada
2. 2Faculty of Health Sciences, Simon Fraser University, Vancouver Canada
3. 3Department of Medicine, University of British Columbia, Vancouver, Canada
1. Correspondence to Evan Wood, BC Centre for Excellence in HIV/AIDS, Department of Medicine, University of British Columbia 608-1081 Burrard Street, Vancouver, BC V6Z 1Y6, Canada; uhri-ew{at}cfenet.ubc.ca

## Abstract

Background Anti-illicit-drug public-service announcements (PSAs) have become a cornerstone of drug policy in the USA. However, studies of the effectiveness of these interventions have not been subjected to a systematic evaluation.

Methods The authors searched 10 electronic databases along with major conference abstract databases (from inception until 15 February 2010) for all articles and abstracts that evaluated the effectiveness of anti-illicit-drug PSAs. The authors evaluated all studies that assessed intention to use illicit drugs and/or levels of illicit-drug use after exposure to PSAs, and conducted meta-analyses of these studies.

Results The authors identified seven randomised trials (n=5428) and four observational trials (n=17 404). Only one randomised trial showed a statistically significant benefit of PSAs on intention to use illicit drugs, and two found evidence that PSAs significantly increased intention to use drugs. A meta-analysis of eligible randomised trials demonstrated no significant effect. Observational studies showed evidence of both harmful and beneficial effects.

Conclusion Existing evidence suggests that the dissemination of anti-illicit-drug PSAs may have a limited impact on the intention to use illicit drugs or the patterns of illicit-drug use among target populations.

• Anti-drug
• PSA
• mass media
• youth
• social marketing
• meta-analysis
• meta-analysis ME
• policy analysis SI
• systematic reviews
• youth health

## Background

Illicit-drug use continues to pose a serious threat to public health in a number of settings, and the prevalence of marijuana use among youth continues to be of particular concern.1–4 In addition, the consumption of drugs such as methamphetamine, heroin and crack cocaine presents complex public-health challenges.1–3 Given their potential for harm, the effective prevention of the consumption of such drugs among vulnerable populations requires the development of policies guided by the best available scientific evidence.

One popular response to illicit-drug use has been the dissemination of anti-illicit-drug public service announcements (PSAs) through media campaigns, most often targeted at youth. PSAs are commercials that provide advice or information, or promote activities regarded as serving community interests. They are produced for a variety of media including television, radio, print, and the internet,4–6 and antitobacco PSAs appear to have been effective in modifying attitudes towards tobacco use among targeted populations.7 In the USA, anti-illicit-drug PSAs have been a cornerstone of the country's drug policy since at least the 1970s,8 and a national anti-illicit-drug use media campaign has been operating since 1999.4 Further, despite announced reforms in American policy on illicit-drug use under the administration of President Barack Obama, the Office of National Drug Control Policy has recently announced an increase of US$21.5 million in funding for the US National Youth Anti-Drug Media Campaign.9 Additionally, Canada, Australia and the UK have all embraced anti-illicit-drug PSAs as part of their national drug-control strategies.5 6 10 11 Despite the popularity of anti-illicit-drug PSAs as a means of combating illicit-drug use among youth, the impact of these strategies in reducing the consumption of, and modifying intentions to use, illicit drugs remains unknown. We therefore conducted the following systematic review and meta-analyses to investigate the state of the research related to the effectiveness of anti-illicit-drug PSAs in modifying behaviour and intention to use illicit drugs among target populations. ## Methods We referred to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting of systematic reviews and meta-analyses in this analysis as well as the ‘Meta-analysis of observational studies in epidemiology’ statement on reporting of meta-analyses of observational studies.12 13 ### Eligibility criteria Our primary outcome of interest was the effectiveness of anti-illicit-drug PSAs in modifying intentions to use and/or reducing self-reported use of illicit drugs. We reviewed both randomised controlled trials (RCTs) and observational studies, but placed primary emphasis on RCTs. We considered all studies published in peer-reviewed journals, abstracts from international conferences and governmental reports. We did not include evaluations of anti-licit-drug (ie, tobacco or alcohol) PSAs in our analyses as alcohol and tobacco are government sanctioned. Studies of illicit-drug prevention campaigns that included multicomponent interventions (ie, school-based and media-based interventions) were only included if the impacts of anti-illicit-drug PSAs were evaluated independently. ### Information sources We searched the following 10 electronic databases (from inception to 15 February 2010): PubMed, PsycINFO, EMBASE, Cochrane CENTRAL, CINAHL, Web of Science, TOXNET, AIDSLINE, AMED and ERIC. ### Search We searched for all English-language articles and abstracts, and set no date limits. Search terms included ‘anti-drug,’ ‘anti-marijuana,’ ‘national anti-drug youth media campaign,’ ‘youth,’ ‘drug prevention,’ ‘adolescent,’ ‘public service announcement’ and ‘PSA.’ We also examined references from relevant articles. ### Study selection Using a predefined protocol (available from corresponding author on request), two investigators (DW, EW), working independently, scanned all of the abstracts and obtained the full text of articles and reports that evaluated a measure of effectiveness. We assessed validity in duplicate using the following criteria: (1) study design and (2) measure of effectiveness. After obtaining the full reports of the candidate studies (a full peer-reviewed article, conference abstract or non-peer-reviewed report), the same reviewers independently assessed eligibility. After all potentially relevant full-text articles and abstracts were identified, three of the authors (DW, EM, EW) met to achieve consensus regarding eligibility. ### Data collection process Between 1 May 2007 and 15 February 2010, we conducted data extraction independently, in duplicate, using a standardised form. Data abstractors collected information about the study design, sample size, methods of effectiveness measurement and outcomes. The data were entered into an electronic database such that duplicate entries existed for each study; when the two entries did not match, we reached consensus through discussion (DW, EW). ### Risk of publication bias Experts have noted that evaluations of school-based anti-illicit-drug interventions may suffer from publication bias that may deter researchers from publishing negative or null findings of these interventions.14 Given the similarities between education-based and social-marketing anti-illicit-drug interventions, a similar bias may affect the publication of evaluations of anti-illicit-drug PSAs. ### Meta-analysis The primary meta-analysis considered all RCTs using random effects, which is an approach that recognises and anchors studies as a sample of all potential studies, and incorporates an additional between-study component to the estimate of variability. In trials that evaluated multiple interventions (ie, more than one anti-drug campaign), we included the outcomes for the substudies as separate entries within the meta-analysis. In cases where study samples were stratified between particular groups, we conducted subgroup analyses and entered each subgroup as a separate data point in our meta-analysis. Because studies reported primary outcomes as continuous, we applied the weighted mean difference as the primary outcome. We used the I2 to detect heterogeneity between studies in our meta-analysis. We also reviewed all observational studies and conducted a second meta-analysis of observational studies again using a weighted means difference design with random effects. Analyses were conducted using StatsDirect version 2.5.2.15 ### Role of the funding source This was an investigator-initiated study without external funding support. No external funder played a role in the collection, analyses, interpretation of data, writing of the report or decision to publish. All authors had complete access to all data, and all had final responsibility to submit for publication. ## Results Our initial search yielded 462 potential studies, of which 49 met our criteria and were further assessed. Thirteen studies were excluded because they were neither peer-reviewed nor published by a governmental agency. A further 18 studies were excluded because they did not directly evaluate the effect of anti-illicit PSAs on intention to use illicit drugs or on self-reported illicit-drug use. Four studies were excluded because they only evaluated the effectiveness of anti-drug media interventions in reducing licit drug use (ie, tobacco or alcohol). Two studies were excluded because they investigated multicomponent interventions and did not provide separate findings for anti-illicit-drug media interventions. Twelve studies,4 5 16–25 published between 1989 and 2008, met the eligibility criteria for our review. However, one was excluded because it reported duplicate findings,17 and we opted to include the more recent publication of this study.16 Of the remaining 11 studies, seven (n=5428) used an RCT design,16 18–23 and four (n=17 404) used observational study designs.4 5 24 25 ### Systematic review of RCTs Eligible studies are presented in table 1. As shown, studies used diverse methodologies to examine the impact of anti-illicit-drug PSAs on intention to use illicit drugs and levels of illicit-drug use among targeted populations. All RCTs used a control program or compared the effectiveness of various PSAs. Of seven RCTs, six compared individuals exposed to anti-illicit-drug PSAs with individuals exposed to a control program,16 18–22 while one study employed a between-groups design in which individuals exposed to different types of anti-illicit-drug PSAs were compared.23 Sample sizes ranged from 28 to 3608 (median: 284, IQR: 47–80). Three RCTs exposed individuals to marijuana-specific anti-illicit-drug PSAs,18 20 23 and four RCTs exposed individuals to a variety of anti-illicit-drug PSAs.16 19 21 22 Two RCTs reported positive effects of anti-illicit-drug PSAs corresponding to a −0.01 reduction in intention to use illicit drugs on a 1–7 scale of intention, and to a 0.06 increase in intention to call a drug-abuse hotline on a 1–5 scale of intention.16 18 Five RCTs reported non-significant and/or negative effects of such interventions.19–23 Furthermore, the RCTs by Fishbein et al,22 Yzer et al21 and David et al23 used novel methodological approaches to measuring PSA effectiveness. Table 1 Characteristics of available eligible studies Fishbein et al conducted an RCT in which they evaluated the relative effectiveness of 30 anti-illicit-drug PSAs in modifying the intention of targeted individuals to use illicit drugs.22 Participants (n=3608) were randomly assigned to view six of a possible 30 anti-illicit-drug PSAs or a control program. They were then immediately evaluated after exposure, and their assessment of the PSAs was recorded. Overall mean relative scores of PSA effectiveness were then generated. The mean scores suggest that 16 PSAs were more effective than the control program in reducing intention to use illicit drugs among study participants, eight did not differ significantly from the control, and six were significantly less effective than the control in reducing intention to use illicit drugs (ie, these PSAs significantly increased the intention to use illicit drugs) among participants. In this study, an effect size of 0 represented a null effect, and the five most effective PSAs were those with content focussing on heroin and methamphetamine, with relative effect sizes ranging from 0.597 to 0.938. By contrast, the five least effective PSAs addressed marijuana use or focused on building the self-esteem of viewers, with the authors reporting relative effect sizes ranging from −0.089 to −0.286.22 These effect sizes suggest that these five PSAs had significant negative effects on reducing intention to use illicit drugs compared with a control program; that is, they actually increased the intention of exposed participants to use illicit drugs. Additionally, Yzer et al observed no significant effects of exposure to anti-illicit-drug PSAs among a sample of youth (n=418) compared with a control program in decreasing intention to use marijuana.21 However, individuals exposed to anti-illicit-drug PSAs that explicitly mentioned the gateway theory of drug use (ie, that marijuana use leads to the use of ‘harder’ drugs such as cocaine and heroin) reported significantly weaker anti-marijuana norms than the control group.21 Finally, the possibility that the effectiveness of anti-marijuana PSAs is mediated by group interaction was investigated using an RCT by David et al, who conducted a post-test only between-subjects study of 7th- and 12th-grade students in the Philadelphia area (n=535), in which group interaction was measured by observation of online ‘chatting’ (ie, participation in an online chat room environment) between study participants.23 The authors found that individuals who participated in online chatting after exposure to anti-marijuana PSAs reported significantly weakened anti-marijuana beliefs compared with those study participants that did not engage in online chatting.23 ### Systematic review of observational studies Among the four observational studies that investigated the effectiveness of anti-illicit-drug PSAs in reducing levels of illicit-drug use, sample sizes ranged from 1490 to 4803 (median: 3186; IQR: 3142–4795), and study periods ranged from 1 to 5 years. All observational studies observed the effects of anti-illicit-drug PSAs within the context of either a community- or national-based media campaign, and two studies examined the effects of the US National Youth Anti-Drug Media Campaign.4 25 Specifically, two observational studies conducted by Palmgreen et al in the state of Kentucky found that anti-illicit-drug PSAs were associated with significant 8.8% and 10.7% reductions in illicit-drug use.24 25 Furthermore, one national study conducted in Australia observed a 3% reduction in use among study participants but did not report on levels of significance.5 Finally, as will now be described, one 5-year US-based national observational study observed a non-significant 0.4% increase in use as well as potential negative effects on attitudes towards illicit drugs.4 Specifically, the US Office of National Drug Control Policy's National Youth Anti-Drug Media Campaign, ongoing since 1999, constitutes the largest PSA-based anti-illicit-drug intervention in the world. A 5-year observational study using a national sample of youth as well as county-level observational studies were conducted in order to determine its potential effectiveness in modifying drug use patterns among youth. It is noteworthy that those observational studies conducted by Palmgreen et al in two counties in Kentucky concluded that components of the National Youth Anti-Drug Media Campaign appeared to be effective in reducing rates of illicit-drug use among youth, as mentioned above.24 25 However, when campaign effects were investigated at the national level by Orwin et al, there was no evidence that the dissemination of anti-illicit-drug PSAs had a significant effect on reducing levels of illicit-drug use.4 Further, the authors found that higher exposure to the campaign was significantly associated with the negative effect of weaker anti-illicit-drug norms among study participants corresponding to a 6.29 decrease in negative attitudes and beliefs related to marijuana on a scale with a baseline mean and SD of 100 each.4 Phase 2 of the Australian government's National Drugs Campaign, targeted towards youth aged 13–24 years old, was also evaluated using a prospective observational study design.5 According to the evaluators, modest attitudinal changes were observed among the entire sample during the 1-year study period. Specifically, significant increases were observed among youth believing that amphetamine and ecstasy use can lead to paranoia, depression, aggression and lethargy. However, no significant differences in rates of illicit-drug use were observed.5 Findings of effectiveness in both RCTs and observational studies in our systematic review were generally restricted to subpopulations identified as high sensation seekers. High sensation seeking is a personality trait characterised by the need for novel, complex, ambiguous and emotionally intense stimuli, and the willingness to take risks to obtain such stimulation. Individuals identified as high sensation seekers are believed to be at higher risk of initiating illicit-drug use compared with low sensation seekers.24 ### Meta-analyses #### RCTs We conducted a meta-analysis restricted to the RCTs included in our systematic review. Three RCTs were reviewed but were ineligible for inclusion in our meta-analysis because, while they evaluated the effect of PSAs on intention to use illicit drugs, they did not present the required outcome data.16 22 23 Also, because the study by Wagner and Sundar included two separate RCTs, we considered these trials independently in the meta-analysis. As shown, when we conducted a weighted means difference meta-analysis of RCTs using random effects, we found a non-significant effect size of 0.29 (95% CI −0.17 to 0.75 (p=0.217), I2=66.1%; 95% CI 0% to 84.9%). The study by Harrington et al stratified participants into high- and low-sensation-seeking youth, and we therefore conducted a subgroup analysis among these subgroups in the meta-analysis. This meta-analysis generated a non-significant effect size of 0.15 (95% CI −0.19 to 0.49 (p=0.382), I2=53.2% to 95% CI 0% to 80.8%). Effect sizes and findings of the subgroup meta-analysis are presented in figure 1. Figure 1 Effect sizes in meta-analysis of randomised controlled trials of anti-illicit-drug public service announcements. HSS, high sensation seekers; LSS, low sensation seekers. ### Observational studies We conducted a second meta-analysis restricted to observational studies. Because certain studies included multiple evaluations of separate interventions or stratified findings by sensation seeking status,24 25 we entered these findings separately in our meta-analysis (full data available from the corresponding author). As described in figure 2, when we conducted a meta-analysis of the four observational studies, we found an effect size of −0.04 (95% CI −0.06 to −0.01 (p=0.004), I2=100%, 95% CI 100%, 100%), corresponding to an estimated 4% reduction in the use of illicit drugs among individuals exposed to anti-illicit-drug PSAs, though it is noteworthy that we observed the maximum level of heterogeneity in this analysis. Figure 2 Effect sizes in meta-analysis of observational studies of anti-illicit-drug public service announcements. HSS, high sensation seekers; LSS, low sensation seekers. ## Conclusion The present systematic review demonstrates limited evidence to support the use of PSAs for illicit-drug prevention among youth. Our meta-analysis of RCTs demonstrated no significant benefit, and no studies reported any long-term effectiveness of these interventions. Importantly, most RCT and observational studies reported non-significant results, and three RCTs and one observational study found that anti-illicit-drug PSAs may have negative effects on anti-illicit-drug-use norms among targeted populations.4 19 20 22 These findings are of immediate importance given the high costs of the production and dissemination of anti-illicit-drug PSAs as well as the high levels of drug use among youth in a variety of settings.26 The government Accountability Office of the US government recently recommended that ‘Congress should consider limiting appropriations for the (National Youth Anti-Drug Media) campaign, beginning in the 2007 fiscal year budget until Office of National Drug Control Policy's provides credible evidence of a media campaign approach that effectively prevents and curtails youth drug use.’27 Despite this recommendation, recent increases in funding towards this program have been approved by the administration of President Barack Obama for the 2010/2011 fiscal year.9 While it is noteworthy that a decline from 55% to 47% in national prevalence of marijuana use was observed among youth in the USA between 1999 and 2007,28 research to date has not demonstrated any association between this decline and the dissemination of anti-illicit-drug PSAs.4 Elsewhere, the FRANK anti-illicit-drug media campaign established by the Home Office of the UK has cost £24 million (US$47 million) and has yet to be the subject of an independent, arms-length scientific evaluation.6 In Australia, over $60 million has been spent on the National Drugs Campaign since 2001, though the long-term effectiveness of the campaign has not been observed.5 The Canadian federal government also announced$10 million in new funding for the dissemination of anti-illicit-drug PSAs in 2007,10 11 which may reflect a greater harmonisation of North American drug policy.29

A secondary potential effect of anti-drug media campaigns is that these interventions may help to reinforce support for existing illicit-drug policy approaches. For instance, a large study from the USA, where most citizens derive their drug-use information from the mass media, demonstrated that most Americans approved of a continuation of a ‘war on drugs’ approach and that only weak support existed for increasing access to addiction treatment.30 These findings imply that, while anti-drug messaging may have little direct benefit in reducing drug use among youth, anti-drug PSAs may nevertheless contribute to support for abstinence- and enforcement-based policy responses to illicit-drug use.

The theoretical framework used to produce and evaluate anti-illicit-drug media campaigns may partially explain the difficulty that scientists have faced in evaluating these interventions. Theories derived from social cognitive theory,31 such as the theory of reasoned action and the theory of planned behaviour,32 33 serve as foundational models for a range of health-behaviour communication interventions, including anti-illicit-drug PSAs.34 While these theories are based on the notion of a specific contiguous relationship between intention and behaviour,35 research has demonstrated that socio-demographic, environmental and other variables may play a critical role in reducing an individual's ability to act according to their intentions.36 In the context of youth drug use, these theories may therefore be unable to explicate associations between behavioural interventions and behaviour, as intentions to use drugs may be mediated by a range of confounding factors. In particular, the exclusion of key sociodemographic variables on study participants such as ethnicity, neighbourhood of residence, income, housing situation and others from the vast majority of anti-illicit-drug PSA evaluations may critically limit the evaluation of these interventions. Furthermore, the fact that the study by David et al found that online chatting decreased the effectiveness of anti-illicit-drug PSAs suggests that social networks likely also act as mediating influences on the effect of these interventions on youth.23 As such, anti-illicit-drug PSAs should be developed with a broader understanding of the range of external factors that influence drug-related decision-making.

Some evaluators have also suggested that the observed negative outcomes of the dissemination of anti-illicit-drug PSAs may be a result of PSA content that increases the perception that drug use among youth is widespread. Specifically, the evaluators of the National Youth Anti-Drug Media Campaign stated that, ‘If the meta-message is that drug use is widespread, higher exposure to Campaign ads should cause an immediate effect on the perception that other kids regularly use marijuana…This perception eventually leads to a more generalised pro-marijuana social norm…and greater likelihood of actual initiation.’4

Our study contains several important limitations, the first relating to the lack of published research on anti-illicit-drug PSAs, which limits the statistical power of our meta-analyses. This limitation is particularly acute with respect to the dearth of studies on the long-term effectiveness of these interventions. A recent commentary also noted that publication bias may have prevented the publication of a number of negative studies regarding the effectiveness of school-based anti-illicit-drug interventions.14 Given that similarities exist between such interventions and anti-illicit-drug PSAs, evaluations with null findings may be under-reported in the literature. Second, all but one of the studies included in our meta-analyses were from the USA, and as such, our meta-analytical findings may not be generalisable to other settings. Third, although experts have recently suggested the importance of meta-analyses, even when heterogeneity between study designs exists,37 it is noteworthy that the studies we considered used diverse designs and outcomes, and that our meta-analysis of observational studies contained the maximum level of heterogeneity. As such, the results of this meta-analysis reflect only a crude estimate of the short-term effectiveness of published observational studies of anti-illicit-drug PSAs. Finally, because two RCT studies only presented outcomes as ratios between groups,16 23 and one RCT study only measured the effectiveness of 30 anti-illicit-drug PSAs relative to each other,22 we were unable to include these studies in our meta-analysis. While these exclusions also likely affected our calculation of overall effect size, it is noteworthy that the majority of these trials reported non-significant or negative effects of anti-illicit-drug PSAs.16 22 23 We stress, however, that our meta-analysis of RCTs contains important limitations related to the fact that, as previously noted, RCTs of anti-illicit-drug PSAs often contain serious methodological problems that restrict their capacity to provide evaluations of effectiveness, particularly in the long-term.

In summary, the present review and meta-analysis indicate that insufficient data exist to support the conclusion that anti-illicit-drug PSAs are effective in modifying intention to use illicit drugs and reducing self-reported illicit-drug use among targeted youth. As such, novel methods of evaluating the effects of these interventions, and particularly their long-term effects, are urgently needed. Our findings should also help reinforce the need for evidence-based approaches to reducing drug-related harm and a re-evaluation of the use of existing modes of media-delivered illicit-drug-prevention messages. Although further research is necessary, several studies have suggested that these interventions may contribute to a weakening of anti-illicit-drug norms and to increased initiation of illicit-drug use among exposed youth. Given the large knowledge gaps that continue to persist, the potential of anti-illicit-drug PSAs to weaken anti-illicit-drug norms among youth and the high cost of anti-illicit-drug media campaigns, funding for these interventions should be contingent on scientific evidence of their effectiveness.

### What is already known on this subject

• Anti-drug PSAs are popular preventive interventions.

• In the USA, over US\$20 million is allocated annually towards anti-drug PSAs, despite a government report suggesting that they are likely ineffective.

• While studies of the effectiveness of anti-drug PSAs have been conducted, findings have been mixed.

• No systematic review or meta-analysis of anti-drug PSAs has yet been conducted.

• The findings of this study suggest that there is little support for the hypothesis that anti-drug PSAs are effective in reducing intention to use drugs, or actual drug use among youth.

• Furthermore, multiple studies have noted the potential of anti-drug PSAs to weaken anti-drug norms among youth, which may in turn lead to increases in the prevalence of drug use among this population.

### Policy implications

• Consistent with the US government Accountability Office's recommendations, funding for anti-drug PSAs should be contingent on scientific evidence of effectiveness.

• These recommendations hold particular weight considering the massive resources currently allocated by governments towards anti-drug PSAs.

• Future evaluations of anti-drug PSAs should consider the impact of sociodemographic factors in modifying the effectiveness of these interventions.

• Policymakers should fund further research on the potential negative impact of anti-drug PSAs.

## Acknowledgments

The authors would like to thank B Rachlis for her advice regarding the implementation of the systematic review. The authors would also like to thank P Palmgreen, N Harrington, M Yzer, M Czyzewska and SS Sundar for providing detailed responses to our requests for additional data.

## Footnotes

• Funding Michael Smith Foundation for Health Research; Canadian Institutes of Health Research.

• Competing interests EM has received grants and served as an ad hoc advisor to Pfizer, and JM has received grants from, served as an ad hoc adviser to, or spoken at events sponsored by Abbott, Argos Therapeutics, Bioject, Boehringer Ingelheim, BMS, Gilead Sciences, GlaxoSmithKline, Hoffmann-La Roche, Janssen-Ortho, Merck Frosst, Panacos, Pfizer, Schering, Serono, TheraTechnologies, Tibotec (J&J) and Trimeris.

• Provenance and peer review Not commissioned; externally peer reviewed.

## Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.