Article Text

PDF

Addressing conflict in strategic literature reviews: disclosure is not enough
  1. David Michaels
  1. Dr David Michaels, The Project on Scientific Knowledge and Public Policy, Department of Environmental and Occupational Health, The George Washington University School of Public Health and Health Services, 2100 M St. NW, Suite 203, Washington DC, 20037 USA; eohdmm{at}gwumc.edu

Statistics from Altmetric.com

How could disclosure of interests work better in medicine, epidemiology and public health?

The success of the tobacco industry’s multi-decade campaign to delay regulation by manufacturing uncertainty about the studies linking cigarette smoking to lung cancer and other diseases is well documented.1 2 A less well-known consequence of this campaign is the appearance of a new, lucrative application of scientific expertise: product defence. Consulting firms working for producers of toxic chemicals are using the same approaches, and even the same scientists, that the tobacco industry relied on to forestall regulation of cigarettes. Today, these firms aim to impede public health regulation by questioning studies that have identified hazardous properties of asbestos, beryllium, chromium, lead and a host of other toxic chemicals.3

Defending hazardous chemicals has become lucrative business. It is increasingly common for scientific studies to be commissioned in order to be deployed in regulatory or legal proceedings. Companies involved in the welding industry paid more than US$12 million to scientists who published papers disputing the link between welding-related manganese exposure and neurological disease.4 Similarly, two product defence firms working for defendants who produced asbestos brake shoes or related friction products received over US$23 million for their services, billing hundreds of dollars an hour to write papers for peer-reviewed journals.5 Publication by paid experts is not limited to scientists employed by polluters and manufacturers of toxic materials. Of the 26 papers published by US authors between 2002 and 2006 on the risk of disease associated with exposure to asbestos in automotive brake shoes and related products, all were written by scientists involved in litigation: 18 associated with the defendants and 8 with plaintiffs.6 The proliferation of strategically sponsored articles allows subsequent literature reviews to report a predominance of articles reaching a certain conclusion, and mistakenly report the apparent existence of a consensus when that consensus is an artefact of sponsorship.

It is a challenging task to evaluate the validity and credibility of studies commissioned for strategic purposes.7 Should these studies be discounted or dismissed, or at least be subjected to more stringent standards of review than other studies? There are at least two types of papers produced for strategic purposes, to influence litigation or regulation: original studies involving data collection and analysis, and data synthesis exercises. For the first type of study, readers may judge for themselves if a paper’s conclusions are supported by the methods and data; full disclosure of funding and competing interests alerts readers to apply an appropriate level of scrutiny.

Much of the work of product defence firms, however, involves not performing studies but conducting strategic reviews: writing papers and reports (including literature reviews and meta-analyses) that review and interpret the results of previously completed investigations. In these data synthesis exercises, scientists examine several or many studies and consider the findings, validity and importance of each study in reaching an overall conclusion. As one would predict, reviews commissioned by manufacturers of dangerous products generally conclude that the products involved are simply not very dangerous, or are less hazardous than had been suggested by independent scientists.

Editors of biomedical journals have become all too familiar with what has become known as the “funding effect”, or the close correlation between the results desired by a paper’s funders and those reported by the authors.8 For example, among 106 papers published between 1980 and 1995 on the health effects of passive exposure to cigarette smoke, 37% concluded passive smoking was not harmful; 74% of the reviews that reached this conclusion were written by scientists affiliated with cigarette manufacturers.9 The funding effect, however, is not limited to tobacco studies; it has been clearly documented in studies of numerous classes of pharmaceuticals10 11 and more recently in industrial chemicals, as well.12 13

Since product defence firms exist not to further the scientific enterprise but rather to generate revenue through assisting their clients, a cynic might say that their scientists are not actually conflicted: they are paid to produce papers that will influence public policy in a way that favours their clients. Where, then, is the conflict? The strategic reviews of product defence scientists are no more than advocacy briefs, and share the same fundamental objective as the briefs produced by attorneys: to present the evidence in a way that makes the strongest case for their client. An attorney is conflicted if he or she does not represent his or her client’s interest. A product defence firm that produces papers that are used to justify stronger rather than weaker regulation, or ones that can be used in a courtroom against the corporate client, will soon be out of business. This is a function of the basic business model on which these firms operate.3

The old joke among academics—for every study there is an equal and opposite study—clearly does not hold true when it comes to literature reviews. Polluters and manufacturers of hazardous products can pay handsomely (at least by academic standards) for literature reviews downplaying the hazards of a given exposure. Government grants are rarely available for literature reviews; there is little or no alternative funding available for independent scientists who might arrive at alternative conclusions.

One strategy developed by cigarette manufacturers and now used widely by product defence firms is to place in peer-reviewed journals reviews that find few or no effects. For example, both the International Agency for Research on Cancer and the US National Toxicology Program have classified silica as a human carcinogen.14 15 In response, a group of trade associations representing corporations that produce or use silica commissioned a review that found no causal association between silica exposure and lung cancer.16 Similarly, numerous studies have implicated diacetyl, a chemical component of artificial butter flavour, in the causation of bronchiolitis obliterans, also known as popcorn workers lung.17 18 Flavour manufacturers have been sued numerous times by sick workers;19 product defence scientists hired by these manufacturers produced a literature review questioning the causal relationship between the chemical and the disease.20 Scientists, as well as public health regulators and judges, place greater credence in peer-reviewed journals—as they should—but peer review is generally unable to detect fraud, and is not a guarantee of either accuracy or objectivity.21

Disclosure of competing interests, indispensable for considering whether to publish and how to interpret actual studies, cannot fully compensate for bias in review articles and editorials. It has long been the policy of the New England Journal of Medicine, for example, not to publish review articles or editorials by authors with significant financial conflicts.2224 This is not a widely held policy, however, especially among those journals that focus primarily on chemical hazards and environmental health.

The credibility given strategic reviews by publication in peer-reviewed journals is undeserved, and potentially hazardous to public health. Strategic data synthesis exercises, whether they be literature reviews or meta-analyses, are often little more than advocacy briefs made to resemble objective scientific papers. Editors should be hesitant to accept them for publication in the peer-reviewed scientific literature.

REFERENCES

View Abstract

Footnotes

  • Competing interests: None declared.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.