Article Text
PDF
Statistics from Altmetric.com
The public requires of the medical research community success. The public ultimately provides money for medical research for one purpose only—to generate improvements in patient care. Medical researchers, editors, and peer reviewers should be under no illusions: the public does not support research for the pleasure of watching a cultural event. If improved medical care is not delivered, support for medical research—and hence for medical journals—will dwindle and atrophy.
—Horrobin1
Introduction
The 2014 Lancet series about waste in research posed the question “Why is research that might transform healthcare and reduce health problems not being successfully produced?”2 Indeed, widely disseminated,3 ,4 albeit challenged5 ,6 estimates suggest that most published research findings are false or exaggerated and that 85% of biomedical research resources are wasted.7 Waste is conceptualised in this context as inefficiencies in all aspects of research, including setting priorities, design and conduct of research, regulation and management of research, and failure to publish results.2
We know our successes in identifying strong causes,8 that is, risk factors which causally contribute to disease in a large proportion of cases: the examples of asbestos-related, smoking-related and drinking water-related diseases illustrate the enormous rewards of successful occupational, environmental and public health research. However, in the recent past, potential risk factors of relatively small effects which may have major impacts on public health proved much more difficult to study. One current challenge in this regard is ‘The Plastics Puzzle’9: what research do we need to falsify the hypothesis that ubiquitous synthetic monomers, such as bisphenol A (BPA), and additives to plastics with expected half-lives of centuries, accumulate in wildlife and humans to possibly become a critical burden?10–14
This journal has continuously addressed facets of how we learn from,15 conduct16 and make use17 of public health research. Moreover, the journal has provided commentary18–21 on important details such as conflicts and transparency of interests when research is conducted and used. Several such problems remain unresolved with regard to waste of research. This article looks beyond such critical details of our work and focuses on key angles to avoid haste and waste in research. These include engaging in more focused research via increasing use of reviews, changing publication practice via new (and old) means of recognising priority and quality, and increasing data transparency and reuse via restricted or unrestricted access.
Encourage reviews as a basis for opening or closing research avenues
Some argue that there are too many reviews.22 We disagree. With the aforementioned estimates regarding the correctness and waste of research, we believe that more high-quality syntheses of existing literature are necessary (whether unpublished or—better, of course—published) to ensure that research is actually relevant and focused. Indeed, too many studies are done without sufficient review of what completed studies already tell us. In this vein, reviews are essential to prevent hasty and wasteful individual studies based on a wish to explore without considering why. Much too often, today's mountains of data lead researchers to embark on unfocused climbing expeditions which are mistaken for ‘original studies’. Disconcertingly, the justification for conducting such ‘research’ (with dire implications for the quality of scientific design) is very similar to George Mallory's answer to the newspaper question “Why do you want to climb Mount Everest?”: “Because it is there.”23
In contrast, reviews can open or pave research avenues by identifying patterns which go unnoticed in individual studies. While reviews and individual studies differ in their unit of observation, they should not differ in terms of scientific rigour or creativity. Indeed, state-of-the-art reviews can be original24 by providing an answer to a new question insofar as they collate and synthesise materials which were previously thought to be unrelated.
If we were to need some ‘historical context’ and even more motivation to do the necessary review work to base our research on,25 here is information bridging 200 years: In 1812, the first sentence of the first article in the first issue of The New England Journal of Medicine stated that “In our inquiries into any particular subject of medicine, our labours will generally be shortened and directed to their proper objects, by a knowledge of preceding discoveries.” Moreover, journal data regarding 2013 suggest that reviews continue to play a key role in research:24 According to the ISI Web of Knowledge Journal Citation Records,26 6 of the top 10 journals publish exclusively reviews.
In terms of the example of ‘The Plastics Puzzle’, reviews would show that research into one piece of the puzzle, viz BPA, began some 100 years ago.27 ,28 As much-needed syntheses of abundant and heterogeneous data, reviews could identify what investigations for basic, clinical and epidemiological research with regard to BPA or other suspected endocrine disruptors and human health are relevant,13 ,29 and in what sequence. These may demonstrate that qualitative and quantitative analyses of health risks in the occupational settings of plastic production will not suffice to reach a verdict on possible health risks posed to citizens by ‘plastics’. Indeed, other time windows of exposure and different concepts of dose may be relevant: to exemplify the complexity of ‘The Plastics Puzzle’, experimental evidence in rodents suggests that very low doses of BPA experienced in fetal and/or neonatal life may lead to cancer development through ‘epigenetic programming’.30 ,31
Facilitate recognition of priority and quality to change publication practice
In our view, two key facets of publication practice can be modified in order to avoid haste in and waste of research. At present, numerous scientists publish too quickly as they fear losing credit for being the first to report a finding, and too piecemeal, following the ‘least-publishable-unit’32 approach. Both strategic peculiarities are consequences and expressions of the prevailing publish-or-perish paradigm.
However, there are means to avoid this fear of losing priority—both old and new. In the 1700s, sealed envelopes were deposited with institutions, journals or editors which could reveal—in an a posteriori way when opened—the priority of discoveries. This pli cacheté33 approach could allow work to be continued without rushing to publish because scientists could establish having ‘discovered something first’ despite someone else's publication. The website arXiv.org is already used by scientists to upload electronic preprints or e-prints, and to establish priority before the work is published in a peer-reviewed journal.34 The advantage of the pli cacheté over this modern approach is that researchers can establish priority and continue work without providing clues or leads for their competitors.
To further avoid haste and waste of precious research resources, we should further promote binding procedural rules throughout academia which allow quality, rather than quantity, to be reliably awarded. Angell35 suggested in the 1980s that researchers should be limited to 3–10 significant publications when ‘selling their scientific work’. This suggestion has been followed by Germany's key research foundation (DFG), the British Research Excellence Framework and the National Institutes of Health, all of which now place greater emphasis on the quality rather than the quantity of publications when scientists apply for grants. To prevent haste and waste in research, we must strengthen such rules to reliably shift recognition towards quality in academic processes.
Applying the pli cacheté and quality considerations to the example of ‘The Plastics Puzzle’ could ensure that results were not published too quickly, leading subsequent reviews and research in the wrong direction. Studies which are thorough, complete and of appropriate quality can be expected to be critical in the complex context of this particular research challenge.
Share data to improve quality and increase analyses
A significant proportion of data used for research are generated via public money. It is therefore both logical and ethical to share these data, that is, to make restricted data accessible to certain organisations or individuals, or whenever possible, to allow open data by providing unrestricted data to everyone. Beyond the fact that making the data available to the public is just, it could contribute to avoiding haste and waste in research as well.
More websites could be used to host raw data, allowing individuals or research groups to process this information. When doing this, appropriate dedication can be expected towards generating high-quality data because many scientists beyond those who collected the data will ‘test’ them, securing the data quality necessary for sound analyses and valid results. When data are shared or open, their plausibility could be examined from many angles and by many scientists. Ultimately, existing hypotheses and possible conclusions could be looked at from different viewpoints. New rationale could be developed and its validity tested with the shared data.
When patient data are concerned, it is of course necessary to seek informed consent and secure patient anonymity wherever possible. Details regarding what data should be shared, with whom and how long the researchers who originally generate the data have exclusive access to the materials and ‘first-shots-at-publications’ should be settled with careful debate.
In any case, sharing data would increase the quality of peer review, reanalyses and additional analyses, and exploitation of quality information by several orders of magnitude.
With regard to ‘The Plastics Puzzle’, as numerous untested compounds ‘are really all around us’9 and as we have little, if any, conclusive idea of how they interact or act alone, sharing data may be a conditio sine qua non. Indeed, making data available to different disciplines such as toxicology, endocrinology or epidemiology at the same time could be(come) a must. From a public and environmental health point of view, we must do all we can to solve all or as many parts of this puzzle as possible; the importance of this research question is ever increasing as it concerns ubiquitous exposures to a cocktail of substances which we continue to introduce to work places, food webs and the environment with insufficient understanding of the possible health consequences.
Conclusions
We cannot overstate our conclusions: there is something wrong in the world of research. Our suggestions regarding how to right these wrongs build on earlier proposals in the journal: reviews are certainly needed to learn from what has worked and what has not in order to ensure that medicine is evidence based.15 ,17 Publishing mature, fully-fledged work can be facilitated by existing means to establish priority of discoveries and by establishing reward systems to promote36 quality and not quantity in academic processes such as peer reviews, editorial decisions, tenure policies and grant applications. With regard to ‘Sharing hypotheses and ideas in public health research’16 The Lancet suggests that ‘Sharing a new idea takes courage’, and so does sharing data. It takes both bravery and altruism to share painstakingly collected or generated data with other researchers who may challenge (rightly so) or use them in more telling ways than oneself. Nonetheless, this should become the expectation rather than the exception and providing incentives to researchers to share data might encourage a shift in attitude. Empirical examples of those who share data having been acknowledged in or being offered the opportunity to contribute more and become coauthors of any resulting publications should be followed up. Importantly, researchers must be persuaded that changes of publication practice can benefit both science and scientists.
Some readers might think that our comments and recommendations are not particularly novel, or not specific to the fields of occupational, environmental and public health research. To address the latter objection first, our suggestions are valid for all research endeavours, including the disciplines on which this journal has a particular focus. With regard to the ‘novelty’ of our recommendations, we would like to point out the following. It will hardly be possible (given the fact that countless people have been grappling with the issues concerned) to come up with entirely new suggestions to contribute to changes of the status quo of research. Importantly, with reference to editorials15 ,37 in the journal that identified what worked and what does not, we should weigh carefully what old and possibly new strategies we can encourage and possibly combine to improve the state-of-affairs. To use the example of reviews: it is hardly novel but highly important to promote such work. Revisiting the pli cacheté is rather more novel, and this approach, possibly adjusted for today's needs and options, could be very effective. Taken together, ‘novelty’ per se is not the yardstick but answers to the questions “what worked?” or “what could work?” are.
If implemented, the strategies we outline here should themselves be reviewed by established scientific methods to ensure that they are effective. We must systematically analyse how to shape the research system and offer incentives or rewards for work that avoids haste and produces less waste.
Overall, the message in the quote that prefaces this article should motivate us all to change the current state of affairs in research. The public provides us with funding for the sole purpose of generating improvements in our understanding of the world within and around us, and to improve public and community health.
Acknowledgments
This manuscript was prepared whilst TE was on sabbatical at the University of Oxford, Sleep and Circadian Neuroscience Institute (SCNi).
References
Footnotes
Correction notice This article has been corrected since it was published Online First. The provenance and peer review statement has been corrected.
Contributors TCE conceived the work and the manuscript was collaboratively developed and written by TCE, DMS and JVG.
Competing interests None.
Provenance and peer review Commissioned; externally peer reviewed.