Elsevier

The Lancet

Volume 350, Issue 9074, 2 August 1997, Pages 326-329
The Lancet

Articles
Language bias in randomised controlled trials published in English and German

https://doi.org/10.1016/S0140-6736(97)02419-7Get rights and content

Summary

Background

Some randomised controlled trials (RCTs) done in German-speaking Europe are published in international English-language journals and others in national German-language journals. We assessed whether authors are more likely to report trials with statistically significant results in English than in German.

Methods

We studied pairs of RCT reports, matched for first author and time of publication, with one report published in German and the other in English. Pairs were identified from reports fround in a manual search of five leading German-language journals and from reports published by the same authors in English found on Medline. Quality of methods and reporting were assessed with two different scales by two investigators who were unaware of authors' identities, affiliations, and other characteristics of trial reports. Main study endpoints were selected by two investigators who were unaware of trial results. Our main outcome was the number of pairs of studies in which the levels of significance (shown by p values) were discordant.

Findings

62 eligible pairs of reports were identified but 19 (31%) were excluded because they were duplicate publications. A further three pairs (5%) were excluded because no p values were given. The remaining 40 pairs were analysed. Design characteristics and quality features were similar for reports in both languages. Only 35% of German-language articles, compared with 62% of English-language articles, reported significant (p<0·05) differences in the main endpoint between study and control groups (p=0·002 by McNemar's test). Logistic regression showed that the only characteristic that predicted publication in an English-language journal was a significant result. The odds ratio for publication of trials with significant results in English was 3·75 (95% CI 1·25–11·3).

Interpretation

Authors were more likely to publish RCTs in an English-language journal if the results were statistically significant. English language bias may, therefore, be introduced in reviews and meta-analyses if they include only trials reported in English. The effort of the Cochrane Collaboration to identify as many controlled trials as possible, through the manual search of many medical journals published in different languages will help to reduce such bias.

Introduction

Systematic review of evidence about the benefits and risks of medical interventions can influence decision-making in clinical practice and public-health medicine, identify areas in which further research is needed, and guide allocation of resources.1, 2 The dissemination of medical evidence, including the publication of results from randomised controlled trials (RCTs), is influenced by several factors, however, that modify the probability of whether a trial is included in a meta-analysis. Publication bias-selective publication of significant findings and the non-publication of those without such findings-has been documented repeatedly.3, 4, 5 (The word “significant” relates to statistical significance here and in the rest of the article.) Consequently, only biased samples of all the existing evidence are likely to be publicly available.

Several factors influence the probability of whether a published study is included in a systematic review.6, 7, 8 One factor that has received little attention is the language in which a paper is published.9 Investigators working in non-English-speaking countries publish some of their work in national journals. Authors may be more likely to report in an international, English-language journal results that are significant, whereas other findings are more likely to be published in local journals. English-language bias could, therefore, be introduced in systematic reviews and meta-analyses that are based exclusively on reports written in English. Although concern has been expressed about this type of bias,9, 10, 11 its significance in meta-analytic research is unclear at present.

In an attempt to identify all published controlled trials, the Cochrane Collaboration has embarked on an extensive manual search of many medical journals published in languages other than English.12 We manually searched through five leading German-language general-medicine journals13 and, at the same time, did a bibliographical study. Our objectives were to describe publication trends and quality features of RCTs done in German-speaking Europe and to assess whether trials with significant results are more likely to be published in an international English-language journal than trials without significant findings.

Section snippets

Methods

Five leading general-medicine journals published in German-speaking Europe were searched manually for RCTs. We searched thoroughly each issue of Deutsche Medizinische Wochenschrift, Schweizerische Medizinische Wochenschrift, Schweizerische Medizinische Rundschau (Praxis), Wiener Klinische Wochenschrift, and Wiener Medizinische Wochenschrift published between 1985 and 1994. A trial was defined as an RCT if assignment of participants to treatment and control groups was described as randomised by

Results

255 trials were identified through the manual search. 32 (13%) reports were excluded (23 in French, five in English, two in German from Hungary, and one in German from the UK and Croatia). Therefore, 223 German-language reports published by 529 key authors fulfilled the inclusion criteria. Of these, the numbers of trials published each year decreased from 40 in 1985 to eight in 1994. The Medline search on the 529 key authors found 570 English-language reports. The average number of

Discussion

English is the predominant language in contemporary medical research. Investigators outside the English-speaking world who want their work to be recognised have little choice but to attempt to publish in English. We found that clinical trials are more likely to be reported in an English-language journal if they contain significant results whereas other trials were published in national journals in German. In the USA19 and the UK,3 surveys have shown that investigators are reluctant to submit

References (32)

  • CD Mulrow

    Rationale for systematic reviews

    BMJ

    (1994)
  • RJ Simes

    Confronting publication bias: a cohort design for meta-analysis

    Stat Med

    (1987)
  • PC GØtzsche

    Reference bias in reports of drug trials

    BMJ

    (1987)
  • Egger M, Davey Smith G. Meta-analysis: bias in location and selection of studies. BMJ 1997; (in...
  • Anon

    Evidence-based medicine, in its place

    Lancet

    (1995)
  • I Chalmers et al.

    Getting to grips with Archie Cochrane's agenda

    BMJ

    (1992)
  • Cited by (856)

    View all citing articles on Scopus
    View full text