Background Well conducted systematic reviews provide transparent and robust evidence syntheses. They are, therefore, important for informing clinical practice and health policy. In systematic reviews where meta-analysis is not possible or appropriate, quantitative data are often synthesised narratively. However, a major concern is that narrative synthesis (NS) can lack transparency, leading to bias in the synthesis and threatening the reliability of these reviews for decision makers. We conducted a systematic assessment of Cochrane reviews to establish prevalence of NS, and to assess current practice and transparency in the conduct and reporting of NS in systematic reviews.

Methods Cochrane systematic reviews published between April 2016 and April 2017 (n=714) were examined to determine the primary method of synthesis. Data were extracted from a sample of reviews (n=60/714) that used NS as the primary method of synthesis. A previously tested data extraction template, based on key guidance on NS, was used. This covered reporting of NS methods; transparency between data and text; management and investigation of heterogeneity; and review authors’ reflections on limitations of the synthesis.

Results NS or text only as the primary method of synthesis was used by 16% (n=113/714). In reviews using NS (n=60/113): 53% stated the data were being synthesised narratively; 18% described the NS methods; and 10% referred to NS guidance. Links between the text and the data were clear in 30% (18/60) and, partially in 23% of reviews. The remaining 47% of reviews did not present tabulated summaries of synthesised data. Of the reviews that provided tabulated summaries of data, 16% (n=5/32) did not present the data in the same order and categories as the narrative text. Heterogeneity in the direction of effect of the primary outcome was identified in 14 reviews, of these 46% (n=6/14) attempted to explain the heterogeneity. While 98% of review authors reflected on the limitations of the evidence, 77% reported limitations of the synthesis process.

Conclusion Despite being commonly used, NS is not reported transparently. The limited reporting of methods, and lack of transparency between the data, the narrative and the conclusions, limits assessment of the validity and threatens the reliability of systematic reviews using NS. There is a need for updated guidance on NS. A Delphi survey is underway to develop consensus-based a reporting guideline for NS. The draft items arising from the Delphi survey will be presented.