Background The COVID-19 pandemic resulted in a substantial amount of research produced with startling speed. While the need to understand the nature of the SARS-COV-2 virus, its spread, impact and possible treatments quickly was necessary, we have observed a plethora of variable quality systematic reviews undermining the confidence associated with these methods and challenging the use of evidence to inform practice. To better understand the issues underlying the infodemic of COVID-19 related evidence, we have used an exemplar topic – the effectiveness of convalescent plasma therapy for COVID-19 – to explore the timelines, characteristics and methods used in this body of evidence and offer solutions to avoid similar research waste in future health emergencies.

Methods The Epistemonikos database was searched on 8th June 2021 for systematic reviews on the effectiveness of convalescent plasma for COVID-19 using the phrase ‘convalescent plasma’ in the COVID-19 evidence section. Any, quantitative, systematic review related to the effectiveness or safety of convalescent plasma in treating COVID-19in hospitalised patients was included. Study selection, data extraction and quality assessment (using AMSTAR-2) were conducted and checked by a team of reviewers. Data were tabulated and explored using interactive visualisation methods including network analysis. Issues in the conduct and reporting of systematic reviews and potential solutions were established by consensus with supporting evidence identified from the literature.

Results 51 systematic reviews were included from 24 countries. We found considerable duplication of effort with 48 reviews on the same topic published within the 17 month time frame of the study. There were inconsistencies in the evidence included within the reviews; of a total of 193 individual primary studies, 75% were included in three or fewer reviews. The reviews were of low methodological quality (45 assessed as critically low on AMSTAR-2) and many did not adhere to PRISMA reporting guidance.

Conclusion As researchers we need to conduct, appraise, interpret and disseminate systematic reviews better. To prevent repetition of what we have observed in the COVID-19 infodemic, we need to promote a greater understanding that not all evidence is equal, value robust methods (even if they take more time), consider what the research adds before we conduct it or replicate it) and keep all communication clear (easy to understand and accessible). We welcome further thoughts and discussions on new ways to resolve these issues to help the research community move forward with positive, dynamic and agile strategies.