Background Publication bias is a major threat to the validity of systematic reviews (SRs). Strategies to identify and reduce publication bias are routinely incorporated into SRs of clinical interventions, but the level of adoption of these strategies in SRs relating to health services and delivery research (HSDR) is unclear. The objectives of this study were to describe the characteristics of SRs of HSDR with regards to assessment of publication bias, and to evaluate factors associated with this.

Methods A stratified random sample of 200 SRs of quantitative HSDR published in English from 2007–2017 was selected from the Health Systems Evidence database. Half (100) of the selected reviews (intervention reviews) concern interventions to improve the effectiveness/efficiency of service delivery, which would mostly involve comparative studies. The other half (association reviews) evaluates associations between different variables along the service delivery causal chain, which often include observational studies. Data extracted included: any reference to publication bias, methods for detecting/mitigating publication bias or reasons for no assessment, number of included studies, inclusion of meta-analyses and whether the use of SR guideline was reported. Journals were divided into those that did or did not formally endorse specific SR guidelines such as PRISMA, and journal impact factors were obtained. Factors associated with assessment of publication bias were explored using multivariable logistic regression.

Results Of all 200 SRs, 48% commented on publication bias. However, only 25% formally assessed publication bias either through statistical analysis (mostly funnel plots) or as part of the quality assessment of included studies (e.g. the Cochrane risk of bias tools). Insufficient number of studies, heterogeneity and lack of pre-registered protocols were common impediments in assessing publication bias. In the multivariable analysis, assessment of publication bias was associated with SRs’ inclusion of a meta-analysis [odds ratio (OR) 9.08 (95% CI 3.67, 22.43)], reviewers reporting the use of SR guidelines [OR 4.09 (1.71, 9.78)], being an intervention review [OR 3.75 (1.42, 9.91)] and higher journal impact factor [OR 1.16 (1.02, 1.30)] but not significantly associated with number of studies included in SRs or journal endorsement of SR guidelines.

Conclusion Overall, the awareness of publication bias in HSDR reviews is comparable to that of reviews of clinical interventions, however, formal assessment of publication bias is less common especially in association reviews. This reflects the heterogeneity of HSDR evidence and the limits of current tools for assessing publication bias. Adherence to existing SR guidelines should be promoted.