You are here

Article Text

Article menu
Download PDFPDF
Research report
Generalisability of an online randomised controlled trial: an empirical analysis
  1. Cheng Wang1,2,3,4,
  2. Katie R Mollan5,
  3. Michael G Hudgens5,6,
  4. Joseph D Tucker2,3,4,
  5. Heping Zheng2,3,
  6. Weiming Tang2,3,4,
  7. Li Ling1
  1. 1 Faculty of Medical Statistics and Epidemiology, School of Public Health, Sun Yat-sen University, Guangzhou, China
  2. 2 Dermatology Hospital of Southern Medical University, Guangzhou, Guangdong, China
  3. 3 Guangdong Provincial Dermatology Hospital, Guangzhou, China
  4. 4 Project China, University of North Carolina, Guangzhou, China
  5. 5 Center for AIDS Research, School of Medicine, University of North Carolina, Chapel Hill, North Carolina, USA
  6. 6 Department of Biostatistics, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina, USA
  1. Correspondence to Mrs Katie R Mollan, Center for AIDS Research, School of Medicine, University of North Carolina, Chapel Hill, CH, USA; kmollan{at}email.unc.edu, Dr Weiming Tang, University of North Carolina Project-ChinaGuangzhouChina; weimingtangscience{at}gmail.com and Professor Li Ling, Faculty of Medical Statistics and Epidemiology, School of Public HealthSun Yat-sen UniversityGuangzhouChina; lingli{at}mail.sysu.edu.cn

Abstract

Background Investigators increasingly use online methods to recruit participants for randomised controlled trials (RCTs). However, the extent to which participants recruited online represent populations of interest is unknown. We evaluated how generalisable an online RCT sample is to men who have sex with men in China.

Methods Inverse probability of sampling weights (IPSW) and the G-formula were used to examine the generalisability of an online RCT using model-based approaches. Online RCT data and national cross-sectional study data from China were analysed to illustrate the process of quantitatively assessing generalisability. The RCT (identifier NCT02248558) randomly assigned participants to a crowdsourced or health marketing video for promotion of HIV testing. The primary outcome was self-reported HIV testing within 4 weeks, with a non-inferiority margin of −3%.

Results In the original online RCT analysis, the estimated difference in proportions of HIV tested between the two arms (crowdsourcing and health marketing) was 2.1% (95% CI, −5.4% to 9.7%). The hypothesis that the crowdsourced video was not inferior to the health marketing video to promote HIV testing was not demonstrated. The IPSW and G-formula estimated differences were −2.6% (95% CI, −14.2 to 8.9) and 2.7% (95% CI, −10.7 to 16.2), with both approaches also not establishing non-inferiority.

Conclusions Conducting generalisability analysis of an online RCT is feasible. Examining the generalisability of online RCTs is an important step before an intervention is scaled up.

Trial registration number NCT02248558.

  • randomised trials
  • hiv
  • epidemiological methods

https://doi.org/10.1136/jech-2017-209976

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Contributors Conception and design of study: MGH, JDT, KRM, CW and LL. Acquisition of data: JDT, WT, CW and HZ. Analysis and/or interpretation of data: CW, KRM and MGH. Drafting the manuscript: CW. Revising the manuscript: KRM, JDT, MGH, WT, LL and HZ.

    • Funding This work was supported by the National Institute of Allergy and Infectious Diseases (NIAID), US National Institutes of Health (1R01AI114310); University of North Carolina (UNC)–South China STD Research Training Center (Fogarty International Center grant number 1D43TW009532); and UNC Center for AIDS Research (NIAID grant number 5P30AI050410). This publication was supported by the National Center for Advancing Translational Sciences at the National Institutes of Health (grant number UL1TR001111).

    • Competing interests None declared.

    • Ethics approval The study protocol was approved by the participating institutional review boards.

    • Provenance and peer review Not commissioned; externally peer reviewed.