You are here

  • Home
  • Online First
  • It depends on how you ask: measuring bias in population surveys of compliance with COVID-19 public health guidance

Article Text

Article menu

other Versions

Download PDFPDF
Short report
It depends on how you ask: measuring bias in population surveys of compliance with COVID-19 public health guidance
  1. Shane Timmons1,2,
  2. Frances McGinnity1,3,4,
  3. Cameron Belton1,
  4. Martina Barjaková1,
  5. Peter Lunn1,5
  1. 1 Economic and Social Research Institute, Dublin, Ireland
  2. 2 School of Psychology, Trinity College Dublin, Dublin, Ireland
  3. 3 Trinity College Dublin, Dublin, Ireland
  4. 4 School of Sociology, Trinity College Dublin, Ireland
  5. 5 Department of Economics, Trinity College Dublin, Ireland
  1. Correspondence to Shane Timmons, Economic and Social Research Institute, Whitaker Square, Sir John Rogerson’s Quay, Dublin, Ireland; shane.timmons{at}esri.ie

Abstract

Objective Accurate measurement of compliance with COVID-19 guidance is important for public health policy and communications. Responses to surveys, however, are susceptible to psychological biases, including framing effects and social desirability. Our aim was to measure the effects of these biases on estimates of compliance with public health guidance (eg, hand-washing, social distancing).

Design We conducted two online experiments (n=1800) and varied whether questions were framed positively or negatively (eg, ‘I always wash my hands…’ vs ‘I don’t always wash my hands…’). We also varied the degree to which anonymity was assured, via a ‘list’ experiment.

Results Reported compliance, despite being generally high, was reduced by negatively framing questions and increasing anonymity using a list experiment technique. Effect sizes were large: compliance estimates diminished by up to 17% points and 10% points, respectively.

Conclusion Estimates of compliance with COVID-19 guidance vary substantially with how the question is asked. Standard tracking surveys tend to pose questions in ways that lead to higher estimates than alternative approaches. Experimental tests of these surveys offer public health officials greater insight into the range of likely compliance estimates to better inform policy and communications.

  • Cognition
  • Health behaviour
  • Measurement
  • Psychology
  • Public health

This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the COVID-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.

https://bmj.com/coronavirus/usage

https://doi.org/10.1136/jech-2020-215256

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Contributors ST developed the study concept, designed the materials in collaboration with FMG, performed the data analysis and interepretation, and drafted the manuscript. MB assisted with material design and reviewing relevant literature. CB programmed the experiment. CB, FMG and PL provided critical revisions. All authors approved the final version of the manuscript for submission.

    • Funding Both experiments were funded through a research programme commissioned by Ireland’s Department of Health, in support of the Behavioural Change Subgroup of the National Public Health Emergency Team (NPHET).

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.