Statistics from Altmetric.com
Postal surveys are a cost effective method of obtaining information from large numbers of geographically disparate medical professionals about their attributes, behaviours, attitudes and beliefs. The validity of the findings of such surveys is primarily dependent upon an adequate response rate and it is of concern that the rates are declining.1 Inclusion of a pen along with the questionnaire is often used to counter this trend. In theory, the availability of a pen would enable immediate completion of the questionnaire. The gift of a pen may also predispose the recipient to look upon the questionnaire more favourably increasing the likelihood of a response. However, the effectiveness of this strategy has not been established. Therefore as part of a national survey of consultant gynaecologists, we conducted a randomised controlled trial, to determine if providing a pen along with the questionnaire increases response rates.
Methods and Results
All consultants identified from the Royal College of Obstetricians and Gynaecologists (RCOG) database were sent a questionnaire with a covering letter and prepaid response envelope. The questionnaire sought views about (1) establishing a central database of outpatient hysteroscopy clinics and (2) interest in participating in a clinical trial of hysteroscopic therapy. Recipients were randomised to receive a simple plastic ballpoint pen or not in addition to the questionnaire. The randomisation sequence was computer generated and group allocation was concealed from the participants throughout the study. One reminder (with or without a pen according to the original allocation) was sent to non-responders three months after the first questionnaire. Based on the response rate from a recent gynaecological survey,2 3we assumed that provision of a pen would increase the proportion of responders by 10%, from 60% to 70%. This meant that the sample size had 95% power (α = 0.05, β = 0.05). Relative response rates were calculated and statistical significance tested for a difference between proportions (independent observations) responding.
Of the 1509 consultants surveyed, 750 were randomised to receive a questionnaire only whereas 759 were to receive a pen in addition to the questionnaire (fig 1). Initially, 680 (45%) consultants responded. The relative rate of response to the questionnaire when a pen was provided compared with when it was not was 0.94 (95% CI 0.84, 1.05, p=0.3). After the reminder, an additional 350 consultants replied taking the number of responders to 1030, an overall response rate of 68%. The relative rate of response associated with provision of a pen was 0.98 (95% CI 0. 92, 1.05, p=0.6).
The inclusion of pens in a national questionnaire survey did not increase response rates. There can be several explanations for this. Negative studies are often due to type II error—that is, inability to detect a difference when one actually exists—associated with small sample size. However, our study was adequately powered. Another reason could be that, if secretarial staff opened consultants' correspondence first, it is possible that some consultants actually never received the pen along with the questionnaire. This may explain lack of an effect, but not a possible trend towards a lower response rate in the group receiving pens seen in our study. Therefore, we are confident that the ineffectiveness of this incentive in our study is not a spurious finding. Given the lack of effectiveness shown in our study and the costs associated with inclusion of a pen in a questionnaire, investigators should be deterred from using this particular strategy to improve response rates.
The authors thank Tracy Bingham, Amy Godwin, Jan Godwin, Christine Lyons, Anthony Morrison and Ian West for their help in mailing the questionnaire.
Contributors KSK generated the concept for the paper with input from TJC. TJC generated the randomisation sequence, collected the responses and analysed the data. TJC wrote the manuscript with comments from KSK and JKG. JKG is the guarantor.
Funding: University of Birmingham Interdisciplinary Research Fund and the Birmingham Women's Hospital Research and Development Programme (TJC).
Conflicts of interest: none.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.