Methods and Results

All consultants identified from the Royal College of Obstetricians and Gynaecologists (RCOG) database were sent a questionnaire with a covering letter and prepaid response envelope. The questionnaire sought views about (1) establishing a central database of outpatient hysteroscopy clinics and (2) interest in participating in a clinical trial of hysteroscopic therapy. Recipients were randomised to receive a simple plastic ballpoint pen or not in addition to the questionnaire. The randomisation sequence was computer generated and group allocation was concealed from the participants throughout the study. One reminder (with or without a pen according to the original allocation) was sent to non-responders three months after the first questionnaire. Based on the response rate from a recent gynaecological survey,2 3we assumed that provision of a pen would increase the proportion of responders by 10%, from 60% to 70%. This meant that the sample size had 95% power (α = 0.05, β = 0.05). Relative response rates were calculated and statistical significance tested for a difference between proportions (independent observations) responding.

Of the 1509 consultants surveyed, 750 were randomised to receive a questionnaire only whereas 759 were to receive a pen in addition to the questionnaire (fig 1). Initially, 680 (45%) consultants responded. The relative rate of response to the questionnaire when a pen was provided compared with when it was not was 0.94 (95% CI 0.84, 1.05, p=0.3). After the reminder, an additional 350 consultants replied taking the number of responders to 1030, an overall response rate of 68%. The relative rate of response associated with provision of a pen was 0.98 (95% CI 0. 92, 1.05, p=0.6).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Response to survey according to pen randomisation. * 78 of the consultants not practising in obstetrics and gynaecology.

Discussion

The inclusion of pens in a national questionnaire survey did not increase response rates. There can be several explanations for this. Negative studies are often due to type II error—that is, inability to detect a difference when one actually exists—associated with small sample size. However, our study was adequately powered. Another reason could be that, if secretarial staff opened consultants' correspondence first, it is possible that some consultants actually never received the pen along with the questionnaire. This may explain lack of an effect, but not a possible trend towards a lower response rate in the group receiving pens seen in our study. Therefore, we are confident that the ineffectiveness of this incentive in our study is not a spurious finding. Given the lack of effectiveness shown in our study and the costs associated with inclusion of a pen in a questionnaire, investigators should be deterred from using this particular strategy to improve response rates.