Statistics from Altmetric.com
Response rates to postal questionnaire surveys are declining,1 particularly in areas of deprivation.2 These may limit the external and internal validity of the findings of such studies. A recent systematic review3 identified interventions that may be effective in improving response rates although no trial was identified that evaluated the effectiveness of feeding back research findings to respondents.
As part of an evaluation of the health impacts of building a traffic calming scheme, we surveyed local residents six months before (first survey) and six months after (second survey) the scheme was built. We carried out a randomised controlled trial to evaluate the effects of a study feedback information leaflet on response rates to the second postal questionnaire survey.
PARTICIPANTS, METHODS, AND RESULTS
We randomly selected 750 addresses from the study population of 2587 residents to send our first survey (see fig 1). The second survey was sent to 576 addresses after we excluded undeliverable addresses and refusals from the first survey. The same self completed postal questionnaire, which asked about sociodemographics, travel behaviour, risk perceptions, attitudes to the local area and health, was used at both stages of the study and followed by two reminders.
Half of the second survey mailings were randomly selected to receive an A5 photographically illustrated colour feedback leaflet describing self reported attitudes and behaviours before the traffic calming scheme was built. The controls were sent the leaflet after collection of questionnaires had closed. The Plain English Campaign edited the leaflet and accompanying letters to ensure clarity for a lay audience and we received a Crystal Mark (http://www.plainenglish.co.uk/crystalmark.html) of endorsement. The colour leaflet took about eight hours of a researcher’s time (DSM) to write and modify according to Plain English Campaign requirements and cost an additional £989.05 (€1549) to produce. Randomisation was carried out using SPSS software.
The response rate in the first survey to deliverable addresses was 40.2% (251 of 624). In the second survey, the response rates for those who did and did not receive an information leaflet were 30.9% (86 of 279) and 34.2% (102 of 297) respectively, although this difference was not statistically significant (OR 0.85, 95% CI 0.60 to 1.21). Those who responded to the first survey were more likely to respond to the second (OR 6.76, 95% CI 4.62 to 9.89).
Of 251 replies to the first survey, 227 agreed to be sent second surveys. Binary logistic regression was used to test hypothesised determinants of replying to the second survey. Having vested interests in the traffic calming scheme (being a car driver, living on the affected road, being elderly or having children at home) and sex were not significantly associated with response rates (see table 1).
The primary purpose of our feedback leaflet was to inform residents of the study area about the interim findings of the study in line with good research practice. We found no evidence that receiving such information increased the likelihood of returning a follow up questionnaire; it may even have reduced response rates despite its additional cost. We hypothesised that residents with vested interests in road safety might be more motivated to reply because they would be potentially most affected. None of these factors was demonstrated to affect likelihood of response.
Our sample for the leaflet intervention was self selected from an initially random address sampling frame. Low response rates to both surveys may have introduced responder biases in both samples that may have either exaggerated or attenuated true effect sizes. The study area is relatively deprived and possible reasons for low response rates may include research saturation and self disqualification.2
Efforts to engage the public in understanding science and disseminating findings to research participants should be promoted as good research practice. However, we found no evidence that feedback information leaflets are an effective way to increase response rates. More intensive methods of improving response rates may need to be considered if postal surveys are to continue to be valid in future public health research.
Funding: Mark Petticrew and Hilary Thomson are employed by the Medical Research Council and funded by the Chief Scientist Office of the Scottish Executive Department of Health. The views expressed in this article are not necessarily those of either the Chief Scientist Office or the MRC.
Copies of the questionnaires and study leaflet can be obtained from David Morrison.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.