METHODS

Study populations

The trial was carried out with two separate Australian female samples: current office workers at a university (Sample A) and patients seen by a consultant in rehabilitation medicine between January 1980 and November 1984 (Sample B). These samples were also used to test the acceptability and appropriateness of a questionnaire to assess past and existing symptoms of work related neck and upper body disorders, as well as the long term consequences of such disorders on the health, occupational and socioeconomic status of those women who had been affected (30% of Sample A and all of Sample B). Sample B was also used to assess accuracy of recall of onset of symptoms, of the body areas affected and of the diagnosis given by a doctor after more than 15 years. These studies were pilots for a long term follow up investigation of a large historical cohort of female office workers employed in the early 1980s during an “epidemic” of work related neck and upper body disorders. The studies were approved by the Australian National University Human Research Ethics Committee.

The first study sample (Sample A) comprised 200 women randomly selected from all current female office workers at an Australian university (n=399). A simple random list generator facility in the Epi Info software¹¹ was used for the random selection. The second sample (Sample B), which consisted of women who had developed work related neck and upper limb disorders between January 1980 and November 1984, was selected from the records of a consultant in rehabilitation medicine with a practice in the Australian Capital Territory. A screening of 60% of all records kept by the consultant allowed 129 cases to be identified, but current addresses could be found for only 92 (71%) cases using telephone directories and the Australian Electoral Roll.

The questionnaire

The questionnaire, which was sent to both samples in September 1999, contained 317 questions about musculoskeletal symptoms, sociodemographic characteristics, general health, symptoms of depression, health services utilisation, tobacco and alcohol consumption, social support, occupational history, and job satisfaction. Pre-testing showed that completion time varied substantially, with a mean of 1.5 hours (range 0.5 to 2.1 hours). The questionnaire was sent with a cover letter and a reply paid return envelope. Informed consent was considered to have been granted by the act of returning a completed questionnaire.

The follow up methods

Figures 1 and 2 show the design of the follow up protocol used and the response rates for Samples A and B. For both samples, a reminder letter was posted after two weeks and after another two weeks non-respondents were sent a second copy of the questionnaire and randomly allocated (within study samples) to one of two groups of approximately equal size. One group (n=72, combined samples) was randomised to receive a reminder telephone call soon after the second questionnaire was mailed, with a subsequent final reminder letter if they had not responded within two weeks. The other group (n=71, combined samples) received only a final reminder letter if they did not respond two weeks after the second questionnaire had been dispatched. No material incentives were used. Telephone numbers were obtained using an electronic telephone directory.

Two researchers using a standardised script implemented the telephone contact. The message focused on the relevance of the study for informing working women of the long term impact of work related neck and upper body disorders and on reiterating the importance of participation to enhance the study findings. Up to five attempts were made to reach the selected women at home. All calls occurred out of office hours, including weekends. Arrangements of the most suitable time to contact women were sought when the telephone was answered by a third party. Messages left on answering machines stated only the identity of the researchers. Cumulative response rates for each group (within samples) were analysed after the survey cut off date both by intention to phone and by telephone contact actually made to determine whether significant differences in responses could be detected.

Response analysis

Differences in responses were calculated using the χ² statistic with Yates's correction. Relative risks for decrease in response rates were also calculated for each of the study samples. The Mantel-Haenszel weighted relative risk across the two samples was then calculated for response rates, and the Breslow-Day test for homogeneity was used to assess heterogeneity across the groups.

Cost analysis

All direct expenses incurred in carrying out the survey protocol and telephone reminders were recorded. Fixed costs included the printing of the questionnaires and reply paid envelopes, postage, and labour involved in mailouts. Variable costs were associated with telephone charges and labour spent on telephone calls. Most calls were local because all women in Sample A and 60% of the women in Sample B were living in the city where the study was conducted. All costs were calculated using standard Australian rates. Infrastructure costs were not included.

The base fixed mailing costs before randomisation were divided by the number of respondents in Samples A and B. After randomisation, the fixed mailing costs incurred were calculated separately for each group, and the variable costs were calculated for the intervention group.

RESULTS

Respondent characteristics

Respondents in Samples A and B were similar in age, marital status and educational level. Overall 63% were 45 years or older, 73% were married or in de facto relationships, and 59% had post secondary school qualifications. However, women in Sample B were more likely to be retired (40% compared with 0%) and to have a lower income (43% had a yearly gross family income of $A 30 000 or less compared with 6%). All of the women in Sample B who were still employed were also office workers.

The randomised groups did not differ with respect to age, the only variable known before randomisation. Among respondents in each sample there was no statistically significant difference at p=0.05 level in age, marital, educational, occupational and retirement status, or income between those in the intervention (telephoned) and control (not telephoned) groups.

Completion rates

Before randomisation, 96 (48%) of Sample A and 16 (17%) of Sample B returned completed questionnaires. At this stage, 19 (10%) and 18 (20%) questionnaires, respectively, were undeliverable or refused. For Sample A, the intervention group (those to be contacted by telephone) consisted of 43 women, with 42 women in the control group. Of the 43 women, 40 (93%) were successfully contacted. Analysed by intention to phone, the completion rates were 47% for the telephoned group versus 21% for the control group (relative risk 2.17, confidence intervals 1.12 to 4.21, p=0.027; table 1). Of those telephoned, 16% were definite refusals (rather than non-responses), compared with 10% for those not telephoned.

For Sample B, 29 women were randomly allocated to each group. Twenty six (90%) were reached by telephone. Analysed by intention to phone, the completion rates obtained were 38% for those telephoned versus 10% for those not telephoned (relative risk 3.67, confidence intervals 1.14 to 11.79, p=0.031; table 1). Forty one per cent of those telephoned were definite refusals, compared with 10% of those not telephoned.

A Mantel-Haenszel weighted relative risk across the two samples and groups was calculated, as there was no significant heterogeneity across these strata. The results were 2.54 (confidence intervals 1.43 to 4.52) for analysis by intention to phone and 2.67 (confidence intervals 1.53 to 4.65) for analysis by successful telephone contact (table 1).

Completion costs

Total costs and costs per respondent are shown in table 2. For the 112 women (combined samples) who returned completed questionnaires before randomisation, the average cost per respondent was $A14. For those who responded after the questionnaire was mailed a second time, the costs were as follows. For the group randomised to receive a telephone reminder, the marginal cost per respondent was $A27. For the control group, the marginal cost per respondent was $A32. The total cost was higher for the intervention group ($A851) compared with the control group ($A386), but the significantly higher number of completed responses (31 versus 12) resulted in a lower marginal cost per respondent.

DISCUSSION

These findings clearly show that a telephone reminder to non-respondents during the middle stage of data collection is an effective strategy to significantly improve response rates to postal surveys.

This was found in two populations recruited in different settings and with different completion rates (48% before randomisation for Sample A compared with 17% before randomisation for Sample B). If the data collection protocol had not included a telephone reminder at all, a 57% final completion rate could have been expected for Sample A, whereas with a telephone reminder to all women (with the same success in making contact), a 68% completion rate was likely to have been achieved. The analogous rates for Sample B are 24% and 41%. These estimations suggest that the improvement in response rate through a telephone reminder is larger in a group with a low initial response rate (Sample B) than in a group with a high initial response rate (Sample A), although there was no significant heterogeneity across the two sample strata. However, because of the small numbers in Sample B, further investigation of this hypothesis is warranted.

The unambiguously stated refusal rates, as compared with non-response rates, were also increased among the telephoned group, particularly for Sample B. This permits a clearer assessment of possible response biases, which must then be taken into account in the interpretation of the results. The high refusal rate in Sample B made us aware of the possible strength of feeling about their condition in this group, which is being further investigated.

The marginal cost per respondent was substantially lower (15% from the time of randomisation at the mailout of the second questionnaire) for the telephoned group. The precise costs will vary, of course, depending on the success of finding telephone numbers and of making contact.

A potential limitation to the generalisability of the results of this trial is that only women were recruited. It is not possible to know whether the results would also hold for men. The samples were drawn from office workers who are relatively well educated, with good English language skills. Again it is not possible to know how generalisable the results are to groups with different occupational, educational, and language status.

These results show that adding a telephone reminder soon after the second mail out of the questionnaire in a mail health survey is a cost effective way of significantly increasing response rates from relatively well educated female populations.