Background Surveys play an important role in providing public health data for researchers and policy-makers. Traditional interviewer-administered surveys are subject to declining response rates and increasing costs. With the spread of the internet among the general population, web surveys potentially provide a cost-effective alternative mode. Volunteer web panels are now widely used for market research and opinion polling, but less for academic and government research due to concerns about their representativeness. Various methods attempt to make web panels more “representative” of the general population. We compared results from four web panels with a national probability sample survey (Natsal3).

Methods A shortened Natsal3 questionnaire was included on four web panels: two used standard demographic quota controls, and two were “modified” using variables correlated with key outcomes as additional quota controls. After weighting for age and sex, comparisons were made with Natsal3 for demographic characteristics, key behaviour outcomes and attitudes, in order to examine whether surveys using modified quotas would lead to “improved” results.

Results All four web panels gave significantly different results from Natsal3 on a majority of the variables. Compared with Natsal3, there were more differences among men than women for all the web panels. There were more differences between the web panels and Natsal3 questions asked in CAPI (Computer Assisted Personal Interview) than in CASI (Computer Assisted Self Interview) format. The web panels also differed significantly from each other. One of the modified quota web panels produced estimates closer to Natsal3 than the standard quota panels, but still significantly differed on three-fifths of the variables. Moreover, meeting the modified quotas proved difficult, and the quotas had to be relaxed for both web panels.

Conclusion When measuring sensitive sexual behaviours in the British population, volunteer web panels provided significantly different estimates than a probability CAPI/CASI survey. Modifying web panel quota controls did not lead to much improvement.