Article Text
Abstract
Background There appears to be considerable variation between different national jurisdictions and between different sectors of public policy in the use of evidence and particularly the use of randomised controlled trials (RCTs) to evaluate non-healthcare sector programmes.
Methods As part of a wider study attempting to identify RCTs of public policy sector programmes and the reasons for variation between countries and sectors in their use, we carried out a pilot study which interviewed 10 policy makers and researchers in six countries to elicit views on barriers to and facilitators of the use of RCTs for social programmes.
Results While in common with earlier studies, those interviewed expressed a need for unambiguous findings, timely results and significant effect sizes, users could, in fact, be ambivalent about robust methods and robust answers about what works, does not work or makes no difference, particularly where investment or a policy announcement was planned. Different national and policy sector cultures varied in their use of and support for RCTs.
Conclusions In order to maximise the use of robust evaluations of public programmes across the world it would be useful to examine, systematically, cross-national and cross-sectoral variations in the use of different methods including RCTs and barriers to and facilitators of their use. Sound research methods, whatever their scientific value, are no guarantee that findings will be useful or used. ‘Stories’ have been shown to influence policy; those advocating the use of RCTs may need to provide convincing narratives to avoid repetition about their value.
- Randomised trials
- policy
- barriers
- levers
- qualitative interviews
- child health
- social inequalities
- social science
- systematic reviews
- public health
- public health policy
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/3.0/ and http://creativecommons.org/licenses/by-nc/3.0/legalcode
Statistics from Altmetric.com
Footnotes
-
Funding SM is funded by the UK Medical Research Council (MRC). This piece of work was funded by the MRC Social & Public Health Sciences Unit (Reference MC_US_A540_0070) and the International Collaboration on Complex Interventions (ICCI). ICCI was funded by the Canadian Institutes of Health Research.
-
Competing interests None.
-
Ethics approval Ethics approval was obtained from the Faculty of Children and Health Research Ethics Committee at the Institute of Education, University of London.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
↵i This was a phrase used by the late Norman Glass in the course of his interview for this study. When asked to sign the consent form—he said breezily that he didn't go in for anonymity—he was happy to stand by his views. We therefore feel it appropriate to acknowledge him by name and to acknowledge the contribution he made more generally to this area, first in HM Treasury in the UK and subsequently as Chief Executive of the UK research agency NatCen.