Background For randomised controlled trials (RCT), the sample size needed to detect an important effect on clinical outcome is commonly believed to be insufficient to firmly establish the efficiency of the intervention.

Objectives To review cost-utility analyses (CUAs) conducted alongside RCTs to determine: (1) if cost-effectiveness is considered in sample size calculations, (2) the frequency with which economic conclusions conflict with clinical conclusions and (3) whether economic evaluations are underpowered and so more likely to come to indeterminate results.

Methods We searched the National Health Service (NHS) Economic Evaluation Database and identified 717 articles. We extracted data from nine high impact/volume journals and from a 50% random sample of the remaining journals that published 3 or more CUAs (n=302). 264 were excluded because they were models (235), had insufficient information (16), failed to measure individual patient data (11) or were not RCTs (2). 38 articles (40 RCTs) were included. Information was collected on study characteristics, primary clinical outcomes, Quality Adjusted Life Years (QALY) and incremental costs. We categorised trials according to the strength of their conclusions on clinical and cost per QALY outcomes.

Results Of 24 RCTs analysed to date, only 1/24 (4%) considered economic factors in sample size calculation. 12/24 (50%) studies reported evidence of one intervention being more effective based on the primary clinical outcome (p<0.05). Fewer studies provided evidence of differences between treatments when using the QALY outcome (8/24; 33%) or cost per QALY (6/24; 25%). In 2/24 (8%) studies, conclusions about the ‘optimal’ intervention strategy, based on the primary clinical outcome, were partially reversed once cost-effectiveness data were taken into consideration. We calculated the median power to detect a minimum important difference for 7 studies with sufficient information and found: primary clinical outcome 77.7%, QALY 31.2% and costs 25.9%.

Conclusions Based on preliminary analysis, economic factors rarely feature in sample size calculations. There was an occasional discrepancy between cost-effectiveness and clinical conclusions but no complete reversal of interpretation. CUAs were more likely to come to indeterminate conclusions. This suggests that RCTs may often cease recruitment before the efficiency of the intervention can be firmly established and therefore only provide incomplete evidence to policy makers about the cost-effectiveness of healthcare technologies.