Cost effectiveness calculations and sample size

This is one of an occasional series of notes on economics

Clinical trials should be large enough to detect a clinically important difference between two treatments. Yet a clinically important difference is often difficult to define and debatable. This difficulty may explain why so few published trials report the clinical reasoning underlying their sample size selection. Furthermore, clinical investigators are often suspected of approaching sample size calculations logistically rather than clinically: they estimate the number of patients who can be recruited into a trial and then ask a statistician to justify the sample size by calculating the “detectable” difference implied by the number of recruitable patients.1 Including economic criteria to aid sample size determination for clinical trials might improve the rigour of sample size selection. 2 3

Consider a recent randomised trial comparing the effectiveness of hysterectomy with hysteroscopic surgery for treating menorrhagia. 4 5 The trial found that hysteroscopic surgery was an acceptable alternative to hysterectomy but had a considerable retreatment rate. It was also unclear which method of hysteroscopic surgery was most effective. Therefore, a further randomised trial compared endometrial laser ablation with transcervical endometrial resection.6 If both techniques are equally safe a key outcome measure is the retreatment rate. However, what difference in retreatment rates should the clinical trial have been designed to detect?

One method of answering this question is to examine the cost differences between the two procedures. Endometrial laser ablation costs £772 per procedure, while transcervical endometrial resection is £727,5 principally because of lower equipment costs. Thus, assuming a retreatment rate of 27% for both groups, there is an increased cost of £5715 for every 100 operations for endometrial laser ablation (table). This cost could be offset if endometrial laser ablation reduced the retreatment rates by about 8 in every 100 (5715/772). Given that an earlier trial of hysteroscopic surgery showed that 27% of patients needed retreatment after one year, in the form of either a hysterectomy or further hysteroscopic surgery,4 this implies that for endometrial laser ablation to be more cost effective than transcervical endometrial resection a reduction in retreatment rates to 19% or less (27%−8%) is required.

It is worth emphasising that 8% is the minimum difference that is economically important. The true difference required for endometrial laser ablation to be more cost effective will probably be even smaller, given the avoidance of negative health effects associated with retreatment.

Using these retreatment rates, the sample size requirement based on economic importance can now be calculated. Assuming we wish to conduct a trial that has an 80% power to detect a 8% difference between 19% and 27%, for a two tailed P value of 5% we need 435 patients in each treatment group.7

Although retreatment rate is clearly an important outcome, the trial has recruited only 350 patients with the aim of detecting a 15% difference in patient satisfaction rates.6 Given its relatively small size, this trial will have only an 80% power to detect a 12% difference in retreatment rates (with a 5% significance level). Indeed, the trial has shown that there is a 4% difference in favour of endometrial laser ablation, but the 95% confidence interval of the difference (−4% to 11%) does not exclude the possibility that there could be an 8% improvement in retreatment rates for endometrial laser ablation.6

Though it is not always possible to set sample sizes by economic criteria, economics can often usefully inform sample size calculations. For example, the minimum economic sample size for a clinical trial of thiazide diuretics for preventing hip fractures should be large enough to detect a 10% reduction in fracture rates as this is the point where cost savings due to averting hip fractures equal the costs of the intervention.2

Another point of economic importance might be where the cost effectiveness ratio is equal to that of the next best alternative treatment. For instance, a sample size calculation for a clinical trial of in vitro fertilisation compared with tubal surgery for treating infertility suggested that for the cost effectiveness ratios of the two treatments to be equal, then in vitro fertilisation must result in 12% more live births than tubal surgery.2

In conclusion, more rigour is required in trial design to capture differences that would be of economic as well as clinical importance. Sometimes relatively simple calculations can aid sample size calculations for controlled trials.