Intended for healthcare professionals

Rapid response to:

Education And Debate Economics notes

Cost effectiveness calculations and sample size

BMJ 2000; 321 doi: https://doi.org/10.1136/bmj.321.7262.697 (Published 16 September 2000) Cite this as: BMJ 2000;321:697

Rapid Response:

Demonstrating cost effectiveness requires larger sample

The article by Torgerson and Campbell [1] suggests using
the total cost break-even point as the hypothetical effect
size for the purpose of power and sample size calculations.
While this criterion appears as good as many others, it
could lead to the mistaken conclusion that a study designed
in this way can test cost efectiveness.

In fact, if the sample size is chosen as in [1] with a
type 1 error rate of 5%, then the power to detect lower
overall cost is only 2.5% at the same 5% level!

Take the example of comparing endometrial laser ablation
with transcervical endometrial resection, as discussed
in [1]. The proposed study design with 435 patients per arm
has a power of 80% to detect a significantly lower rate
of re-treatment after laser ablation than the 27% observed
for transcervical resection. However to establish lower
overall cost, we must show that the re-treatment rate is
significantly lower than the break-even point of 19%.
The probability of observing a rate significantly lower
than 19% under the hypothesis that the rate is 19% is,
unsurprisingly, very low. In fact it is precisely half
the type 1 error rate of 5%.

So the proposed cost-effectiveness criterion for selecting
a hypothetical effect size cannot be used for testing
cost effectiveness. This leaves its rationale looking
rather slight. In fact I support the "logistic" procedure
that the authors appear to denigrate in [1]: calculating
the effect size which yields a practical sample size.
This is an effective way of reducing the arcane logic
of power calculations to a parameter that a clinician
can use to decide if the trial should proceed.

I believe this is compatible with Goodman's agenda [2] to
incorporate clinical understanding into medical
statistics. It is quite different to using post-hoc
power calculations to explain away negative results [3].

Regards,

Graham Byrnes

[1] Torgerson DJ, Campbell MK. Cost effectiveness
calculations and sample size. BMJ 2000;321:697.

[2] Goodman SN. Towards evidence-based medical statistics.
1: The p-value fallacy. Ann Intern Med 1999;130:
995-1004.

[3] Goodman SN. The use of predicted confidence intervals
when planning experiments and the misuse of power
when interpreting the results. Ann Intern Med 1994;121:
200-206.

Competing interests: No competing interests

15 September 2000
Graham Byrnes
Biostatistician
Victorian Infectious Diseases Reference Lab