Rapid Responses to:

RESEARCH: Christopher Rao, Omer Aziz, Sukhmeet Singh Panesar, Catherine Jones, Stephen Morris, Ara Darzi, and Thanos Athanasiou Cost effectiveness analysis of minimally invasive internal thoracic artery bypass versus percutaneous revascularisation for isolated lesions of the left anterior descending artery BMJ 2007; 334: 621 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Certainty about uncertainty

Matthias Bischof   (4 April 2007)

Handling Uncertainty

Thanos Athanasiou, Christopher Rao   (6 April 2007)

Certainty about uncertainty 4 April 2007
Matthias Bischof, research fellow Basel Institute for Clinical Epidemiology, University Hospital Basel, 4031 Basel, Switzerland Send response to journal: Re: Certainty about uncertainty	In their cost-effectiveness analysis on minimally invasive internal thoracic artery bypass vs. percutaneous revascularisation Rao et al conclude that there is no significant uncertainty associated with the incremental cost-effectiveness ratio [1]. However the results of the Monte Carlo simulation that are presented on the cost-effectiveness plane (Fig 3) show enormous variation on both axes and weak correlation. The authors claim a 95% confidence interval (CI) for the cost-effectiveness estimate of £5008/QALY to £8505/QALY. This interval was obtained by applying Fieller’s theorem. It is unclear which data the authors analysed and whether these included variations in outcomes between individual patients in the model (first order uncertainty) and/or parameter uncertainty (uncertainty in model input parameters, i.e. 2nd order uncertainty)[2]. Once the joint density of incremental costs and effects is available (i.e. the cost and effect pairs; Fig 3) the CI can be obtained without applying Fieller’s theorem. I recalculated the CI with a digitalised dataset obtained from figure 3. When doing so I find that the 95% CI ranges from £1146/QALY to dominated (i.e. less effective and more costly; at a 5% significance level it is uncertain whether the intervention under analysis is cost-effective or not). This conclusion is also supported by the cost-effectiveness acceptability curve (Fig 4)[3]. The 95% CI is undefined for willingness to pay values between £0 and £50 000 (Fig 4)[4]. It is likely that Rao et al by applying Fieller’s theorem or by using the wrong data underestimate the size of the CI. Since Rao et al’s findings may have important implications [1], it is important to exactly determine the degree of uncertainty with regards to the cost-effectiveness of minimally invasive internal thoracic artery bypass surgery. 1. Rao C, Aziz O, Panesar SS, Jones C, Morris S, Darzi A et al. Cost effectiveness analysis of minimally invasive internal thoracic artery bypass versus percutaneous revascularisation for isolated lesions of the left anterior descending artery. BMJ 2007;334:621. 2. Briggs AH. Handling uncertainty in cost-effectiveness models. Pharmacoeconomics 2000;17:479-500. 3. Fenwick E, O'Brien BJ, Briggs A. Cost-effectiveness acceptability curves--facts, fallacies and frequently asked questions. Health Econ. 2004;13:405-15. 4. Briggs A. Handling uncertainty in economic evaluations and presenting the results. Economic evaluation in health care. Merging theory with practice, Oxford University Press, 2001. Competing interests: None declared
Handling Uncertainty 6 April 2007
Thanos Athanasiou, Consultant Cardiac Surgeon St Mary's Hospital, Christopher Rao Send response to journal: Re: Handling Uncertainty	We used Fieller’s method, accounting for both first and second order uncertainty [1]. We also considered several other methods to quantify the uncertainty associated with the incremental cost effectiveness ratio (ICER), although we rejected the confidence-box method because of its methodological limitations [2] and bootstrap methods because a large amount of the efficacy data in our analysis was clustered around the origin, which can result in spurious results when uncertainty is quantified using these methods [3]. Whilst the intervals given by Fieller’s method appear to be incongruous with the cost-effectiveness acceptability curve [figure 4] [4], this highlights the methodological difficulty in assigning conventional 95% confidence intervals to the incremental cost effectiveness ratio [4]. The conclusion that MIDCAB "may" provide more favorable long term outcomes than stenting, and "could" be more a cost effective alternative to Stenting is clearly based on the estimates of uncertainty detailed in the first part of the discussion, derived from the cost effectiveness acceptability curve [Figure 4]. We feel that the cost-effectiveness acceptability cure is the most robust method of quantifying the uncertainty associated with our decision analytical model. Unlike the full version, the shortened version of this paper that appeared in the print edition of the BMJ does state without qualification that there was no significant uncertainty associated with the ICER, however the reader is referred at this point to the full article where further explanation of this is available. Finally, it is not possible to comment either on the validity of Dr Bischof’s synthesized data set or on the methodology he used in his analysis based on the contents of Dr Bischof’s letter, however we thank him for clarifying an aspect of our study that some readers may find confusing. 1. Heitjan DF. Fieller's Method and Net Health Benefits. Health Econ. 9: 327–335 (2000) 2. Briggs AH (2001): Handling uncertainty in economic evaluation and presenting the results, chapter 8 in Drummond & McGuire (eds.) Economic evaluation in health care, Oxford University Press: 3. Briggs AH, Wonderling DE, Mooney CZ. Pulling cost-effectiveness analysis up by its bootstraps: A non-parametric approach to confidence interval estimation. Health Econ. 6: 327–340 (1997) 4. Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddard GL. Methods for the Economic Evaluation of Health Care Programmes. Third ed: Oxford University Press, 2004. Competing interests: None declared