Rapid Responses to:

PAPERS: Dan Greenberg, Allison B Rosen, Natalia V Olchanski, Patricia W Stone, John Nadai, and Peter J Neumann Delays in publication of cost utility analyses conducted alongside clinical trials: registry analysis BMJ 2004; 328: 1536-1537 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Reporting of economic evaluations alongside clinical trials

Waseem Sharieff   (27 June 2004)

Rapid publication cannot compensate for unreliable economic data

Damian G Walker, Godfrey J A Walker   (14 July 2004)

Further evidence on the delay

Boyka A Stoykova   (14 October 2004)

Reporting of economic evaluations alongside clinical trials 27 June 2004
Waseem Sharieff, PhD Candidate The Hospital for Sick Children, 555 University Avenue, Toronto, M5G 1X8 Send response to journal: Re: Reporting of economic evaluations alongside clinical trials	Dr Greenberg and colleagues find delay in reporting of results of economic evaluations carried out alongside clinical trials; a phenomenon that may impede timely decision making (1). I share the authors’ view that delays often occur because of researchers’ higher enthusiasm in reporting their clinical findings and the difficulty in extrapolating these findings beyond the trial setting and time horizon. However, first, delays are natural as for a treatment not shown to be effective (in a trial) there is no point of doing an economic evaluation. Second, even when shown to be effective, economic evaluations are of little value for informing decisions on resource allocation, as a common measure of treatment effect for all diseases does not exist. For example, a decision maker with a limited budget may find it difficult to determine how much funds to allocate for curable diseases (cost effectiveness = cost X number needed to treat per cure) compared to incurable ones (cost utility = cost X number needed to treat per quality adjusted life year gained) while measures of quality of life and willingness to pay have their own shortcomings (2,3). Third, it can be shown algebraically that cost effectiveness of an intervention for a given disease is inversely proportional to the prevalence of that disease; an intervention shown to be cost-effective in one setting may not be such in another setting. Therefore, results of economic evaluations should be applied with caution and constructing a decision analytic model for one’s own setting might be a better option. 1. Greenberg D, Rosen A, Olchanski N, Stone P, Nadai J, Neumann P. Delays in publication of cost utility analyses conducted alongside clinical trials: registry analysis. BMJ 2004;328:1536 –7. 2. Ubel PA, Loewenstein G, Jepson C. Whose quality of life? A commentary exploring discrepancies between health state evaluations of patients and the general public. Qual Life Res. 2003;12(6):599-607. 3. Robinson R. Cost-benefit analysis. BMJ 1993;307:924-6. Competing interests: None declared
Rapid publication cannot compensate for unreliable economic data 14 July 2004
Damian G Walker, Lecturer in Health Economics London School of Hygiene & Tropical Medicine, WC1E 7HT, Godfrey J A Walker Send response to journal: Re: Rapid publication cannot compensate for unreliable economic data	Editor Greenberg et al.(1) conclude that “… reliable economic data are usually not available, at least in peer reviewed journals, for decision- makers when decisions ... are typically made.” Their findings are of concern to those of us who believe not just in evidence-based medicine, but evidence-based policy more generally. However, to some extent we consider that the authors have missed the point and furthermore perhaps misled readers, by suggesting that the economic data reported in the 41 cost-utility analyses identified were “reliable”. It may be that the authors’ 41 cost-utility analyses are examples of ‘best practice’ among the several thousand economic evaluations published over the past few decades (2, 3), but the reliability of the economic data, we believe, is the most important issue. If economic articles are of poor quality, and therefore of little use to decision-makers, it is irrelevant when they are published. Two of the authors published a review of the quality of reporting in 228 cost-utility analyses (4), including many of the 41 analyses included in this paper. They concluded that the quality of economic analyses was variable. Therefore the usefulness of many economic evaluations in making resource allocation choices has to be questioned irrespective of when they were published. Nevertheless, assuming reliable economic data, it would have been very interesting to know of examples of decisions being reversed, based on economic evaluations of the results of clinical trials, which would have illustrated the importance of rapid publication (assuming reliable economic data). For example, are the authors aware of any instances where a trial showed that a new technology was more effective than existing practice and a decision was thus made to replace the status quo, and yet the subsequent cost-utility analysis illustrated that the new technology was less cost-effective vis-à-vis existing practice (e.g. according to the decision-maker’s implicit willingness to pay per QALY gained) and therefore the decision was reversed? Such examples would clearly illustrate the importance of rapid publication. Guidelines for submissions of cost-utility studies will hopefully help to improve the quality of economic evaluations published alongside clinical trials (5). But ultimately, it should be stressed that rapid publication cannot compensate for unreliable economic data. 1. Greenberg D, Rosen AB, Olchanski V, Stone PW, Nadai J, Neumann PJ. Delays in publication of cost-utility analyses conducted alongside clinical trials: registry analysis. BMJ 2004; 328: 1536-1537. 2. Elixhauser A, Luce BR, Taylor WR, Reblando J. Health care CBA/CEA: an update on the growth and composition of the literature. Med Care 1993; 31(7 Suppl):JS1-11, JS18-149. 3. Elixhauser A, Halpern M, Schmier J, Luce BR. Health care CBA and CEA from 1991 to 1996: an updated bibliography. Med Care 1998; 36(5 Suppl):MS1-9, MS18-147. 4. Neumann PJ, Stone PW, Chapman RH, Sandberg EA, Bell CM. The quality of reporting in published cost-utility analyses, 1976-1997. Ann Intern Med 2000; 20;132(12):964-72. 5. Jefferson T, Smith R, Yee Y, Drummond M, Pratt M, Gale R. Evaluating the BMJ guidelines for economic submissions: prospective audit of economic submissions to BMJ and The Lancet. JAMA 1998; 15;280(3):275-7. Competing interests: None declared
Further evidence on the delay 14 October 2004
Boyka A Stoykova, Research Fellow in Health Economics Health Economics Research Group, Brunel University, Uxbridge UB8 3PH, UK Send response to journal: Re: Further evidence on the delay	Greenberg et al. report delays in the publication of cost-utility analyses conducted alongside clinical trials. A paper by Stoykova et al.(1) suggests that if the delay is measured for the available effectiveness and cost-effectiveness evidence of new drugs the findings are even more depressing. Since these drugs were being assessed by policy makers in health care, decision in many cases had to be made in the absence of any economic evidence. Reference: 1. Stoykova B, Drummond M, Barbieri M, Kleijnen J (2003) The lag between effectiveness and cost-effectiveness evidence of new drugs. Implications for decision-making in health care, European Journal of Health Economics, vol 4, pp 313-8 Accessible at: http://springerlink.metapress.com/link.asp?id=pp4nlp1wg94wrgd5 Competing interests: None declared