BMJ 1998;317:1195-1200 ( 31 October )

Papers

Analysis and interpretation of cost data in randomised controlled trials: review of published studies

Julie A Barber, research fellow in medical statistics,  Simon G Thompson, professor of medical statistics and epidemiology. 

Department of Medical Statistics and Evaluation, Imperial College School of Medicine, London W12 0NN

Correspondence to: Ms Barber j.barber{at}rpms.ac.uk

Objective To review critically the statistical methods used for health economic evaluations in randomised controlled trials where an estimate of cost is available for each patient in the study.
Design Survey of published randomised trials including an economic evaluation with cost values suitable for statistical analysis; 45 such trials published in 1995 were identified from Medline.
Main outcome measures The use of statistical methods for cost data was assessed in terms of the descriptive statistics reported, use of statistical inference, and whether the reported conclusions were justified.
Results Although all 45 trials reviewed apparently had cost data for each patient, only 9 (20%) reported adequate measures of variability for these data and only 25 (56%) gave results of statistical tests or a measure of precision for the comparison of costs between the randomised groups. Only 16 (36%) of the articles gave conclusions which were justified on the basis of results presented in the paper. No paper reported sample size calculations for costs.
Conclusions The analysis and interpretation of cost data from published trials reveal a lack of statistical awareness. Strong and potentially misleading conclusions about the relative costs of alternative therapies have often been reported in the absence of supporting statistical evidence. Improvements in the analysis and reporting of health economic assessments are urgently required. Health economic guidelines need to be revised to incorporate more detailed statistical advice.

Key messages

  • Health economic evaluations required for important healthcare policy decisions are often carried out in randomised controlled trials

  • A review of such published economic evaluations assessed whether statistical methods for cost outcomes have been appropriately used and interpreted

  • Few publications presented adequate descriptive information for costs or performed appropriate statistical analyses

  • In at least two thirds of the papers, the main conclusions regarding costs were not justified

  • The analysis and reporting of health economic assessments within randomised controlled trials urgently need improving



© BMJ 1998
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to StumbleUpon StumbleUpon   Add to Technorati Technorati    What's this?

Relevant Articles

Importance of health economics must be recognised when trials are designed
Alan Haycox, Adrian Bagust, and Tom Walley
BMJ 1999 318: 1696. [Extract] [Full Text]

Fifty years of randomised controlled trials
BMJ 1998 317: 0. [Full Text] [PDF]

This article has been cited by other articles:

  • Byford, S., Barrett, B., Roberts, C., Wilkinson, P., Dubicka, B., Kelvin, R. G., White, L., Ford, C., Breen, S., Goodyer, I. (2007). Cost-effectiveness of selective serotonin reuptake inhibitors and routine specialist care with and without cognitive behavioural therapy in adolescents with major depression. Br. J. Psychiatry 191: 521-527 [Abstract] [Full text]  
  • BYFORD, S., BARRETT, B., ROBERTS, C., CLARK, A., EDWARDS, V., SMETHURST, N., GOWERS, S. G. (2007). Economic evaluation of a randomised controlled trial for anorexia nervosa in adolescents. Br. J. Psychiatry 191: 436-440 [Abstract] [Full text]  
  • McCahon, D, Murray, E T, Jowett, S, Sandhar, H S, Holder, R L, Hussain, S, O'Donoghue, B, Fitzmaurice, D A (2007). Patient self management of oral anticoagulation in routine care in the UK. J. Clin. Pathol. 60: 1263-1267 [Abstract] [Full text]  
  • Callaghan, R., Prabu, A., Allan, R. B., Clarke, A. E., Sutcliffe, N., Pierre, Y. St., Gordon, C., Bowman, S. J., the UK Sjogren's Interest Group, (2007). Direct healthcare costs and predictors of costs in patients with primary Sjogren's syndrome. Rheumatology (Oxford) 46: 105-111 [Abstract] [Full text]  
  • Polsky, D., Doshi, J. A., Bauer, M. S., Glick, H. A. (2006). Clinical Trial-Based Cost-Effectiveness Analyses of Antipsychotic Use. Am. J. Psychiatry 163: 2047-2056 [Abstract] [Full text]  
  • Halpin, D. M. G. (2006). Health Economics of Chronic Obstructive Pulmonary Disease. Proc Am Thorac Soc 3: 227-233 [Abstract] [Full text]  
  • Pritchard, C., Duffy, S., Edington, J., Pang, F. (2006). Enteral Nutrition and Oral Nutrition Supplements: A Review of the Economics Literature. JPEN J Parenter Enteral Nutr 30: 52-59 [Abstract] [Full text]  
  • Stasi, R., Amadori, S., Newland, A. C., Provan, D., Casadevall, N., Durieux, P., Dreyfus, F., Fenaux, P. (2004). Comparison of rHuEpo plus rHuG-CSF and supportive care: apples to oranges. Blood 104: 2201-2202 [Full text]  
  • Casadevall, N., Durieux, P., Dubois, S., Hemery, F., Lepage, E., Quarre, M.-C., Damaj, G., Giraudier, S., Guerci, A., Laurent, G., Dombret, H., Chomienne, C., Ribrag, V., Stamatoullas, A., Marie, J.-P., Vekhoff, A., Maloisel, F., Navarro, R., Dreyfus, F., Fenaux, P. (2004). Health, economic, and quality-of-life effects of erythropoietin and granulocyte colony-stimulating factor for the treatment of myelodysplastic syndromes: a randomized, controlled trial. Blood 104: 321-327 [Abstract] [Full text]  
  • Gilbert, F. J., Grant, A. M., Gillan, M. G. C., Vale, L. D., Campbell, M. K., Scott, N. W., Knight, D. J., Wardlaw, D. (2004). Low Back Pain: Influence of Early MR Imaging or CT on Treatment and Outcome--Multicenter Randomized Trial. Radiology 231: 343-351 [Abstract] [Full text]  
  • Thomas, R., Grimshaw, J., Mollison, J, McClinton, S, McIntosh, E, Deans, H, Repper, J (2003). Cluster randomized trial of a guideline-based open access urological investigation service. Fam Pract 20: 646-654 [Abstract] [Full text]  
  • Bower, P., Byford, S., Barber, J., Beecham, J., Simpson, S., Friedli, K., Corney, R., King, M., Harvey, I. (2003). Meta-analysis of data on costs from trials of counselling in primary care: using individual patient data to overcome sample size limitations in economic analyses. BMJ 326: 1247-1250 [Abstract] [Full text]  
  • Cooper, N. J., Sutton, A. J., Mugford, M., Abrams, K. R. (2003). Use of Bayesian Markov Chain Monte Carlo Methods to Model Cost-of-Illness Data. Med Decis Making 23: 38-53 [Abstract]  
  • Kenkre, J. E, Allan, T. F, Tobias, R. S, Parry, D. J, Bryan, S., Carter, Y. H (2002). Breaking bones, breaking budgets: a clinical and economic evaluation of a prospective, randomized, practice controlled, intervention study in the prevention of accidents in primary care. Fam Pract 19: 675-681 [Abstract] [Full text]  
  • Whynes, D.K. (2002). Health care costs .... Eur Heart J 23: 1237-1239  
  • Creed, F., Morgan, R., Fiddler, M., Marshall, S., Guthrie, E., House, A. (2002). Depression and Anxiety Impair Health-Related Quality of Life and Are Associated With Increased Costs in General Medical Inpatients. Psychosomatics 43: 302-309 [Abstract] [Full text]  
  • Zhou, X.-H. (2002). Inferences about population means of health care costs. Stat Methods Med Res 11: 327-339 [Abstract]  
  • Goodacre, S, McCabe, C (2002). Being economical with the truth: how to make your idea appear cost effective. Emerg. Med. J. 19: 301-304 [Abstract] [Full text]  
  • Cooper, N. J., Mugford, M., Symmons, D. P. M., Barrett, E. M., Scott, D. G. I. (2002). Total costs and predictors of costs in individuals with early inflammatory polyarthritis: a community-based prospective study. Rheumatology (Oxford) 41: 767-774 [Abstract] [Full text]  
  • Jefferson, T., Demicheli, V., Vale, L. (2002). Quality of Systematic Reviews of Economic Evaluations in Health Care. JAMA 287: 2809-2812 [Abstract] [Full text]  
  • Brown, C A, Belfield, C R, Field, S J (2002). Cost effectiveness of continuing professional development in health care: a critical review of the evidence. BMJ 324: 652-655 [Full text]  
  • Benade, M. M., Warlow, C. P. (2002). Cost of Identifying Patients for Carotid Endarterectomy. Stroke 33: 435-439 [Abstract] [Full text]  
  • Benade, M. M., Warlow, C. P. (2002). Costs and Benefits of Carotid Endarterectomy and Associated Preoperative Arterial Imaging: A Systematic Review of Health Economic Literature. Stroke 33: 629-638 [Abstract] [Full text]  
  • Lee, A., Gin, T. (2002). Applying the Results of Quantitative Systematic Reviews to Clinical Practice. Anesth. Analg. 94: 372-377 [Full text]  
  • Murman, D. L. (2001). The Costs of Caring: Medical Costs of Alzheimer's Disease and The Managed Care Environment. J Geriatr Psychiatry Neurol 14: 168-178 [Abstract]  
  • BYFORD, S., BARBER, J. A., FIANDER, M., MARSHALL, S., GREEN, J. (2001). Factors that influence the cost of caring for patients with severe psychotic illness: Report from the UK 700 trial. Br. J. Psychiatry 178: 441-447 [Abstract] [Full text]  
  • Sutcliffe, N., Clarke, A. E., Taylor, R., Frost, C., Isenberg, D. A. (2001). Total costs and predictors of costs in patients with systemic lupus erythematosus. Rheumatology (Oxford) 40: 37-47 [Abstract] [Full text]  
  • Bower, P., Byford, S., Sibbald, B., Ward, E., King, M., Lloyd, M., Gabbay, M. (2000). Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy, and usual general practitioner care for patients with depression. II: Cost effectiveness. BMJ 321: 1389-1392 [Abstract] [Full text]  
  • Harrington, R., Peters, S., Green, J., Byford, S., Woods, J., McGowan, R. (2000). Randomised comparison of the effectiveness and costs of community and hospital based mental health services for children with behavioural disorders. BMJ 321: 1047-1050 [Abstract] [Full text]  
  • Wu, C. L., Fleisher, L. A. (2000). Outcomes Research in Regional Anesthesia and Analgesia. Anesth. Analg. 91: 1232-1242 [Full text]  
  • LEFF, J., VEARNALS, S., WOLFF, G., ALEXANDER, B., CHISHOLM, D., EVERITT, B., ASEN, E., JONES, E., BREWIN, C. R., DAYSON, D. (2000). The London Depression Intervention Trial: Randomised controlled trial of antidepressants v. couple therapy in the treatment and maintenance of people with depression living with a partner: clinical outcome and costs. Br. J. Psychiatry 177: 95-100 [Abstract] [Full text]  
  • Barber, J. A, Thompson, S. G, Coomarasamy, A, Berg, D V. D., Williams, J G, Cohen, D R, Russell, I T (2000). Open access follow up for inflammatory bowel disease. BMJ 320: 1730-1730 [Full text]  
  • BYFORD, S., FIANDER, M., TORGERSON, D. J., BARBER, J. A., THOMPSON, S. G., BURNS, T., VAN HORN, E., GILVARRY, C., CREED, F. (2000). Cost-effectiveness of intensive v. standard case management for severe psychotic illness: UK700 case management trial. Br. J. Psychiatry 176: 537-543 [Abstract] [Full text]  
  • Thompson, S. G, Barber, J. A (2000). How should cost data in pragmatic randomised trials be analysed?. BMJ 320: 1197-1200 [Full text]  
  • Venning, P, Durie, A, Roland, M, Roberts, C, Leese, B (2000). Randomised controlled trial comparing cost effectiveness of general practitioners and nurse practitioners in primary care. BMJ 320: 1048-1053 [Abstract] [Full text]  
  • Lattimer, V., Sassi, F., George, S., Moore, M., Turnbull, J., Mullee, M., Smith, H. (2000). Cost analysis of nurse telephone consultation in out of hours primary care: evidence from a randomised controlled trial. BMJ 320: 1053-1057 [Abstract] [Full text]  
  • Kernick, D. (2000). Costing interventions in primary care. Fam Pract 17: 66-70 [Abstract] [Full text]  
  • Haycox, A., Bagust, A., Walley, T. (1999). Importance of health economics must be recognised when trials are designed. BMJ 318: 1696a-1696 [Full text]  
Access jobs at BMJ Careers
Whats new online at Student BMJ