Education and debate

Why we need observational studies to evaluate the effectiveness of health care

^a Health Services Research Unit, Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT

The view is widely held that experimental methods (randomised controlled trials) are the "gold standard" for evaluation and that observational methods (cohort and case control studies) have little or no value. This ignores the limitations of randomised trials, which may prove unnecessary, inappropriate, impossible, or inadequate. Many of the problems of conducting randomised trials could often, in theory, be overcome, but the practical implications for researchers and funding bodies mean that this is often not possible. The false conflict between those who advocate randomised trials in all situations and those who believe observational data provide sufficient evidence needs to be replaced with mutual recognition of the complementary roles of the two approaches. Researchers should be united in their quest for scientific rigour in evaluation, regardless of the method used.

Despite the essential role of observational methods in shedding light on the effectiveness of . . . [Full text of this article]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings

Richard L Tannen, Mark G Weiner, and Dawei Xie
BMJ 2009 338: b81. [Abstract] [Full Text] [PDF]

Developing and evaluating complex interventions: the new Medical Research Council guidance

Peter Craig, Paul Dieppe, Sally Macintyre, Susan Michie, Irwin Nazareth, and Mark Petticrew
BMJ 2008 337: a1655. [Full Text]

Evidence for the impact of quality improvement collaboratives: systematic review

Loes M T Schouten, Marlies E J L Hulscher, Jannes J E van Everdingen, Robbert Huijsman, and Richard P T M Grol
BMJ 2008 336: 1491-1494. [Abstract] [Full Text] [PDF]

Prognostic modelling in traumatic brain injury

David Menon and David Harrison
BMJ 2008 336: 397-398. [Extract] [Full Text] [PDF]

Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies

Erik von Elm, Douglas G Altman, Matthias Egger, Stuart J Pocock, Peter C Gøtzsche, Jan P Vandenbroucke STROBE Initiative
BMJ 2007 335: 806-808. [Extract] [Full Text] [PDF]

Why stop at antidepressants?

Simon Hatcher
BMJ 2005 331: 158. [Extract] [Full Text] [PDF]

Reader's guide to critical appraisal of cohort studies: 1. Role and design

Paula A Rochon, Jerry H Gurwitz, Kathy Sykora, Muhammad Mamdani, David L Streiner, Susan Garfinkel, Sharon-Lise T Normand, and Geoffrey M Anderson
BMJ 2005 330: 895-897. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• La Caze, A. (2009). Evidence-Based Medicine Must Be .... J Med Philos 34: 509-527 [Abstract] [Full text]  
• Robinson, G., McNulty, J. E., Krasno, J. S. (2009). Observing the Counterfactual? The Search for Political Experiments in Nature. POLIT ANAL 17: 341-357 [Abstract] [Full text]  
• Grob, D., Bartanusz, V., Jeszenszky, D., Kleinstuck, F. S., Lattig, F., O'Riordan, D., Mannion, A. F. (2009). A prospective, cohort study comparing translaminar screw fixation with transforaminal lumbar interbody fusion and pedicle screw fixation for fusion of the degenerative lumbar spine. J Bone Joint Surg Br 91-B: 1347-1353 [Abstract] [Full text]  
• Hoppe, D. J., Schemitsch, E. H., Morshed, S., Tornetta, P. III, Bhandari, M. (2009). Hierarchy of Evidence: Where Observational Studies Fit in and Why We Need Them. JBJS 91: 2-9 [Abstract] [Full text]  
• Diaz-Insua, M. (2009). Review: Methodological quality in medical evidence, quo vadis?. Therapeutic Advances in Urology 1: 51-59 [Abstract]  
• Daumer, M., Neuhaus, A., Morrissey, S., Hintzen, R., Ebers, G. C. (2009). MRI as an outcome in multiple sclerosis clinical trials. Neurology 72: 705-711 [Abstract] [Full text]  
• Tannen, R. L, Weiner, M. G, Xie, D. (2009). Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings. BMJ 338: b81-b81 [Abstract] [Full text]  
• Cook, S., Chambers, E., Coleman, J. H (2009). Occupational therapy for people with psychotic conditions in community settings: a pilot randomized controlled trial. Clin Rehabil 23: 40-52 [Abstract]  
• Simon, S, Higginson, I. (2009). Evaluation of hospital palliative care teams: strengths and weaknesses of the before-after study design and strategies to improve it. Palliat Med 23: 23-28 [Abstract]  
• Ebrahim, S., Adamson, J., Ayis, S., Beswick, A., Gooberman-Hill, R. (2008). Locomotor disability: meaning, causes and effects of interventions. J Health Serv Res Policy 13: 38-46 [Abstract] [Full text]  
• Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337: a1655-a1655 [Full text]  
• Penrod, J., Morrison, R. S., Meier, D. E. (2008). Studying the Effectiveness of Palliative Care. JAMA 300: 1022-1023 [Full text]  
• Schouten, L. M T, Hulscher, M. E J L, Everdingen, J. J E v., Huijsman, R., Grol, R. P T M (2008). Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 336: 1491-1494 [Abstract] [Full text]  
• Hannan, E. L. (2008). Randomized Clinical Trials and Observational Studies: Guidelines for Assessing Respective Strengths and Limitations. J Am Coll Cardiol Intv 1: 211-217 [Abstract] [Full text]  
• Hill, M, Ben-Shlomo, Y (2008). Neurological care and risk of hospital mortality for patients with myasthenia gravis in England. J. Neurol. Neurosurg. Psychiatry 79: 421-425 [Abstract] [Full text]  
• Parkhurst, J. (2008). Understanding determinants of health service use from a systems perspective. J Health Serv Res Policy 13: 122-123 [Full text]  
• Malfair Taylor, S. C, de Lemos, M. L, Jang, D., Man, J., Annable, D., Mithani, S., John, L., Vu, T., O'Brien, R. K (2008). Impact on patient satisfaction with a structured counselling approach on natural health products. J Oncol Pharm Pract 14: 37-43 [Abstract]  
• McCarthy, M., Datta, P., Sherlaw-Johnson, C., Coleman, M., Rachet, B. (2008). Is the performance of cancer services influenced more by hospital factors or by specialization?. J Public Health (Oxf) 30: 69-74 [Abstract] [Full text]  
• Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., Hawe, P. (2008). Improving the reporting of public health intervention research: advancing TREND and CONSORT. J Public Health (Oxf) 30: 103-109 [Abstract] [Full text]  
• Menon, D., Harrison, D. (2008). Prognostic modelling in traumatic brain injury. BMJ 336: 397-398 [Full text]  
• Otero-Fernandez, R., Gomez-Outes, A., Martinez-Gonzalez, J., Rocha, E., Fontcuberta, J. (2008). Evaluation of the Effectiveness and Safety of Bemiparin in a Large Population of Orthopedic Patients in a Normal Clinical Practice. CLIN APPL THROMB HEMOST 14: 75-83 [Abstract]  
• Bunge, E. M., Juttmann, R. E., van Biezen, F. C., Creemers, H., Hazebroek-Kampschreur, A. A. J. M., Luttmer, B. C. F., Wiegersma, P. A., de Koning, H. J., for the Netherlands Evaluation Study on Screening, (2008). Estimating the Effectiveness of Screening for Scoliosis: A Case-Control Study. Pediatrics 121: 9-14 [Abstract] [Full text]  
• Boesen, E., Boesen, S., Christensen, S., Johansen, C. (2007). Comparison of Participants and Non-Participants in a Randomized Psychosocial Intervention Study Among Patients With Malignant Melanoma. Psychosomatics 48: 510-516 [Abstract] [Full text]  
• Ishihara, L. S, Cheesbrough, A., Brayne, C., Schrag, A. (2007). Estimated life expectancy of Parkinson's patients compared with the UK population. J. Neurol. Neurosurg. Psychiatry 78: 1304-1309 [Abstract] [Full text]  
• Kievit, W, Fransen, J, Oerlemans, A J M, Kuper, H H, van der Laar, M A F J, de Rooij, D J R A M, De Gendt, C M A, Ronday, K H, Jansen, T L, van Oijen, P C M, Brus, H L M, Adang, E M, van Riel, P L C M (2007). The efficacy of anti-TNF in rheumatoid arthritis, a comparison between randomised controlled trials and clinical practice. Ann Rheum Dis 66: 1473-1478 [Abstract] [Full text]  
• van den Meerschaut, L., Sunder, A. (2007). The Homeopathic Preparation Nervoheel N can Offer an Alternative to Lorazepam Therapy for Mild Nervous Disorders. Evid Based Complement Alternat Med 0: nem144v1-nem144 [Abstract] [Full text]  
• Elm, E. v., Altman, D. G, Egger, M., Pocock, S. J, Gotzsche, P. C, Vandenbroucke, J. P, STROBE Initiative, (2007). Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335: 806-808 [Full text]  
• von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., Vandenbroucke, J. P., for the STROBE Initiative, (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. ANN INTERN MED 147: 573-577 [Abstract] [Full text]  
• Egger, M., Altman, D. G, Vandenbroucke, J. P, of the STROBE group, (2007). Commentary: Strengthening the reporting of observational epidemiology the STROBE statement. Int J Epidemiol 0: dym199v1-dym199 [Full text]  
• Kelly, C., Arnold, R., Galloway, Y., O'Hallahan, J. (2007). A Prospective Study of the Effectiveness of the New Zealand Meningococcal B Vaccine. Am J Epidemiol 166: 817-823 [Abstract] [Full text]  
• Berry, D. A., Ravdin, P. M. (2007). Breast Cancer Trends: A Marriage Between Clinical Trial Evidence and Epidemiology. JNCI J Natl Cancer Inst 99: 1139-1141 [Full text]  
• Hodgson, R., Bushe, C., Hunter, R. (2007). Measurement of long-term outcomes in observational and randomised controlled trials. Br. J. Psychiatry 191: s78-s84 [Abstract] [Full text]  
• Boutron, I., Ravaud, P., Nizard, R. (2007). The design and assessment of prospective randomised, controlled trials in orthopaedic surgery. J Bone Joint Surg Br 89-B: 858-863 [Abstract] [Full text]  
• Stewart, W. F., Shah, N. R., Selna, M. J., Paulus, R. A., Walker, J. M. (2007). Bridging The Inferential Gap: The Electronic Health Record And Clinical Evidence. Health Aff (Millwood) 26: w181-w191 [Abstract] [Full text]  
• Weinstein, J. N., Lurie, J. D., Tosteson, T. D., Skinner, J. S., Hanscom, B., Tosteson, A. N. A., Herkowitz, H., Fischgrund, J., Cammisa, F. P., Albert, T., Deyo, R. A. (2006). Surgical vs Nonoperative Treatment for Lumbar Disk Herniation: The Spine Patient Outcomes Research Trial (SPORT) Observational Cohort. JAMA 296: 2451-2459 [Abstract] [Full text]  
• MERLO, J., CHAIX, B. (2006). Neighbourhood effects and the real world beyond randomized community trials: a reply to Michael J Oakes. Int J Epidemiol 35: 1361-1363 [Full text]  
• Wong, D, Kyle, G (2006). Some ethical considerations for the "off-label" use of drugs such as Avastin.. Br J Ophthalmol 90: 1218-1219 [Full text]  
• Harris, M, Smith, B J, Veale, A, Esterman, A, Frith, P A, Selim, P (2006). Providing patients with reviews of evidence about COPD treatments: a controlled trial of outcomes. Chronic Respiratory Disease 3: 133-140 [Abstract]  
• Brady, A. R., Harrison, D., Black, S., Jones, S., Rowan, K., Pearson, G., Ratcliffe, J., Parry, G. J., on behalf of the UK PICOS Study Group, (2006). Assessment and Optimization of Mortality Prediction Tools for Admissions to Pediatric Intensive Care in the United Kingdom. Pediatrics 117: e733-e742 [Abstract] [Full text]  
• Gluud, L. L. (2006). Bias in Clinical Intervention Research. Am J Epidemiol 163: 493-501 [Abstract] [Full text]  
• Zhou, Z., Rahme, E., Pilote, L. (2006). An evaluation of class effect.. CMAJ 174: 502-502 [Full text]  
• Baillard, C., Clec'h, C., Catineau, J., Salhi, F., Gehan, G., Cupa, M., Samama, C. M. (2005). Postoperative residual neuromuscular block: a survey of management. Br J Anaesth 95: 622-626 [Abstract] [Full text]  
• Hollingworth, W. (2005). Radiology Cost and Outcomes Studies: Standard Practice and Emerging Methods. Am. J. Roentgenol. 185: 833-839 [Full text]  
• Walker, W. (2005). The strengths and weaknesses of research designs involving quantitative measures. Journal of Research in Nursing 10: 571-582 [Abstract]  
• Du, X. L., Jones, D. V., Zhang, D. (2005). Effectiveness of Adjuvant Chemotherapy for Node-Positive Operable Breast Cancer in Older Women. Journals of Gerontology Series A: Biological Sciences and Medical Sciences 60: 1137-1144 [Abstract] [Full text]  
• Hatcher, S. (2005). Why stop at antidepressants?. BMJ 331: 158-158 [Full text]  
• Hartling, L., McAlister, F. A., Rowe, B. H., Ezekowitz, J., Friesen, C., Klassen, T. P. (2005). Challenges in Systematic Reviews of Therapeutic Devices and Procedures. ANN INTERN MED 142: 1100-1111 [Abstract] [Full text]  
• Freeman, J A, Hobart, J C, Playford, E D, Undy, B, Thompson, A J (2005). Evaluating neurorehabilitation: lessons from routine data collection. J. Neurol. Neurosurg. Psychiatry 76: 723-728 [Abstract] [Full text]  
• Munro, A J (2005). The conventional wisdom and the activities of the middle range. Br. J. Radiol. 78: 381-383 [Full text]  
• Zhang, W, Doherty, M, Arden, N, Bannwarth, B, Bijlsma, J, Gunther, K-P, Hauselmann, H J, Herrero-Beaumont, G, Jordan, K, Kaklamanis, P, Leeb, B, Lequesne, M, Lohmander, S, Mazieres, B, Martin-Mola, E, Pavelka, K, Pendleton, A, Punzi, L, Swoboda, B, Varatojo, R, Verbruggen, G, Zimmermann-Gorska, I, Dougados, M (2005). EULAR evidence based recommendations for the management of hip osteoarthritis: report of a task force of the EULAR Standing Committee for International Clinical Studies Including Therapeutics (ESCISIT). Ann Rheum Dis 64: 669-681 [Abstract] [Full text]  
• Rochon, P. A, Gurwitz, J. H, Sykora, K., Mamdani, M., Streiner, D. L, Garfinkel, S., Normand, S.-L. T, Anderson, G. M (2005). Reader's guide to critical appraisal of cohort studies: 1. Role and design. BMJ 330: 895-897 [Full text]  
• Speller, V., Wimbush, E., Morgan, A. (2005). Evidence-based health promotion practice: how to make it work. Global Health Promotion 12: 15-20  
• Devereaux, P J, Bhandari, M., Clarke, M., Montori, V. M, Cook, D. J, Yusuf, S., Sackett, D. L, Cina, C. S, Walter, S D, Haynes, B., Schunemann, H. J, Norman, G. R, Guyatt, G. H (2005). Need for expertise based randomised controlled trials. BMJ 330: 88- [Full text]  
• Kirkwood, B. (2004). Making public health interventions more evidence based. BMJ 328: 966-967 [Full text]  
• Victora, C. G., Habicht, J.-P., Bryce, J. (2004). Evidence-Based Public Health: Moving Beyond Randomized Trials. AJPH 94: 400-405 [Abstract] [Full text]  
• Anyanwu, A. C., Treasure, T. (2004). Surgical research revisited: clinical trials in the cardiothoracic surgical literature. Eur. J. Cardiothorac. Surg. 25: 299-303 [Abstract] [Full text]  
• Thornton, J, Elliott, R, Tully, M P, Dodd, M, Webb, A K (2004). Long term clinical outcome of home and hospital intravenous antibiotic treatment in adults with cystic fibrosis. Thorax 59: 242-246 [Abstract] [Full text]  
• Nobilio, L, Fortuna, D, Vizioli, M, Berti, E, Guastaroba, P, Taroni, F, Grilli, R (2004). Impact of regionalisation of cardiac surgery in Emilia-Romagna, Italy. J. Epidemiol. Community Health 58: 97-102 [Abstract] [Full text]  
• Thomson, H., Hoskins, R., Petticrew, M., Ogilvie, D., Craig, N., Quinn, T., Lindsay, G. (2004). Evaluating the health effects of social interventions. BMJ 328: 282-285 [Full text]  
• Shrier, I (2003). Cochrane Reviews: new blocks on the kids. Br. J. Sports. Med. 37: 473-474 [Full text]  
• Byford, S., Sefton, T. (2003). Economic Evaluation of Complex Health and Social Care Interventions. National Institute Economic Review 186: 98-108 [Abstract]  
• Aletaha, D, Stamm, T, Kapral, T, Eberl, G, Grisar, J, Machold, K P, Smolen, J S (2003). Survival and effectiveness of leflunomide compared with methotrexate and sulfasalazine in rheumatoid arthritis: a matched observational study. Ann Rheum Dis 62: 944-951 [Abstract] [Full text]  
• Walker, J., Sofaer, B. (2003). Randomised controlled trials in the evaluation of non-biomedical therapeutic interventions for pain: The gold standard?. Journal of Research in Nursing 8: 317-329 [Abstract]  
• (2003). The pharmacoepidemiology of COPD: Recent advances and methodological discussion. Eur Respir J 22: 1s-44s [Full text]  
• Boutron, I., Tubach, F., Giraudeau, B., Ravaud, P. (2003). Methodological Differences in Clinical Trials Evaluating Nonpharmacological and Pharmacological Treatments of Hip and Knee Osteoarthritis. JAMA 290: 1062-1070 [Abstract] [Full text]  
• Harvey, G, Wensing, M (2003). Methods for evaluation of small scale quality improvement projects. Qual Saf Health Care 12: 210-214 [Abstract] [Full text]  
• Merlo, J., Lynch, J. W., Yang, M., Lindstrom, M., Ostergren, P. O., Rasmusen, N. K., Rastam, L. (2003). Effect of Neighborhood Social Participation on Individual Use of Hormone Replacement Therapy and Antihypertensive Medication: A Multilevel Analysis. Am J Epidemiol 157: 774-783 [Abstract] [Full text]  
• Hallberg, A.-C., Hakansson, A. (2003). Training Programme for Parents of Teenagers: Active Parental Participation in Development and Implementation. J Child Health Care 7: 7-16 [Abstract]  
• Song, F., Altman, D. G, Glenny, A.-M., Deeks, J. J (2003). Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 326: 472-472 [Abstract] [Full text]  
• Clancy, M J (2002). Overview of research designs. Emerg. Med. J. 19: 546-549 [Abstract] [Full text]  
• Elphick, H. E, Tan, A., Ashby, D., Smyth, R. L (2002). Systematic reviews and lifelong diseases. BMJ 325: 381-384 [Full text]  
• McCulloch, P., Taylor, I., Sasako, M., Lovett, B., Griffin, D. (2002). Randomised trials in surgery: problems and possible solutions. BMJ 324: 1448-1451 [Full text]  
• Ray, J. G. (2002). Evidence in Upheaval: Incorporating Observational Data Into Clinical Practice. Arch Intern Med 162: 249-254 [Full text]  
• Rychetnik, L, Frommer, M, Hawe, P, Shiell, A (2002). Criteria for evaluating evidence on public health interventions. J. Epidemiol. Community Health 56: 119-127 [Abstract] [Full text]  
• Heller, R F, Page, J (2002). A population perspective to evidence based medicine: ""evidence for population health"". J. Epidemiol. Community Health 56: 45-47 [Abstract] [Full text]  
• Herman, W. H. (2002). Evidence-Based Diabetes Care. Clin. Diabetes 20: 22-23 [Full text]  
• GILBODY, S., WHITTY, P. (2002). Improving the delivery and organisation of mental health services: beyond the conventional randomised controlled trial. Br. J. Psychiatry 180: 13-18 [Abstract] [Full text]  
• Attride-Stirling, J. (2001). Thematic networks: an analytic tool for qualitative research. Qualitative Research 1: 385-405 [Abstract]  
• Padkin, A., Rowan, K., Black, N. (2001). Using high quality clinical databases to complement the results of randomised controlled trials: the case of recombinant human activated protein C. BMJ 323: 923-926 [Full text]  
• Carding, P., Hillman, R. (2001). More randomised controlled studies in speech and language therapy. BMJ 323: 645-646 [Full text]  
• McMahon, A. D. (2001). Observation and Experiment with the Efficacy of Drugs: A Warning Example from a Cohort of Nonsteroidal Anti-inflammatory and Ulcer-healing Drug Users. Am J Epidemiol 154: 557-562 [Abstract] [Full text]  
• Brown, C., Lloyd, K. (2001). Qualitative methods in psychiatric research. Adv. Psychiatr. Treat. 7: 350-356 [Full text]  
• Berger, R. L., Celli, B. R., Meneghetti, A. L., Bagley, P. H., Wright, C. D., Ingenito, E. P., Gray, A., Snider, G. L. (2001). Limitations of randomized clinical trials for evaluating emerging operations: the case of lung volume reduction surgery. Ann. Thorac. Surg. 72: 649-657 [Abstract] [Full text]  
• McPherson, K. (2001). Safer discharge from intensive care to hospital wards. BMJ 322: 1261-1262 [Full text]  
• Coriat, P., Beaussier, M. (2001). Fast-Tracking After Coronary Artery Bypass Graft Surgery. Anesth. Analg. 92: 1081-1083 [Full text]  
• Boston, N. K, Boynton, P. M, Hood, S. (2001). An inner city GP unit versus conventional care for elderly patients: prospective comparison of health functioning, use of services and patient satisfaction. Fam Pract 18: 141-148 [Abstract] [Full text]  
• Grilli, R. (2001). Specialization and cancer: words with too many meanings should be handled with care. CMAJ 164: 210-211 [Full text]