Education and debate

Why we need observational studies to evaluate the effectiveness of health care

^a Health Services Research Unit, Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT

The view is widely held that experimental methods (randomised controlled trials) are the "gold standard" for evaluation and that observational methods (cohort and case control studies) have little or no value. This ignores the limitations of randomised trials, which may prove unnecessary, inappropriate, impossible, or inadequate. Many of the problems of conducting randomised trials could often, in theory, be overcome, but the practical implications for researchers and funding bodies mean that this is often not possible. The false conflict between those who advocate randomised trials in all situations and those who believe observational data provide sufficient evidence needs to be replaced with mutual recognition of the complementary roles of the two approaches. Researchers should be united in their quest for scientific rigour in evaluation, regardless of the method used.

Despite the essential role of observational methods in shedding light on the effectiveness of . . . [Full text of this article]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

Developing and evaluating complex interventions: the new Medical Research Council guidance

Peter Craig, Paul Dieppe, Sally Macintyre, Susan Michie, Irwin Nazareth, and Mark Petticrew
BMJ 2008 337: a1655. [Full Text]

Evidence for the impact of quality improvement collaboratives: systematic review

Loes M T Schouten, Marlies E J L Hulscher, Jannes J E van Everdingen, Robbert Huijsman, and Richard P T M Grol
BMJ 2008 336: 1491-1494. [Abstract] [Full Text] [PDF]

Prognostic modelling in traumatic brain injury

David Menon and David Harrison
BMJ 2008 336: 397-398. [Extract] [Full Text] [PDF]

Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies

Erik von Elm, Douglas G Altman, Matthias Egger, Stuart J Pocock, Peter C Gøtzsche, Jan P Vandenbroucke STROBE Initiative
BMJ 2007 335: 806-808. [Extract] [Full Text] [PDF]

Why stop at antidepressants?

Simon Hatcher
BMJ 2005 331: 158. [Extract] [Full Text] [PDF]

Reader's guide to critical appraisal of cohort studies: 1. Role and design

Paula A Rochon, Jerry H Gurwitz, Kathy Sykora, Muhammad Mamdani, David L Streiner, Susan Garfinkel, Sharon-Lise T Normand, and Geoffrey M Anderson
BMJ 2005 330: 895-897. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Ebrahim, S., Adamson, J., Ayis, S., Beswick, A., Gooberman-Hill, R. (2008). Locomotor disability: meaning, causes and effects of interventions. J Health Serv Res Policy 13: 38-46 [Abstract] [Full text]  
• Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337: a1655-a1655 [Full text]  
• Penrod, J., Morrison, R. S., Meier, D. E. (2008). Studying the Effectiveness of Palliative Care. JAMA 300: 1022-1023 [Full text]  
• Schouten, L. M T, Hulscher, M. E J L, Everdingen, J. J E v., Huijsman, R., Grol, R. P T M (2008). Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 336: 1491-1494 [Abstract] [Full text]  
• Hannan, E. L. (2008). Randomized Clinical Trials and Observational Studies: Guidelines for Assessing Respective Strengths and Limitations. J Am Coll Cardiol Intv 1: 211-217 [Abstract] [Full text]  
• Hill, M, Ben-Shlomo, Y (2008). Neurological care and risk of hospital mortality for patients with myasthenia gravis in England. J. Neurol. Neurosurg. Psychiatry 79: 421-425 [Abstract] [Full text]  
• Parkhurst, J. (2008). Understanding determinants of health service use from a systems perspective. J Health Serv Res Policy 13: 122-123 [Full text]  
• Malfair Taylor, S. C, de Lemos, M. L, Jang, D., Man, J., Annable, D., Mithani, S., John, L., Vu, T., O'Brien, R. K (2008). Impact on patient satisfaction with a structured counselling approach on natural health products. J Oncol Pharm Pract 14: 37-43 [Abstract]  
• McCarthy, M., Datta, P., Sherlaw-Johnson, C., Coleman, M., Rachet, B. (2008). Is the performance of cancer services influenced more by hospital factors or by specialization?. J Public Health (Oxf) 30: 69-74 [Abstract] [Full text]  
• Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., Hawe, P. (2008). Improving the reporting of public health intervention research: advancing TREND and CONSORT. J Public Health (Oxf) 30: 103-109 [Abstract] [Full text]  
• Menon, D., Harrison, D. (2008). Prognostic modelling in traumatic brain injury. BMJ 336: 397-398 [Full text]  
• Otero-Fernandez, R., Gomez-Outes, A., Martinez-Gonzalez, J., Rocha, E., Fontcuberta, J. (2008). Evaluation of the Effectiveness and Safety of Bemiparin in a Large Population of Orthopedic Patients in a Normal Clinical Practice. CLIN APPL THROMB HEMOST 14: 75-83 [Abstract]  
• Bunge, E. M., Juttmann, R. E., van Biezen, F. C., Creemers, H., Hazebroek-Kampschreur, A. A. J. M., Luttmer, B. C. F., Wiegersma, P. A., de Koning, H. J., for the Netherlands Evaluation Study on Screening, (2008). Estimating the Effectiveness of Screening for Scoliosis: A Case-Control Study. Pediatrics 121: 9-14 [Abstract] [Full text]  
• Boesen, E., Boesen, S., Christensen, S., Johansen, C. (2007). Comparison of Participants and Non-Participants in a Randomized Psychosocial Intervention Study Among Patients With Malignant Melanoma. Psychosomatics 48: 510-516 [Abstract] [Full text]  
• Ishihara, L. S, Cheesbrough, A., Brayne, C., Schrag, A. (2007). Estimated life expectancy of Parkinson's patients compared with the UK population. J. Neurol. Neurosurg. Psychiatry 78: 1304-1309 [Abstract] [Full text]  
• Kievit, W, Fransen, J, Oerlemans, A J M, Kuper, H H, van der Laar, M A F J, de Rooij, D J R A M, De Gendt, C M A, Ronday, K H, Jansen, T L, van Oijen, P C M, Brus, H L M, Adang, E M, van Riel, P L C M (2007). The efficacy of anti-TNF in rheumatoid arthritis, a comparison between randomised controlled trials and clinical practice. Ann Rheum Dis 66: 1473-1478 [Abstract] [Full text]  
• van den Meerschaut, L., Sunder, A. (2007). The Homeopathic Preparation Nervoheel N can Offer an Alternative to Lorazepam Therapy for Mild Nervous Disorders. Evid Based Complement Alternat Med 0: nem144v1-nem144 [Abstract] [Full text]  
• Elm, E. v., Altman, D. G, Egger, M., Pocock, S. J, Gotzsche, P. C, Vandenbroucke, J. P, STROBE Initiative, (2007). Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335: 806-808 [Full text]  
• von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., Vandenbroucke, J. P., for the STROBE Initiative, (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. ANN INTERN MED 147: 573-577 [Abstract] [Full text]  
• Egger, M., Altman, D. G, Vandenbroucke, J. P, of the STROBE group, (2007). Commentary: Strengthening the reporting of observational epidemiology the STROBE statement. Int J Epidemiol 0: dym199v1-dym199 [Full text]  
• Kelly, C., Arnold, R., Galloway, Y., O'Hallahan, J. (2007). A Prospective Study of the Effectiveness of the New Zealand Meningococcal B Vaccine. Am J Epidemiol 166: 817-823 [Abstract] [Full text]  
• Berry, D. A., Ravdin, P. M. (2007). Breast Cancer Trends: A Marriage Between Clinical Trial Evidence and Epidemiology. JNCI J Natl Cancer Inst 99: 1139-1141 [Full text]  
• Hodgson, R., Bushe, C., Hunter, R. (2007). Measurement of long-term outcomes in observational and randomised controlled trials. Br. J. Psychiatry 191: s78-s84 [Abstract] [Full text]  
• Boutron, I., Ravaud, P., Nizard, R. (2007). The design and assessment of prospective randomised, controlled trials in orthopaedic surgery. J Bone Joint Surg Br 89-B: 858-863 [Abstract] [Full text]  
• Stewart, W. F., Shah, N. R., Selna, M. J., Paulus, R. A., Walker, J. M. (2007). Bridging The Inferential Gap: The Electronic Health Record And Clinical Evidence. Health Aff (Millwood) 26: w181-w191 [Abstract] [Full text]  
• Weinstein, J. N., Lurie, J. D., Tosteson, T. D., Skinner, J. S., Hanscom, B., Tosteson, A. N. A., Herkowitz, H., Fischgrund, J., Cammisa, F. P., Albert, T., Deyo, R. A. (2006). Surgical vs Nonoperative Treatment for Lumbar Disk Herniation: The Spine Patient Outcomes Research Trial (SPORT) Observational Cohort. JAMA 296: 2451-2459 [Abstract] [Full text]  
• MERLO, J., CHAIX, B. (2006). Neighbourhood effects and the real world beyond randomized community trials: a reply to Michael J Oakes. Int J Epidemiol 35: 1361-1363 [Full text]  
• Wong, D, Kyle, G (2006). Some ethical considerations for the "off-label" use of drugs such as Avastin.. Br. J. Ophthalmol. 90: 1218-1219 [Full text]  
• Harris, M, Smith, B J, Veale, A, Esterman, A, Frith, P A, Selim, P (2006). Providing patients with reviews of evidence about COPD treatments: a controlled trial of outcomes. Chronic Respiratory Disease 3: 133-140 [Abstract]  
• Brady, A. R., Harrison, D., Black, S., Jones, S., Rowan, K., Pearson, G., Ratcliffe, J., Parry, G. J., on behalf of the UK PICOS Study Group, (2006). Assessment and Optimization of Mortality Prediction Tools for Admissions to Pediatric Intensive Care in the United Kingdom. Pediatrics 117: e733-e742 [Abstract] [Full text]  
• Gluud, L. L. (2006). Bias in Clinical Intervention Research. Am J Epidemiol 163: 493-501 [Abstract] [Full text]  
• Zhou, Z., Rahme, E., Pilote, L. (2006). An evaluation of class effect.. CMAJ 174: 502-502 [Full text]  
• Baillard, C., Clec'h, C., Catineau, J., Salhi, F., Gehan, G., Cupa, M., Samama, C. M. (2005). Postoperative residual neuromuscular block: a survey of management. Br J Anaesth 95: 622-626 [Abstract] [Full text]  
• Hollingworth, W. (2005). Radiology Cost and Outcomes Studies: Standard Practice and Emerging Methods. Am. J. Roentgenol. 185: 833-839 [Full text]  
• Walker, W. (2005). The strengths and weaknesses of research designs involving quantitative measures. Journal of Research in Nursing 10: 571-582 [Abstract]  
• Du, X. L., Jones, D. V., Zhang, D. (2005). Effectiveness of Adjuvant Chemotherapy for Node-Positive Operable Breast Cancer in Older Women. J. Gerontol. A Biol. Sci. Med. Sci. 60: 1137-1144 [Abstract] [Full text]  
• Hatcher, S. (2005). Why stop at antidepressants?. BMJ 331: 158-158 [Full text]  
• Hartling, L., McAlister, F. A., Rowe, B. H., Ezekowitz, J., Friesen, C., Klassen, T. P. (2005). Challenges in Systematic Reviews of Therapeutic Devices and Procedures. ANN INTERN MED 142: 1100-1111 [Abstract] [Full text]  
• Freeman, J A, Hobart, J C, Playford, E D, Undy, B, Thompson, A J (2005). Evaluating neurorehabilitation: lessons from routine data collection. J. Neurol. Neurosurg. Psychiatry 76: 723-728 [Abstract] [Full text]  
• Munro, A J (2005). The conventional wisdom and the activities of the middle range. Br. J. Radiol. 78: 381-383 [Full text]  
• Zhang, W, Doherty, M, Arden, N, Bannwarth, B, Bijlsma, J, Gunther, K-P, Hauselmann, H J, Herrero-Beaumont, G, Jordan, K, Kaklamanis, P, Leeb, B, Lequesne, M, Lohmander, S, Mazieres, B, Martin-Mola, E, Pavelka, K, Pendleton, A, Punzi, L, Swoboda, B, Varatojo, R, Verbruggen, G, Zimmermann-Gorska, I, Dougados, M (2005). EULAR evidence based recommendations for the management of hip osteoarthritis: report of a task force of the EULAR Standing Committee for International Clinical Studies Including Therapeutics (ESCISIT). Ann Rheum Dis 64: 669-681 [Abstract] [Full text]  
• Rochon, P. A, Gurwitz, J. H, Sykora, K., Mamdani, M., Streiner, D. L, Garfinkel, S., Normand, S.-L. T, Anderson, G. M (2005). Reader's guide to critical appraisal of cohort studies: 1. Role and design. BMJ 330: 895-897 [Full text]  
• Speller, V., Wimbush, E., Morgan, A. (2005). Evidence-based health promotion practice: how to make it work. Promotion & Education 12: 15-20  
• Devereaux, P J, Bhandari, M., Clarke, M., Montori, V. M, Cook, D. J, Yusuf, S., Sackett, D. L, Cina, C. S, Walter, S D, Haynes, B., Schunemann, H. J, Norman, G. R, Guyatt, G. H (2005). Need for expertise based randomised controlled trials. BMJ 330: 88- [Full text]  
• Kirkwood, B. (2004). Making public health interventions more evidence based. BMJ 328: 966-967 [Full text]  
• Victora, C. G., Habicht, J.-P., Bryce, J. (2004). Evidence-Based Public Health: Moving Beyond Randomized Trials. Am. J. Public Health 94: 400-405 [Abstract] [Full text]  
• Anyanwu, A. C., Treasure, T. (2004). Surgical research revisited: clinical trials in the cardiothoracic surgical literature. Eur. J. Cardiothorac. Surg. 25: 299-303 [Abstract] [Full text]  
• Thornton, J, Elliott, R, Tully, M P, Dodd, M, Webb, A K (2004). Long term clinical outcome of home and hospital intravenous antibiotic treatment in adults with cystic fibrosis. Thorax 59: 242-246 [Abstract] [Full text]  
• Nobilio, L, Fortuna, D, Vizioli, M, Berti, E, Guastaroba, P, Taroni, F, Grilli, R (2004). Impact of regionalisation of cardiac surgery in Emilia-Romagna, Italy. J. Epidemiol. Community Health 58: 97-102 [Abstract] [Full text]  
• Thomson, H., Hoskins, R., Petticrew, M., Ogilvie, D., Craig, N., Quinn, T., Lindsay, G. (2004). Evaluating the health effects of social interventions. BMJ 328: 282-285 [Full text]  
• Shrier, I (2003). Cochrane Reviews: new blocks on the kids. Br. J. Sports. Med. 37: 473-474 [Full text]  
• Aletaha, D, Stamm, T, Kapral, T, Eberl, G, Grisar, J, Machold, K P, Smolen, J S (2003). Survival and effectiveness of leflunomide compared with methotrexate and sulfasalazine in rheumatoid arthritis: a matched observational study. Ann Rheum Dis 62: 944-951 [Abstract] [Full text]  
• Walker, J., Sofaer, B. (2003). Randomised controlled trials in the evaluation of non-biomedical therapeutic interventions for pain: The gold standard?. Journal of Research in Nursing 8: 317-329 [Abstract]  
• (2003). The pharmacoepidemiology of COPD: Recent advances and methodological discussion. Eur Respir J 22: 1s-44s [Full text]  
• Boutron, I., Tubach, F., Giraudeau, B., Ravaud, P. (2003). Methodological Differences in Clinical Trials Evaluating Nonpharmacological and Pharmacological Treatments of Hip and Knee Osteoarthritis. JAMA 290: 1062-1070 [Abstract] [Full text]  
• Harvey, G, Wensing, M (2003). Methods for evaluation of small scale quality improvement projects. Qual Saf Health Care 12: 210-214 [Abstract] [Full text]  
• Merlo, J., Lynch, J. W., Yang, M., Lindstrom, M., Ostergren, P. O., Rasmusen, N. K., Rastam, L. (2003). Effect of Neighborhood Social Participation on Individual Use of Hormone Replacement Therapy and Antihypertensive Medication: A Multilevel Analysis. Am J Epidemiol 157: 774-783 [Abstract] [Full text]  
• Song, F., Altman, D. G, Glenny, A.-M., Deeks, J. J (2003). Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 326: 472-472 [Abstract] [Full text]  
• Clancy, M J (2002). Overview of research designs. Emerg. Med. J. 19: 546-549 [Abstract] [Full text]  
• Elphick, H. E, Tan, A., Ashby, D., Smyth, R. L (2002). Systematic reviews and lifelong diseases. BMJ 325: 381-384 [Full text]  
• McCulloch, P., Taylor, I., Sasako, M., Lovett, B., Griffin, D. (2002). Randomised trials in surgery: problems and possible solutions. BMJ 324: 1448-1451 [Full text]  
• Miller, D. R., Tierney, M. (2002). Observational studies and "real world" anesthesia pharmacoeconomics/Les etudes par observation et la realite pharmacoeconomique de l'anesthesie. Canadian J. Anesthesia 49: 329-334 [Full text]  
• Beaussier, M., Decorps, A., Tilleul, P., Megnigbeto, A., Balladur, P., Lienhart, A. (2002). Desflurane improves the throughput of patients in the PACU. A cost-effectiveness comparison with isoflurane: [L'usage du desflurane permet de reduire l'occupation de la SDR : etude de rentabilite comparee avec l'isoflurane]. Canadian J. Anesthesia 49: 339-346 [Abstract] [Full text]  
• Ray, J. G. (2002). Evidence in Upheaval: Incorporating Observational Data Into Clinical Practice. Arch Intern Med 162: 249-254 [Full text]  
• Rychetnik, L, Frommer, M, Hawe, P, Shiell, A (2002). Criteria for evaluating evidence on public health interventions. J. Epidemiol. Community Health 56: 119-127 [Abstract] [Full text]  
• Heller, R F, Page, J (2002). A population perspective to evidence based medicine: ""evidence for population health"". J. Epidemiol. Community Health 56: 45-47 [Abstract] [Full text]  
• Herman, W. H. (2002). Evidence-Based Diabetes Care. Clin. Diabetes 20: 22-23 [Full text]  
• GILBODY, S., WHITTY, P. (2002). Improving the delivery and organisation of mental health services: beyond the conventional randomised controlled trial. Br. J. Psychiatry 180: 13-18 [Abstract] [Full text]  
• Padkin, A., Rowan, K., Black, N. (2001). Using high quality clinical databases to complement the results of randomised controlled trials: the case of recombinant human activated protein C. BMJ 323: 923-926 [Full text]  
• Carding, P., Hillman, R. (2001). More randomised controlled studies in speech and language therapy. BMJ 323: 645-646 [Full text]  
• McMahon, A. D. (2001). Observation and Experiment with the Efficacy of Drugs: A Warning Example from a Cohort of Nonsteroidal Anti-inflammatory and Ulcer-healing Drug Users. Am J Epidemiol 154: 557-562 [Abstract] [Full text]  
• Brown, C., Lloyd, K. (2001). Qualitative methods in psychiatric research. Adv. Psychiatr. Treat. 7: 350-356 [Full text]  
• Berger, R. L., Celli, B. R., Meneghetti, A. L., Bagley, P. H., Wright, C. D., Ingenito, E. P., Gray, A., Snider, G. L. (2001). Limitations of randomized clinical trials for evaluating emerging operations: the case of lung volume reduction surgery. Ann. Thorac. Surg. 72: 649-657 [Abstract] [Full text]  
• McPherson, K. (2001). Safer discharge from intensive care to hospital wards. BMJ 322: 1261-1262 [Full text]  
• Coriat, P., Beaussier, M. (2001). Fast-Tracking After Coronary Artery Bypass Graft Surgery. Anesth. Analg. 92: 1081-1083 [Full text]  
• Boston, N. K, Boynton, P. M, Hood, S. (2001). An inner city GP unit versus conventional care for elderly patients: prospective comparison of health functioning, use of services and patient satisfaction. Fam Pract 18: 141-148 [Abstract] [Full text]  
• Grilli, R. (2001). Specialization and cancer: words with too many meanings should be handled with care. CMAJ 164: 210-211 [Full text]  
• Maetzel, A., Wong, A., Strand, V., Tugwell, P., Wells, G., Bombardier, C. (2000). Meta-analysis of treatment termination rates among rheumatoid arthritis patients receiving disease-modifying anti-rheumatic drugs. Rheumatology (Oxford) 39: 975-981 [Abstract] [Full text]  
• Saunders, J. (2000). The practice of clinical medicine as an art and as a science. Med. Humanities 26: 18-22 [Abstract] [Full text]  
• Turkington, P M, Elliott, M W (2000). Rationale for the use of non-invasive ventilation in chronic ventilatory failure. Thorax 55: 417-423 [Full text]  
• Jager, G. J., Severens, J. L., Thornbury, J. R., de la Rosette, J. J. M. C. H., Ruijs, S. H. J., Barentsz, J. O. (2000). Prostate Cancer Staging: Should MR Imaging Be Used?-A Decision Analytic Approach. Radiology 215: 445-451 [Abstract] [Full text]  
• Ayanian, J.Z. (1999). Using administrative data to assess health care outcomes. Eur Heart J 20: 1689-1691  
• Lewsey, J.D, Murray, G.D, Leyland, A.H, Boddy, F.A (1999). Comparing outcomes of percutaneous transluminal coronary angioplasty with coronary artery bypass grafting. Can routine health service data complement and enhance randomized controlled trials?. Eur Heart J 20: 1731-1735 [Abstract]  
• Khunti, K. (1999). Use of multiple methods to determine factors affecting quality of care of patients with diabetes. Fam Pract 16: 489-494 [Abstract] [Full text]  
• Haigh, R., Bateman, A. W, Fonagy, P., Pearce, S, Dare, C, Parry, G., Kisely, S. (1999). Psychotherapy for severe personality disorder. BMJ 319: 709a-709 [Full text]  
• Ukoumunne, O. C, Gulliford, M. C, Chinn, S., Sterne, J. A C, Burney, P. G J, Donner, A. (1999). Methods in health service research: Evaluation of health interventions at area and organisation level. BMJ 319: 376-379 [Full text]  
• McKee, M., Britton, A., Black, N., McPherson, K., Sanderson, C., Bain, C. (1999). Methods in health services research: Interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 319: 312-315 [Full text]  
• Altman, D. G, Bland, J M. (1999). Statistics notes: Treatment allocation in controlled trials: why randomise?. BMJ 318: 1209-1209 [Full text]  
• Parry, R H, Lincoln, N B, Vass, C D (1999). Effect of severity of arm impairment on response to additional physiotherapy early after stroke. Clin Rehabil 13: 187-198 [Abstract]