BMJ  2007;334:349-351 (17 February), doi:10.1136/bmj.39070.527986.68

Analysis

When are randomised trials unnecessary? Picking signal from noise

The relation between a treatment and its effect is sometimes so dramatic that bias can be ruled out as an explanation. Paul Glasziouand colleagues suggest how to determine when observations speak for themselves

¹ Centre for Evidence-Based Medicine, Department of Primary Health Care, University of Oxford, Oxford OX3 7LF, ² James Lind Library, James Lind Initiative, Oxford OX2 7LG , ³ London School of Hygiene and Tropical Medicine, London WC1E 7HT, ⁴ Nuffield Department of Surgery, John Radcliffe Hospital, Oxford OX3 9DU

Abstract

Although randomised trials are widely accepted as the ideal way of obtaining unbiased estimates of treatment effects, some treatments have dramatic effects that are highly unlikely to reflect inadequately controlled biases. We compiled a list of historical examples of such effects and identified the features of convincing inferences about treatment effects from sources other than randomised trials. A unifying principle is the size of the treatment effect (signal) relative to the expected prognosis (noise) of the condition. A treatment effect is inferred most confidently when the signal to noise ratio is large and its timing is rapid compared with the natural course of the condition. For the examples we considered in detail the rate ratio often exceeds 10 and thus is highly unlikely to reflect bias or factors other than a treatment effect. This model may help to reduce controversy about evidence for treatments whose effects are so dramatic that randomised trials are unnecessary.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

What is "quality of evidence" and why is it important to clinicians?

Gordon H Guyatt, Andrew D Oxman, Regina Kunz, Gunn E Vist, Yngve Falck-Ytter, Holger J Schünemann for the GRADE Working Group
BMJ 2008 336: 995-998. [Extract] [Full Text] [PDF]

Too high a pedestal

Jeffrey K Aronson
BMJ 2008 336: 735. [Extract] [Full Text] [PDF]

The urge to sprinkle statistics is irresistible

James Penston
BMJ 2007 334: 440. [Extract] [Full Text] [PDF]

Beware the "Texas sharp shooter" in rate ratios of progression

Anna C Goodman
BMJ 2007 334: 440. [Extract] [Full Text] [PDF]

Controversies revisited

Jane Smith
BMJ 2007 334: 0. [Extract] [Full Text]

Money can't buy you satisfaction

Chris Ham
BMJ 2005 330: 597-599. [Extract] [Full Text] [PDF]

Grading quality of evidence and strength of recommendations
BMJ 2004 328: 1490. [Abstract] [Full Text] [PDF]

This article has been cited by other articles:

• Stephenson, T. (2008). Improving patient safety in paediatrics and child health. Arch. Dis. Child. 93: 650-653 [Full text]  
• Guyatt, G. H., Cook, D. J., Jaeschke, R., Pauker, S. G., Schunemann, H. J. (2008). Grades of Recommendation for Antithrombotic Agents: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest 133: 123S-131S [Abstract] [Full text]  
• Macbeth, F. R., Williams, M. V. (2008). Proton Therapy Should Be Tested in Randomized Trials. JCO 26: 2590-2591 [Full text]  
• Guyatt, G. H, Oxman, A. D, Kunz, R., Vist, G. E, Falck-Ytter, Y., Schunemann, H. J, for the GRADE Working Group, (2008). What is "quality of evidence" and why is it important to clinicians?. BMJ 336: 995-998 [Full text]  
• Aronson, J. K (2008). Too high a pedestal. BMJ 336: 735-735 [Full text]  
• Loudon, I. (2008). The use of historical controls and concurrent controls to assess the effects of sulphonamides, 1936-1945. JRSM 101: 148-155 [Full text]  
• Hemila, H. (2008). THE AUTHOR REPLIES. Am J Epidemiol 167: 501-502 [Full text]  
• Robinson, J. K., Dellavalle, R. P., Bigby, M., Callen, J. P. (2008). Systematic Reviews: Grading Recommendations and Evidence Quality. Arch Dermatol 144: 97-99 [Full text]  
• Lloyd, G. (2007). More than high standards needed. BMJ 335: 900-900 [Full text]  
• Jacobson, P. D., Rettig, R. A., Aubry, W. M. (2007). Litigating the Science of Breast Cancer Treatment. Journal of Health Politics, Policy and Law 32: 785-818 [Abstract]  
• Auerbach, A. D., Landefeld, C. S., Shojania, K. G. (2007). The Tension between Needing to Improve Care and Knowing How to Do It. NEJM 357: 608-613 [Full text]  
• Meakins, J. L (2007). Evaluating surgery. JRSM 100: 357-359 [Full text]  
• Boutron, I., Ravaud, P., Nizard, R. (2007). The design and assessment of prospective randomised, controlled trials in orthopaedic surgery. J Bone Joint Surg Br 89-B: 858-863 [Abstract] [Full text]  
• Penston, J. (2007). The urge to sprinkle statistics is irresistible. BMJ 334: 440-440 [Full text]  
• Goodman, A. C (2007). Beware the "Texas sharp shooter" in rate ratios of progression. BMJ 334: 440-440 [Full text]  

Rapid Responses:

Read all Rapid Responses

All or none studies

Olivier Steichen
bmj.com, 16 Feb 2007 [Full text]

The irresistible urge to sprinkle statistics

James Penston
bmj.com, 19 Feb 2007 [Full text]

Beware of the Texas sharp shooter in calculating rate ratios of progression

Anna C Goodman
bmj.com, 20 Feb 2007 [Full text]

Evidence from case series

Mounia N Hocine, et al.
bmj.com, 20 Feb 2007 [Full text]

Mother's Kiss

Daniel Polowetzky
bmj.com, 24 Feb 2007 [Full text]

Response from the Authors

Paul P Glasziou
bmj.com, 5 Mar 2007 [Full text]

75 years ago: 24. December 1932. Streptococci kill, antibiotics help to survive!

Friedrich Flachsbart
bmj.com, 26 Dec 2007 [Full text]