BMJ  2005;330:753 (2 April), doi:10.1136/bmj.38356.424606.8F (published 28 January 2005)

Paper

Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors

¹ University Health Network, Department of Medicine, Suite RFE 3-805, 190 Elizabeth Street, Toronto, ON M5G 2C4, Canada, ² Cancer Research UK/NHS Centre for Statistics in Medicine, Oxford

Objective To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials.

Design Retrospective review of publications and follow up survey of authors.

Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000.

Main outcome measures Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance.

Results 519 trials with 553 publications and 10 557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable—for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance.

Conclusions Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

Risk of bias versus quality assessment of randomised controlled trials: cross sectional study

Lisa Hartling, Maria Ospina, Yuanyuan Liang, Donna M Dryden, Nicola Hooton, Jennifer Krebs Seida, and Terry P Klassen
BMJ 2009 339: b4012. [Abstract] [Full Text] [PDF]

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration

Alessandro Liberati, Douglas G Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C Gøtzsche, John P A Ioannidis, Mike Clarke, P J Devereaux, Jos Kleijnen, and David Moher
BMJ 2009 339: b2700. [Abstract] [Full Text]

Reporting and interpretation of SF-36 outcomes in randomised trials: systematic review

Despina G Contopoulos-Ioannidis, Anastasia Karvouni, Ioanna Kouri, and John P A Ioannidis
BMJ 2009 338: a3006. [Abstract] [Full Text] [PDF]

Believability of relative risks and odds ratios in abstracts: cross sectional study

Peter C Gøtzsche
BMJ 2006 333: 231-234. [Abstract] [Full Text] [PDF]

Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1)

Karmela Krleza-Jeric, An-Wen Chan, Kay Dickersin, Ida Sim, Jeremy Grimshaw, Christian Gluud for the Ottawa Group
BMJ 2005 330: 956-958. [Extract] [Full Text] [PDF]

Reporting of trial outcomes is incomplete and biased
BMJ 2005 330: 0. [Full Text]

This article has been cited by other articles:

• Hartling, L., Ospina, M., Liang, Y., Dryden, D. M, Hooton, N., Krebs Seida, J., Klassen, T. P (2009). Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ 339: b4012-b4012 [Abstract] [Full text]  
• Ioannidis, J. P.A. (2009). Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ 181: 488-493 [Full text]  
• Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P.A., Clarke, M., Devereaux, P. J., Kleijnen, J., Moher, D. (2009). The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. ANN INTERN MED 151: W-65-W-94 [Abstract] [Full text]  
• Liberati, A., Altman, D. G, Tetzlaff, J., Mulrow, C., Gotzsche, P. C, Ioannidis, J. P A, Clarke, M., Devereaux, P J, Kleijnen, J., Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 339: b2700-b2700 [Abstract] [Full text]  
• Ip, S., Lichtenstein, A. H., Chung, M., Lau, J., Balk, E. M. (2009). Systematic Review: Association of Low-Density Lipoprotein Subfractions With Cardiovascular Outcomes. ANN INTERN MED 150: 474-484 [Abstract] [Full text]  
• Little, J., Higgins, J. P.T., Ioannidis, J. P.A., Moher, D., Gagnon, F., von Elm, E., Khoury, M. J., Cohen, B., Davey-Smith, G., Grimshaw, J., Scheet, P., Gwinn, M., Williamson, R. E., Zou, G. Y., Hutchings, K., Johnson, C. Y., Tait, V., Wiens, M., Golding, J., van Duijn, C., McLaughlin, J., Paterson, A., Wells, G., Fortier, I., Freedman, M., Zecevic, M., King, R., Infante-Rivard, C., Stewart, A., Birkett, N. (2009). STrengthening the REporting of Genetic Association Studies (STREGA): An Extension of the STROBE Statement. ANN INTERN MED 150: 206-215 [Abstract] [Full text]  
• Thorlund, K., Devereaux, P J, Wetterslev, J., Guyatt, G., Ioannidis, J. P A, Thabane, L., Gluud, L.-L., Als-Nielsen, B., Gluud, C. (2009). Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?. Int J Epidemiol 38: 276-286 [Abstract] [Full text]  
• Contopoulos-Ioannidis, D. G, Karvouni, A., Kouri, I., Ioannidis, J. P A (2009). Reporting and interpretation of SF-36 outcomes in randomised trials: systematic review. BMJ 338: a3006-a3006 [Abstract] [Full text]  
• Wager, E., Abbasi, K. (2009). Medical editors and trial reporting: A betrayal of patient care. JRSM 102: 4-5 [Full text]  
• Mathoulin-Pelissier, S., Gourgou-Bourgade, S., Bonnetain, F., Kramar, A. (2008). Survival End Point Reporting in Randomized Cancer Clinical Trials: A Review of Major Journals. JCO 26: 3721-3726 [Abstract] [Full text]  
• Hackbarth, D. (2008). Research Reporting and Evidence of Effectiveness: Why "No Difference" Matters. Am J Crit Care 17: 218-220 [Full text]  
• Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., Rosenthal, R. (2008). Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy. NEJM 358: 252-260 [Abstract] [Full text]  
• Patsopoulos, N. A., Tatsioni, A., Ioannidis, J. P. A. (2007). Claims of Sex Differences: An Empirical Assessment in Genetic Associations. JAMA 298: 880-893 [Abstract] [Full text]  
• Cleophas, T. J., Zwinderman, A. H. (2007). Meta-Analysis. Circulation 115: 2870-2875 [Full text]  
• Ioannidis, J. P., Trikalinos, T. A (2007). Authors' response to V Johnson and Y Yuan. Clin Trials 4: 256-257  
• Coultas, D. (2007). Ethical Considerations in the Interpretation and Communication of Clinical Trial Results. Proc Am Thorac Soc 4: 194-199 [Abstract] [Full text]  
• Ioannidis, J. P.A., Trikalinos, T. A. (2007). The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. CMAJ 176: 1091-1096 [Abstract] [Full text]  
• Jefferson, T., Zarra, L., Stoica, L. (2006). Attenti Alle Bufale ('Beware of Red Herrings'), or, How to Make Evidence-Based Medicine Work for You. JRSM 99: 625-627 [Full text]  
• Bero, L A (2006). Clinical trial registration at Tobacco Control. Tobacco Control 15: 417-418 [Full text]  
• Gotzsche, P. C (2006). Believability of relative risks and odds ratios in abstracts: cross sectional study. BMJ 333: 231-234 [Abstract] [Full text]  
• Gluud, L. L. (2006). Bias in Clinical Intervention Research. Am J Epidemiol 163: 493-501 [Abstract] [Full text]  
• Steensma, D. P., Bennett, J. M. (2006). The Myelodysplastic Syndromes: Diagnosis and Treatment. Mayo Clin Proc. 81: 104-130 [Abstract] [Full text]  
• Williamson, P R, Gamble, C, Altman, D G, Hutton, J L (2005). Outcome selection bias in meta-analysis. Stat Methods Med Res 14: 515-524 [Abstract]  
• (2005). From the Library. Br J Ophthalmol 89: 930-930 [Full text]  
• Krleza-Jeric, K., Chan, A.-W., Dickersin, K., Sim, I., Grimshaw, J., Gluud, C., for the Ottawa Group, (2005). Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ 330: 956-958 [Full text]  

Online poll

Find out more on doc2doc >>