Papers

Publication bias: evidence of delayed publication in a cohort study of clinical research projects

^a National Health and Medical Research Council Clinical Trials Centre, Mallet Street Campus, University of Sydney, Sydney NSW 2006, Australia

Correspondence to: Professor Simes john@ctc.trials.su.oz.au

Objectives: To determine the extent to which publication is influenced by study outcome.
Design: A cohort of studies submitted to a hospital ethics committee over 10 years were examined retrospectively by reviewing the protocols and by questionnaire. The primary method of analysis was Cox's proportional hazards model.
Setting: University hospital, Sydney, Australia.
Studies: 748 eligible studies submitted to Royal Prince Alfred Hospital Ethics Committee between 1979 and 1988.
Main outcome measures: Time to publication.
Results: Response to the questionnaire was received for 520 (70%) of the eligible studies. Of the 218 studies analysed with tests of significance, those with positive results (P<0.05) were much more likely to be published than those with negative results (P0.10) (hazard ratio 2.32 (95% confidence interval 1.47 to 3.66), P=0.0003), with a significantly shorter time to publication (median 4.8 v 8.0 years). This finding was even stronger for the group of 130 clinical trials (hazard ratio 3.13 (1.76 to 5.58), P=0.0001), with median times to publication of 4.7 and 8.0 years respectively. These results were not materially changed after adjusting for other significant predictors of publication. Studies with indefinite conclusions (0.05P<0.10) tended to have an even lower publication rate and longer time to publication than studies with negative results (hazard ratio 0.39 (0.13 to 1.12), P=0.08). For the 103 studies in which outcome was rated qualitatively, there was no clear cut evidence of publication bias, although the number of studies in this group was not large.
Conclusions: This study confirms the evidence of publication bias found in other studies and identifies delay in publication as an additional important factor. The study results support the need for prospective registration of trials to avoid publication bias and also support restricting the selection of trials to those started before a common date in undertaking systematic reviews.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

Anders W Jørgensen, Jørgen Hilden, and Peter C Gøtzsche
BMJ 2006 333: 782. [Abstract] [Full Text] [PDF]

Fate of biomedical research protocols and publication bias in France: retrospective cohort study

Evelyne Decullier, Véronique Lhéritier, and François Chapuis
BMJ 2005 331: 19. [Abstract] [Full Text] [PDF]

Time to publication of studies was not affected by whether results were positive

Michael L Callaham, Ellen Weber, Gary Young, Robert Wears, and Chris Barton
BMJ 1998 316: 1536. [Extract] [Full Text]

This article has been cited by other articles:

• Arshad, A, Arkwright, P D (2008). Status of healthcare studies submitted to UK research ethics committees for approval in 2004-5. J. Med. Ethics 34: 393-395 [Abstract] [Full text]  
• Chan, J. K., Ueda, S. M., Sugiyama, V. E., Stave, C. D., Shin, J. Y., Monk, B. J., Sikic, B. I., Osann, K., Kapp, D. S. (2008). Analysis of Phase II Studies on Targeted Agents and Subsequent Phase III Trials: What Are the Predictors for Success?. JCO 26: 1511-1518 [Abstract] [Full text]  
• Strevel, E. L., Chau, N. G., Pond, G. R., Murgo, A. J., Ivy, P. S., Siu, L. L. (2008). Improving the Quality of Abstract Reporting for Phase I Cancer Trials. Clin. Cancer Res. 14: 1782-1787 [Abstract] [Full text]  
• Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., Rosenthal, R. (2008). Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy. NEJM 358: 252-260 [Abstract] [Full text]  
• Dartnell, J., Hemming, M., Collier, J., Ollenschlaeger, G. (2008). Putting evidence into context: some advice for guideline writers. Evid. Based Nurs. 11: 6-8 [Full text]  
• Garg, A. X., Hackam, D., Tonelli, M. (2008). Systematic Review and Meta-analysis: When One Study Is Just not Enough. CJASN 3: 253-260 [Full text]  
• Berry, A. M., Davidson, P. M., Masters, J., Rolls, K. (2007). Systematic Literature Review of Oral Hygiene Practices for Intensive Care Patients Receiving Mechanical Ventilation. Am J Crit Care 16: 552-562 [Abstract] [Full text]  
• Dartnell, J., Hemming, M., Collier, J., Ollenschlaeger, G. (2007). Putting evidence into context: some advice for guideline writers. Evid. Based Med. 12: 130-132 [Full text]  
• Lawlor, D. A (2007). Quality in epidemiological research: should we be submitting papers before we have the results and submitting more hypothesis-generating research?. Int J Epidemiol 36: 940-943 [Full text]  
• Moher, D. (2007). Reporting research results: a moral obligation for all researchers/Presenter les resultants de recherche : une obligation morale pour tous les chercheurs. Canadian J. Anesthesia 54: 331-335 [Full text]  
• Hall, R., de Antueno, C., Webber, A. (2007). Publication bias in the medical literature: a review by a Canadian Research Ethics Board: [Biais de publication dans la litterature medicale : un compte rendu d'un Comite d'ethique de recherche canadien]. Canadian J. Anesthesia 54: 380-388 [Abstract] [Full text]  
• Lynch, J. R., Cunningham, M. R.A., Warme, W. J., Schaad, D. C., Wolf, F. M., Leopold, S. S. (2007). Commercially Funded and United States-Based Research Is More Likely to Be Published; Good-Quality Studies with Negative Outcomes Are Not. JBJS 89: 1010-1018 [Abstract] [Full text]  
• Ioannidis, J. P.A., Trikalinos, T. A. (2007). The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. CMAJ 176: 1091-1096 [Abstract] [Full text]  
• Decullier, E., Chapuis, F. (2007). Oral presentation bias: a retrospective cohort study. J. Epidemiol. Community Health 61: 190-193 [Abstract] [Full text]  
• Allison, G. T. (2007). Statistical and Clinical Interpretation of Research Results. J. Am. Podiatr. Med. Assoc. 97: 165-170 [Abstract] [Full text]  
• Bero, L A (2006). Clinical trial registration at Tobacco Control. Tobacco Control 15: 417-418 [Full text]  
• Jorgensen, A. W, Hilden, J., Gotzsche, P. C (2006). Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ 333: 782- [Abstract] [Full text]  
• Liebeskind, D. S., Kidwell, C. S., Sayre, J. W., Saver, J. L. (2006). Evidence of publication bias in reporting acute stroke clinical trials.. Neurology 67: 973-979 [Abstract] [Full text]  
• Falagas, M. E., Rosmarakis, E. S. (2006). Clinical decision-making based on findings presented in conference abstracts: is it safe for our patients?. Eur Heart J 27: 2038-2039 [Abstract] [Full text]  
• Oliveira, G. J., Leles, C. R. (2006). Critical appraisal and positive outcome bias in case reports published in brazilian dental journals.. J Dent Educ 70: 869-874 [Abstract] [Full text]  
• Gluud, L. L. (2006). Bias in Clinical Intervention Research. Am J Epidemiol 163: 493-501 [Abstract] [Full text]  
• Gold, J. L., Studdert, D. M. (2005). Clinical Trials Registries: A Reform that is Past Due. J Law Med Ethics 33: 811-820  
• Littner, Y., Mimouni, F. B., Dollberg, S., Mandel, D. (2005). Negative Results and Impact Factor: A Lesson From Neonatology. Arch Pediatr Adolesc Med 159: 1036-1037 [Abstract] [Full text]  
• Williamson, P R, Gamble, C, Altman, D G, Hutton, J L (2005). Outcome selection bias in meta-analysis. Stat Methods Med Res 14: 515-524 [Abstract]  
• Metcalfe, A. (2005). Universal precautions: A review of knowledge, compliance and strategies to improve practice. Journal of Research in Nursing 10: 549-550  
• Ioannidis, J. P. A. (2005). Contradicted and Initially Stronger Effects in Highly Cited Clinical Research. JAMA 294: 218-228 [Abstract] [Full text]  
• Decullier, E., Lheritier, V., Chapuis, F. (2005). Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 331: 19- [Abstract] [Full text]  
• Hartling, L., McAlister, F. A., Rowe, B. H., Ezekowitz, J., Friesen, C., Klassen, T. P. (2005). Challenges in Systematic Reviews of Therapeutic Devices and Procedures. ANN INTERN MED 142: 1100-1111 [Abstract] [Full text]  
• Cure, S., Chua, W. L., Duggan, L., Adams, C. (2005). Randomised controlled trials relevant to aggressive and violent people, 1955-2000: a survey. Br. J. Psychiatry 186: 185-189 [Abstract] [Full text]  
• Hartling, L., Craig, W. R., Russell, K., Stevens, K., Klassen, T. P. (2004). Factors Influencing the Publication of Randomized Controlled Trials in Child Health Research. Arch Pediatr Adolesc Med 158: 983-987 [Abstract] [Full text]  
• Dowd, M. D. (2004). Breaching the Contract: The Ethics of Nonpublication of Research Studies. Arch Pediatr Adolesc Med 158: 1014-1015 [Full text]  
• Maher, C. G, Sherrington, C., Elkins, M., Herbert, R. D, Moseley, A. M (2004). Challenges for Evidence-Based Physical Therapy: Accessing and Interpreting High-Quality Evidence on Therapy. ptjournal 84: 644-654 [Abstract] [Full text]  
• Landon, B. E., Wilson, I. B., McInnes, K., Landrum, M. B., Hirschhorn, L., Marsden, P. V., Gustafson, D., Cleary, P. D. (2004). Effects of a Quality Improvement Collaborative on the Outcome of Care of Patients with HIV Infection: The EQHIV Study. ANN INTERN MED 140: 887-896 [Abstract] [Full text]  
• Krzyzanowska, M. K., Pintilie, M., Brezden-Masley, C., Dent, R., Tannock, I. F. (2004). Quality of Abstracts Describing Randomized Trials in the Proceedings of American Society of Clinical Oncology Meetings: Guidelines for Improved Reporting. JCO 22: 1993-1999 [Abstract] [Full text]  
• Hirsch, L. (2004). Randomized clinical trials: What gets published, and when?. CMAJ 170: 481-483 [Full text]  
• Krzyzanowska, M. K., Pintilie, M., Tannock, I. F. (2003). Factors Associated With Failure to Publish Large Randomized Trials Presented at an Oncology Meeting. JAMA 290: 495-501 [Abstract] [Full text]  
• Lexchin, J., Bero, L. A, Djulbegovic, B., Clark, O. (2003). Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326: 1167-1170 [Abstract] [Full text]  
• Melander, H., Ahlqvist-Rastad, J., Meijer, G., Beermann, B. (2003). Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326: 1171-1173 [Abstract] [Full text]  
• Baird, P. (2003). Getting it right: industry sponsorship and medical research. CMAJ 168: 1267-1269 [Full text]  
• Mabry, I. R., Richmond, T., Bialostozky, A., Rushton, J. (2003). Interpreting Negative Results: Postpartum Interview Position Not Associated With Improved Outcomes. Arch Pediatr Adolesc Med 157: 333-335 [Full text]  
• de Camargo, K. R. Jr (2002). The Thought Style of Physicians: Strategies for Keeping up with Medical Knowledge. Social Studies of Science 32: 827-855 [Abstract]  
• Wyatt, J, Guly, H (2002). Identifying the research question and planning the project. Emerg. Med. J. 19: 318-321 [Abstract] [Full text]  
• Slimmon, D., Bennell, K., Brukner, P., Crossley, K., Bell, S. N. (2002). Long-Term Outcome of Fasciotomy with Partial Fasciectomy for Chronic Exertional Compartment Syndrome of the Lower Leg. Am J Sports Med 30: 581-588 [Abstract] [Full text]  
• Field, T. S., Hill, M. D., Connor, M. D. (2002). Weather, Chinook, and Stroke Occurrence * Editorial Comment. Stroke 33: 1751-1758 [Abstract] [Full text]  
• Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J. W., Zhu, Q., Reiling, J., Pace, B. (2002). Publication Bias in Editorial Decision Making. JAMA 287: 2825-2828 [Abstract] [Full text]  
• Dickersin, K., Olson, C. M., Rennie, D., Cook, D., Flanagin, A., Zhu, Q., Reiling, J., Pace, B. (2002). Association Between Time Interval to Publication and Statistical Significance. JAMA 287: 2829-2831 [Abstract] [Full text]  
• Moher, D., Schachter, H. M. (2002). Potential solutions to the problem of conducting systematic reviews of new health technologies. CMAJ 166: 1674-1675 [Full text]  
• Klassen, T. P., Wiebe, N., Russell, K., Stevens, K., Hartling, L., Craig, W. R., Moher, D. (2002). Abstracts of Randomized Controlled Trials Presented at the Society for Pediatric Research Meeting: An Example of Publication Bias. Arch Pediatr Adolesc Med 156: 474-479 [Abstract] [Full text]  
• Stewart, L. A., Tierney, J. F. (2002). To IPD or not to IPD?: Advantages and Disadvantages of Systematic Reviews Using Individual Patient Data. Eval Health Prof 25: 76-97 [Abstract]  
• Dickersin, K. (2002). Systematic reviews in epidemiology: why are we so far behind?. Int J Epidemiol 31: 6-12 [Full text]  
• Yaphe, J., Edman, R., Knishkowy, B., Herman, J. (2001). The association between funding by commercial interests and study outcome in randomized controlled drug trials. Fam Pract 18: 565-568 [Abstract] [Full text]  
• McNeil, B. J. (2001). Hidden Barriers to Improvement in the Quality of Care. NEJM 345: 1612-1620 [Full text]  
• Haidich, A.-B., Ioannidis, J. P. A. (2001). Effect of Early Patient Enrollment on the Time to Completion and Publication of Randomized Controlled Trials. Am J Epidemiol 154: 873-880 [Abstract] [Full text]  
• McQueen, M. J. (2001). Overview of Evidence-based Medicine: Challenges for Evidence-based Laboratory Medicine. Clin. Chem. 47: 1536-1546 [Abstract] [Full text]  
• Sterne, J. A C, Egger, M., Smith, G. D. (2001). Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ 323: 101-105 [Full text]  
• Evans, A., Kalra, L. (2001). Are the Results of Randomized Controlled Trials on Anticoagulation in Patients With Atrial Fibrillation Generalizable to Clinical Practice?. Arch Intern Med 161: 1443-1447 [Abstract] [Full text]  
• Sutton, A. J, Song, F., Gilbody, S. M, Abrams, K. R (2000). Modelling publication bias in meta-analysis: a review. Stat Methods Med Res 9: 421-445 [Abstract]  
• Loewen, P. S., Marra, C. A., Zed, P. J. (2000). 5-HT3 receptor antagonists vs traditional agents for the prophylaxis of postoperative nausea and vomiting. Canadian J. Anesthesia 47: 1008-1018 [Abstract]  
• Gøtzsche, P. C (2000). Why we need a broad perspective on meta-analysis. BMJ 321: 585-586 [Full text]  
• Shields, P. G. (2000). Publication Bias Is a Scientific Problem with Adverse Ethical Outcomes: The Case for a Section for Null Results. Cancer Epidemiol. Biomarkers Prev. 9: 771-772 [Full text]  
• WEST, R R (2000). Evidence based medicine overviews, bulletins, guidelines, and the new consensus. Postgrad. Med. J. 76: 383-389 [Full text]  
• Schwid, S. R., Trotter, J. L. (2000). Lessons from linomide: A failed trial, but not a failure. Neurology 54: 1716-1717 [Full text]  
• Coleman, B. D., Khan, K. M., Kiss, Z. S., Bartlett, J., Young, D. A., Wark, J. D. (2000). Open and Arthroscopic Patellar Tenotomy for Chronic Patellar Tendinopathy: A Retrospective Outcome Study. Am J Sports Med 28: 183-190 [Abstract] [Full text]  
• Mariani, L., Marubini, E. (2000). Content and Quality of Currently Published Phase II Cancer Trials. JCO 18: 429-429 [Abstract] [Full text]  
• Tonks, A. (1999). Registering clinical trials. BMJ 319: 1565-1568 [Full text]  
• Johansen, H. K., Gotzsche, P. C. (1999). Problems in the Design and Reporting of Trials of Antifungal Agents Encountered During Meta-analysis. JAMA 282: 1752-1759 [Abstract] [Full text]  
• Jaakkimainen, R L., Boyle, E., Tudiver, F. (1999). Is Helicobacter pylori associated with non-ulcer dyspepsia and will eradication improve symptoms? A meta-analysis. BMJ 319: 1040-1044 [Abstract] [Full text]  
• Whatling, P. (1999). Having non-medical readers of papers on internet will enhance peer review. BMJ 318: 1144-1144 [Full text]  
• Sykes, R. (1998). Being a modern pharmaceutical company. BMJ 317: 1172-1180 [Full text]  
• Kunz, R., Oxman, A. D (1998). The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 317: 1185-1190 [Abstract] [Full text]  
• Plotnick, L. H, Ducharme, F. M (1998). Should inhaled anticholinergics be added to beta 2 agonists for treating acute childhood and adolescent asthma? A systematic review. BMJ 317: 971-977 [Abstract] [Full text]  
• Rush, J. (1998). Cleaning the umbilical cord with water rather than alcohol shortened the time to separation with no change in colonisation. Evid. Based Nurs. 1: 112-112 [Full text]  
• Rush, J. (1998). Review: topical agents for cord care have not been shown to be effective in newborn infants in developed countries. Evid. Based Nurs. 1: 113-113 [Full text]  
• Misakian, A. L., Bero, L. A. (1998). Publication Bias and Research on Passive Smoking: Comparison of Published and Unpublished Studies. JAMA 280: 250-253 [Abstract] [Full text]  
• McKibbon, K A., Marks, S. (1998). Searching for the best evidence. Part 1: where to look. Evid. Based Nurs. 1: 68-70 [Full text]  
• Clarke, M., Stewart, L., Arida, A. U., Ioannidis, J. P. A., Jadad, A. R., Rennie, D. (1998). Time Lag Bias in Publishing Clinical Trials. JAMA 279: 1952-1953 [Full text]  
• Gøtzsche, P. C, Johansen, H. K. (1998). Meta-analysis of short term low dose prednisolone versus placebo and non-steroidal anti-inflammatory drugs in rheumatoid arthritis. BMJ 316: 811-818 [Abstract] [Full text]  
• Stuck, A. E, Rubenstein, L. Z, Wieland, D., Vandenbroucke, J. P, Irwig, L., Macaskill, P., Berry, G., Glasziou, P., Seagroatt, V., Stratton, I., Egger, M., Smith, G. D., Minder, C., Langhorne, P., Song, F., Gilbody, S. (1998). Bias in meta-analysis detected by a simple, graphical test. BMJ 316: 469-469 [Full text]  
• Jadad, A. R., Rennie, D. (1998). The Randomized Controlled Trial Gets a Middle-aged Checkup. JAMA 279: 319-320 [Full text]  
• Julian, D., Le Lorier, J., Grégoire, G. (1998). Meta-analysis and the meta-epidemiology of clinical research. BMJ 316: 311b-311 [Full text]  
• Egger, M., Smith, G. D. (1998). meta-analysis bias in location and selection of studies. BMJ 316: 61-66 [Full text]  
• Naylor, C D. (1997). Meta-analysis and the meta-epidemiology of clinical research. BMJ 315: 617-619 [Full text]