Published 4 December 2008, doi:10.1136/bmj.a2299
Cite this as: BMJ 2008;337:a2299

Research

Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols

¹ Mayo Clinic, Rochester, USA, ² Nordic Cochrane Centre, Copenhagen, Denmark, ³ Centre for Statistics in Medicine, University of Oxford, Oxford

Objective To evaluate how often sample size calculations and methods of statistical analysis are pre-specified or changed in randomised trials.

Design Retrospective cohort study.

Data source Protocols and journal publications of published randomised parallel group trials initially approved in 1994-5 by the scientific-ethics committees for Copenhagen and Frederiksberg, Denmark (n=70).

Main outcome measure Proportion of protocols and publications that did not provide key information about sample size calculations and statistical methods; proportion of trials with discrepancies between information presented in the protocol and the publication.

Results Only 11/62 trials described existing sample size calculations fully and consistently in both the protocol and the publication. The method of handling protocol deviations was described in 37 protocols and 43 publications. The method of handling missing data was described in 16 protocols and 49 publications. 39/49 protocols and 42/43 publications reported the statistical test used to analyse primary outcome measures. Unacknowledged discrepancies between protocols and publications were found for sample size calculations (18/34 trials), methods of handling protocol deviations (19/43) and missing data (39/49), primary outcome analyses (25/42), subgroup analyses (25/25), and adjusted analyses (23/28). Interim analyses were described in 13 protocols but mentioned in only five corresponding publications.

Conclusion When reported in publications, sample size calculations and statistical methods were often explicitly discrepant with the protocol or not pre-specified. Such amendments were rarely acknowledged in the trial publication. The reliability of trial reports cannot be assessed without having access to the full protocols.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

Reporting of sample size calculation in randomised controlled trials: review

Pierre Charles, Bruno Giraudeau, Agnes Dechartres, Gabriel Baron, and Philippe Ravaud
BMJ 2009 338: b1732. [Abstract] [Full Text] [PDF]

Waiving confidentiality for the greater good

An-Wen Chan, Ross Upshur, Jerome A Singh, Davina Ghersi, François Chapuis, and Douglas G Altman
BMJ 2006 332: 1086-1089. [Full Text] [PDF]

Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1)

Karmela Krleza-Jeric, An-Wen Chan, Kay Dickersin, Ida Sim, Jeremy Grimshaw, Christian Gluud for the Ottawa Group
BMJ 2005 330: 956-958. [Extract] [Full Text] [PDF]

Trial protocols at the BMJ

Giselle Jones and Kamran Abbasi
BMJ 2004 329: 1360. [Extract] [Full Text] [PDF]

Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group

Heloisa P Soares, Stephanie Daniels, Ambuj Kumar, Mike Clarke, Charles Scott, Suzanne Swann, and Benjamin Djulbegovic
BMJ 2004 328: 22-24. [Abstract] [Full Text] [PDF]

Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications

Hans Melander, Jane Ahlqvist-Rastad, Gertie Meijer, and Björn Beermann
BMJ 2003 326: 1171-1173. [Abstract] [Full Text] [PDF]

Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis

Dean Fergusson, Shawn D Aaron, Gordon Guyatt, and Paul Hébert
BMJ 2002 325: 652-654. [Extract] [Full Text] [PDF]

Are trust doctors the new lost tribe?

Nicola Cooper and Bill Burr
BMJ 2002 325: 491. [Extract] [Full Text] [PDF]

What is meant by intention to treat analysis? Survey of published randomised controlled trials

Sally Hollis and Fiona Campbell
BMJ 1999 319: 670-674. [Abstract] [Full Text] [PDF]

This article has been cited by other articles:

• Charles, P., Giraudeau, B., Dechartres, A., Baron, G., Ravaud, P. (2009). Reporting of sample size calculation in randomised controlled trials: review. BMJ 338: b1732-b1732 [Abstract] [Full text]  

Rapid Responses:

Read all Rapid Responses

Statistical Power for Evaluating Adverse Drug Reactions in Randomised Controlled Trials -- Transparency Needed

Chi-Tai Fang, et al.
bmj.com, 27 Jan 2009 [Full text]

Re: Statistical Power for Evaluating Adverse Drug Reactions in Randomised Controlled Trials -- Transparency Needed

ZEKRIA IBRAHIMI
bmj.com, 29 Jan 2009 [Full text]

A post script--power and protocol: the dangers of manipulating data and methods

ZEKRIA IBRAHIMI
bmj.com, 2 Mar 2009 [Full text]

Online poll

Find out more >>