Letters

Peer review of statistics in medical research

BMJ 2002; 325 doi: https://doi.org/10.1136/bmj.325.7362.491/a (Published 31 August 2002) Cite this as: BMJ 2002;325:491

This article has a correction. Please see:

Journal reviewers are even more baffled by sample size issues than grant proposal reviewers

  1. Merrick Zwarenstein (merrick.zwarenstein@mrc.ac.za), director,health systems research
  1. Medical Research Council, Tygerberg, South Africa, 7505
  2. Laxdale, Stirling FK7 9JQ
  3. Centre for Statistics in Medicine, Institute of Health Sciences, Oxford OX3 7LF
  4. Thomas C Chalmers Centre for Systematic Reviews, University of Ottawa, Children's Hospital of Eastern Ontario Research
  5. Family Health International and Department of Obstetrics and Gynecology, School of Medicine, University of North Carolina at Chapel Hill, PO Box 13950 Research Triangle Park, NC 27709, USA
  6. University of California, San Francisco, CA 94143, USA
  7. Research and Development Support Unit, PPMS, University of Plymouth, Room N17, ITTC Building, Tamar Science Park, Plymouth PL6 8BX
  8. Department of Mental Health, University of Exeter, Exeter EX2 5AF

    EDITOR—With reference to the article by Bacchetti,1 the confusion surrounding sample size estimates in research protocols elicits quite strange responses from reviewers when they are faced with the completed research in a report submitted to a journal for publication. One of our submissions was rejected because the planned sample size was not attained. But the effect size was greater in the study than we had anticipated, and thus the difference was of clinical and statistical significance. Another submission met the same fate for a similar reason—it was an equivalence trial—and even though the difference in effect between intervention and control arms (and both sides of the confidence interval around this difference) lay completely within the equivalence interval, the fact that the planned sample size was not attained in some way invalidated the result in the mind of the reviewer.

    Although sample size estimation is useful in considering the feasibility of conducting a study (and protocol reviewers should discourage funding for studies that are plainly too small to be meaningful) attainment of the planned sample size does not seem to me to be a useful indicator by which journal reviewers should assess the validity of a completed research report in which clinically and statistically meaningful results have been obtained.

    Footnotes

    • Conflict of interest My competing interest in relation to this question is my desire to publish research in the face of overoptimistic sample size estimates in my grant proposals.

    References

    1. 1.

    Rationale for requiring power calculations is needed

    1. David F Horrobin (agreen@laxdale.co.uk), research director.
    1. Medical Research Council, Tygerberg, South Africa, 7505
    2. Laxdale, Stirling FK7 9JQ
    3. Centre for Statistics in Medicine, Institute of Health Sciences, Oxford OX3 7LF
    4. Thomas C Chalmers Centre for Systematic Reviews, University of Ottawa, Children's Hospital of Eastern Ontario Research
    5. Family Health International and Department of Obstetrics and Gynecology, School of Medicine, University of North Carolina at Chapel Hill, PO Box 13950 Research Triangle Park, NC 27709, USA
    6. University of California, San Francisco, CA 94143, USA
    7. Research and Development Support Unit, PPMS, University of Plymouth, Room N17, ITTC Building, Tamar Science Park, Plymouth PL6 8BX
    8. Department of Mental Health, University of Exeter, Exeter EX2 5AF

      EDITOR—The article by Bacchetti with its comments about uncertainties surrounding power calculations prompted me to seek advice about an issue that has implications for clinical research.1 The company I work for, Laxdale Limited, often conducts pilot studies on new entities. Our usual practice is to state in the protocol that there is no reasonable basis for a power calculation. In order …

      View Full Text

      Sign in

      Log in through your institution

      Free trial

      Register for a free trial to thebmj.com to receive unlimited access to all content on thebmj.com for 14 days.
      Sign up for a free trial

      Subscribe