Errors when statistical hypothesis testingBMJ 2010; 340 doi: https://doi.org/10.1136/bmj.c2348 (Published 05 May 2010) Cite this as: BMJ 2010;340:c2348
All rapid responses
In this lesson authors wrote that "However, no difference was seen
between treatment groups in mean total NHS costs per patient (P=0.89)."
The good provocative question would be:
d) No difference exists between treatment groups in mean total NHS
costs per patient.
Correct answer would be: "wrong". Difference probably do exists, but
is "not seen", may be because it does not exists, or because the power of
the study is low. Statistical power may be low because of the small sample
I decided to write about this rather trivial variant of interpretation
not just because it's wrong but because it can confuse readers and regulators.
The example: recently Russian drug firm Biokad produced the beta-
interferon drug Ronbetal. To prove that it is equivalent to the product on
the market (Betaferon) they did a an open randomized trial on 49 patients
during 6 months. Because trial did not found statistically significant
differences between two drugs, paper concluded that "study demonstrated
equal safety, tolerability and efficacy" of two drugs.
In course of weeks the drug was registered and purchased for all national
The study "Results of clinical study of safety, tolerability and efficacy
of first Russian bioanalog interferon-beta-1b" by E.V. Baidina is not
published yet, but was officially presented to the Multiple Sclerosis
The less malignant cases of misuse of this side of hypothesis testing are
prevalent in medical literature.
Competing interests: No competing interests