Rapid Responses to:

EDUCATION AND DEBATE: Ted J Kaptchuk Effect of interpretive bias on research evidence BMJ 2003; 326: 1453-1455 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

mechanism bias

Thomas M Zipp   (27 June 2003)

Prestudy bias in statistics

Lorne Basskin, PharmD, Fort Lauderdale, FL, USA, 33026   (7 July 2003)

Donor's desirability bias

Syed Abdul Mujeeb   (12 July 2003)

Statistical error rates in the medical literature are high.

A.K. Al-Sheikhli.   (26 September 2003)

mechanism bias 27 June 2003
Thomas M Zipp, physician MetroHealth Medical Center, Clevelan 44109 Send response to journal: Re: mechanism bias	external validity, to a large degree, depends on scientific data that is often "mechanistic". besides an awareness of potential bias what suggestions do you have ? After all, decisions have to be made. Competing interests:   None declared
Prestudy bias in statistics 7 July 2003
Lorne Basskin, PharmD, Director of Clinical Programs Procare Pharmacy, Fort Lauderdale, FL, USA, 33026 Send response to journal: Re: Prestudy bias in statistics	Sirs, I found your article discussing the impact of prestudy bias to be an excellent discussion. Let me add one more example of such bias that seems to be rather prevalent in the literature. As one knows, studies are designed to show that "a statistically significant difference" exists between the outcomes of two alternative treatments. When one fails to find such a difference, the temptation for the authors is to conclude, often incorrectly, that no difference exists between the two treatments: i.e. the treatment under investigation is "just as good" as the gold standard. As we know, to make such a statement, the study needs to have adequate statistical power to ensure that the chance of a beta error, or that of incorrectly accepting the null hypothesis, is sufficiently small. Since power can generally be increased by enlarging the sample size, it has become in vogue for researchers to retrospectively calculate power and state the potential error in making a statement of "no difference". However, it seems to have become equally acceptable for researchers who fail to have sufficient power to qualify their results in a way that makes the actual power meaningless. For example, one study recently cites "While the study failed to have suficient power to confirm the findings that the drugs were not different, had the sample size been increased from 10 to 180, then the power would have been sufficient to so state". In this way, the researcher implies that it's only some silly statistical convention that is preventing he or she from stating that no difference in fact exists between the two drugs. Of course, we know that had the sample size been so increased, there is no guarantee as to what the researchers may have found. I have read similar statements when a researcher finds the variance or standard deviation too large for their liking, and qualifies those results in the same way, since increasing sample size "IN THE ABSENCE OF NO OTHER CHANGES" does in fact reduce the variance, standard deviation, and standard error of the mean. For those who cannot resist such hypothetical conclusions, I have only one suggestion (tongue-in-cheek). Next time, skip doing the study and save yourself a fortune. Examine one patient, report the results, and speculate away that whatever you find could be of greatest statistical significance if only the study had been conducted with more people. Competing interests:   None declared
Donor's desirability bias 12 July 2003
Syed Abdul Mujeeb, Asstt.Prof/AIDS Surveillance Center, Jinnah Postgraduate Medical Center, Karachi,75510, Pakistan Send response to journal: Re: Donor's desirability bias	Sir, I agree with the author that research outcome seems to be affected by what the researcher looking. Similarly development of idea and study design also get affected with the donor's desire and amount of allocated resources- a donor's/ sponsor'sdesribality bias. Competing interests:   None declared
Statistical error rates in the medical literature are high. 26 September 2003
A.K. Al-Sheikhli., Loc.Consultant Psychiatrist. Medical centre,Nuneaton,England. Send response to journal: Re: Statistical error rates in the medical literature are high.	EDITOR-Kaptchuk's interesting paper, Effect of interpretive bias on research evidence[1], draws our attention to published papers in medical journals, and in particular drug trials studies. McGuigan[2] showed that out of 248 papers, 66%presented numerical results, 40% of the 164 papers contained statistical errors. Davies[3] reviewed 29 analytical papers; 10 methodological errors were found. Peron-Magnan[4] found that errors of statistical procedures are in important amount of papers in the reliable psychiatric journals. Thanking you, Yours sincerely, A.K.Al-Sheikhli References, 1.Kaptchuk TJ,Effect of interpretive bias on research evidence,BMJ (2003);326:1453-1455. 2.McGuigan SM,The use of statistics in the British Journal of Psychiatry,Br J Psychiatry(1995);167:683-688. 3.Davies J,A critical survey of scientific methods in two psychiatry journals,Aust N Z J Psychiatry(1987);21:367-373. 4.Peron-Magnan P,Importance and limits of statistical methods in psychiatry,Ann Med Psychol(1992);150:187-191. Competing interests:   None declared