Rapid Responses to:

EDITOR'S CHOICE: Kamran Abbasi Do mistakes matter? BMJ 2004; 328: 0-g [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Single Patient Based Medicine and Statistical Inconsistencies of Peer Reviews Papers.

Sergio Stagnaro   (11 June 2004)

misuse of statistics in medical research

Tzippy Shochat   (11 June 2004)

Can we believe what we hear ?

Abid Mahmood   (12 June 2004)

Mistakes as a continuing learning paradigm?

Dr.Naseem A. Qureshi MD, IMAPA, LMIPS   (17 June 2004)

Identifying and preventing statistical errors

Joan T McClusky   (17 June 2004)

BMJ statistical errors

Tim J Cole, Douglas Altman, Deborah Ashby, Mike Campbell, Jonathan Deeks, Stephen Evans, Hazel Inskip, Julie Morris, and Gordon Murray   (19 July 2004)

Single Patient Based Medicine and Statistical Inconsistencies of Peer Reviews Papers. 11 June 2004
Sergio Stagnaro, Specialist in Blood, Gastrointestinal, and Metabolic Diseases. Researcher in Biophhysical Semeiotic Via Erasmo Piaggio 23/8 16037 Riva Trigoso (Genova) Italy Send response to journal: Re: Single Patient Based Medicine and Statistical Inconsistencies of Peer Reviews Papers.	Sirs, The interesting considerations of K. Abbasi’s intriguing editorial, e.g., about statistical inconsistencies in 38% of papers in Nature and 25% in the BMJ, according to study recently published on Biomedcentral (www.biomedcentral.com), as well as the evidence that suggests the world regards scientific peer review is imperfect, in my oipion, worsens in a tremendous way if we take into account that beside EBM does really esists the Single Patient Based Medicine (2), suggested also by Network of Competent Authorities Health Europe, as a tool in primary prevention: website http://www.epha.org/a/355 URL: http://europa.eu.int/comm/health/ph_information/documents/ev_20030710_co01_en.pdf (Fo further information, See HONCode website 233736, www.semeioticabiofisica.it, URL: www.semeioticabiofisica.it/Documenti/Ita/Costituzioni%20DM.doc). Although <> BMJ is “rare among scientific journals because all published research papers have been evaluated by a statistician and statisticians are present at our editorial meetings when we decide which papers to publish”, based in a 46-year-long clinical experience, I think that transferring the results gathered in such studies, performed on a largest cases popupation, on a single patient, cannot not avoid some other mistakes. In fact, even in western technological Medicine world, we need now-a-day a steady single patient’s knowledge: for example, diabetic, rheumatic, dyslipidemic, glaucomatous, hypertensive, osteoporotic, “oncological” (3), a.s.o.,constitutions. For example, there is convincing evidene that cigarette smoking is a risk factor for type 2 diabetes. Cigarette smoking has been consistently associated with a relatively small byt significantly increased risk of type 2 diabetes in both men (4) and women (5) in large prospective cohort studies. However, only individuals with both “diabetic “ and “dyslipidemic” biophysical-semeiotic constitutions – according to Josslin’s 35-year-old statement – can suffer from diabetes mellitus type 2. (6) (See above-cites website, Diabetes, 6 articles). 1)Abbasi K. Do mistakes matter? BMJ 2004;328 (12 June), doi:10.1136/bmj.328.7453.0-g 2) Stagnaro S. “Single Patient Based Medicine” versus EBM. http://bmj.com/cgi/eletters/326/7398/1048#32299 (16 May 2003) 3) Stagnaro-Neri Marina, Stagnaro Sergio. Introduzione alla Semeiotica Biofisica. Il Terreno oncologico”. Travel Factory SRL., Roma, 2004. http://www.travelfactory.it/semeiotica_biofisica.htm 4) Rimmm EB., Chan J., Stampfer MJ., et al. Prospective study of cigarette smoking, alcohol use, and the risk of diabetes in men. BMJ 1995; 310 555- 559. 5) Rimmm EB., Manson JE., Stampfer MJ., et al. A prospective study of cigarette smoking and the risk of diabetes in women. Am.J Public Health 1993; 83:211-214. 6) Stagnaro S., Diet and Risk of Type 2 Diabetes. N Engl J Med. 2002 Jan 24;346(4):297-298. letter [PubMed –indexed for MEDLINE]. Competing interests: None declared
misuse of statistics in medical research 11 June 2004
Tzippy Shochat, statistical consultant Israel 71908 Send response to journal: Re: misuse of statistics in medical research	The attempt to summarize statistical inconsistincies in medical papers is applaudable, though it seems that the search for statistical misuse occurred at the least appropiate place - p-values. In my 18 years as a statistical consultant, all errors I've encountered have been at the planning phase of the study: Formulating a wrong or imprecise study hypothesis. Choosing a biased sample. Improper randomization. Confusing correlation with causality. Generating hypotheses from a pilot study, but ignoring the need for a followup research (this also comes under the heading of data-dredging) Using inaduquate sample sizes. And (though quite rarely) fitting the wrong statistical model. Looking for errors in the consistincy between p-values and test statistics will only serve to equate statistical consulting with p-value conjuring. I would much rather see an article that checks the quality of the research planning, and not how many digits were printed in a table. Competing interests: None declared
Can we believe what we hear ? 12 June 2004
Abid Mahmood, Consultant Microbiologist United Nations Level II Hospital Tubmanbarg, Liberia Send response to journal: Re: Can we believe what we hear ?	I fully agree with the author that we get conflicting results of many studies published in international journals of good repute and at many a times altogether opposite results of similar studies. It is difficult for the reader to draw conclusions out of these . It is not only the good statistical evaluation which can give reliable results. Most of the studies sent to journals these days have good statistical work done before hand. But the tempering of the actual data to draw some good conclusions is the main problem. It is again done by most of the authors due to the strict selection criteria for the papers by the journals. Most of the studies conducted in the developing countries especially are not well planned. Even in the planned studies the patients do not present the way the criteria are set in the studies. The doctors and clinicians themselves are so overworked and busy that they can not extract the required information from the patients in addition to the fact that the patients are mostly illiterate and cannot give the correct information. More over there is dearth of lab and other investigations even if the patient can afford these. Over and above this the follow up mechanisms are so poor and patients themselves once find a little improvement do not turn up for follow up. In this situation the data collected is a broken one and complete information is available in only limited number of cases. So to make a study to be acceptable to an international journal the information is at times cooked up. This leads to false studies despite being correct according to statistical evaluation. But the actual facts observed by the doctors are there and are very important for the future management of the patients. The doctors are basically the professional workers. They have very limited training apart from having time for the job of writing. In this regard I would like to suggest that the journals should encourage the doctors to provide raw data which is actually correct. Then the experts in the editorial board should suggest to the doctors after evaluating the data that whether some good results/conclusions can be obtained from this data. Then the submitting doctors make the study in a paper format and submit for publication. This is the only way we can get correct information from the studies and the reader can get benefit out of these and apply this knowledge to his patients. Competing interests: None declared
Mistakes as a continuing learning paradigm? 17 June 2004
Dr.Naseem A. Qureshi MD, IMAPA, LMIPS, Medical Director(A), Director CME&R Buraidah Mental Health Hospital, Postcode:2292, Saudi Arabia Send response to journal: Re: Mistakes as a continuing learning paradigm?	Sir: Every mistake has both multiple meanings and consequences. Every mistake is repeatable sparingly by one who commits it for the first time but more by others. Every individual secretly analyse cognitively the consequences of the mistake to be made. Largely many individuals rectify their mistakes and learn from them and hence mistakes may model their moral behaviours. Notably, mistakes may be considered as a continuing learning paradigm(s) in our life span. Those who commit mistakes, rectify it, don't repeat it, and learn from its no or bad consequences, are highly adjustable, flexible, productive, diplomatic, have high moral, decent, and intellectual people. Base on this construct, the other side of the coin is well understood. Notably, the learning paradigm of mistakes can be applied to all pursuits in our life, medical or nonmedical. Finally, arguably, mistakes must be avoided but by nature humans tend to err. Reference: Kamran Abbasi. Do mistakes matter? BMJ 2004; 328: 0-g Competing interests: None declared
Identifying and preventing statistical errors 17 June 2004
Joan T McClusky, Medical writer New York, NY Send response to journal: Re: Identifying and preventing statistical errors	One approach to identifying and preventing statistical errors that is fairly effective is to have the authors submit, when asked, the specific pages of data in question, with all cited statistics highlighted. The need to prepare these pages--whether or not the data are eventually requested-- is also a very good means of double checking and preventing errors in the first place. Competing interests: None declared
BMJ statistical errors 19 July 2004
Tim J Cole, Professor of medical statistics Institute of Child Health, University College London WC1N 1EH, Douglas Altman, Deborah Ashby, Mike Campbell, Jonathan Deeks, Stephen Evans, Hazel Inskip, Julie Morris, and Gordon Murray Send response to journal: Re: BMJ statistical errors	Kamran Abbasi in Editor’s choice (12 June) discussed a study that found statistical errors in 25% of papers published by the BMJ in 2001.[1] As BMJ statistical advisers we aim to improve the quality of published papers by ensuring that their conclusions are consistent with the data. To this end we hope to identify important errors that affect the interpretation of the findings, but care less about more minor errors. Any stricter policy would be impossibly time-consuming. That said, we recognise that important errors do slip through from time to time, and are always keen to improve our performance. The particular errors flagged in the paper [1] were inconsistencies between test statistics and P values. Out of 63 tests seven (11%) were wrong (for example chi-squared on 1 d.f. = 4.2, P reported = 0.024, P actual = 0.0404). Yet in no case did the error affect the test’s interpretation as to whether or not the results could have arisen by chance. This supports our belief that more extreme errors are likely to be weeded out at the review stage. The paper is disappointing in focussing on P values and by implication hypothesis testing. By contrast the BMJ’s policy is to present the main findings as confidence intervals where the emphasis is on estimation.[2] 1. Garcia-Berthou E, Alcaraz C. Incongruence between test statistics and P values in medical papers. BMC Medical Research Methodology 2004;4:13. 2. Gardner MJ, Altman DG. Confidence intervals rather than P values - estimation rather than hypothesis testing. BMJ 1986;292:746-50. Competing interests: We take responsibility for the statistical quality of papers published in the BMJ to the extent that the study design, data and analysis appear appropriate and internally consistent, and that they support the conclusions drawn.