Re: Dutch trial of probiotics in acute pancreatitis is to be investigated after deaths
Title: Correction of errors in Dutch Probiotica trial remains necessary
Subtitle: Discovery of an unexpected false negative error may help to improve the quality of treatment of acute pancreatitis.
Response to an article on 31 March 2013 in the Dutch daily journal NRC, written by the journalists Joke Mat and Sander Voormolen entitled: “How the death of 24 patients remained without consequences”, and in sequel to my own BMJ-online Rapid Response on 5 September 2009 concerning the Dutch Probiotica Trial.
Introduction: Evidence based medicine (EBM) and randomised clinical trials (RCT) have an important role in the progress of medical science. They permit the discovery of unexpected errors and the subsequent development of new hypotheses that may help to improve the quality of medical health care.
Analysis: The Dutch Propatria trial published in 2008 in the Lancet is such a RCT: It aimed at improving the quality of treatment of patients with acute pancreatitis (ref.1) and at assessment of the prophylactic effect of enterally administrated probiotics on infectious complications and mortality in patients with acute pancreatitis. The expectations were that probiotics would have a positive effect on mortality. However, the results of this carefully planned, placebo-controlled RCT were embarrassing, that is they were in sharp contrast with expectations of the authors: no prophylactic effect of probiotics on infectious complications whatsoever, instead: severe increase of mortality in the group of patients that had received the “extremely safe” probiotic bacteria in their enteral feeding!.The results indicated that 24 (16% of 150) patients in the probiotic group died, compared with only 9 (6% of 150) patients who died in the “placebo group”. In fact in the probiotic group there were 9 deaths more than expected and in the placebo group there were 6 deaths less the expected. "incredible"! In total about 300 patients were included in the trial. The expected mortality in the natural course of the disease, had been estimated by the authors to be about 10% that is 30 patients (10% of 300). The mortality found in the trial was about 33 patients (11 % of 300), that is 3 patients more than expected to occur without the RCT. In the press these results were incorrectly summarized as: “the Dutch trial of probiotics in acute pancreatitis resulted in 15 unexpected deaths”.
In 2009 in a BMJ-online Rapid Response I warned for the first time on dangerous underreporting in the probiotica trial and reminded that such concealment of data is scientific misconduct (ref 2). Later on, after questions in the Parliament the Dutch Healthcare Inspectorate (IGZ) and the Central Committee on Research involving Human Subjects (CCMO) decided to investigate why the trial produced such dramatic results. The investigation remained without answers to the question.
Underreporting: - Analysis of the data reported in the Lancet 2008 showed that mistakes have been made in the planning, concept, title and reporting on the probiotica trial. Firstly: The fact that all patients in both groups received prebiotic, fibre rich enteral feeding was underreported. This is probably the central error made in the report. Secondly: the study was not at all a placebo controlled trial. No, in stead it was a comparative trial, that compared the effect of prebiotics + probiotics versus prebiotics + placebo. The role of prebiotics remained largely underreported, it was more or less concealed, kept under the cap or as we say in Dutch: onder de pet gehouden. Such underreporting of research is scientific misconduct. Such conduct may be unintentional and forgivable but if it is intentional it should lead to retraction of the article (ref.3). (by the way: In the guide for authors of the Lancet it is stated that actual errors should always be corrected). - False positive (“Type 1”) error: In group A, +/- 150 patients, prebiotics + probiotics, 24 patients died, 9 more than expected. This false, toxic, effect should not be attributed to probiotics alone. The 150 patients in this group had a socalled “synbiotic” mixture in their enteral feeding.
-False negative (“Type 2”) error. Please pay attention to what happened in this group. It is group B, the placebo group that received prebiotics + placebo, 6 patients died less than expected. Of course, this favourable effect can not be attributed to the placebo and must be attributed to the prebiotic fibre-rich enteral feeding.
- In 2009 (ref.4) I summarized my ideas as follows: the authors of the Propatria trial have failed to give an adequate account of their RCT. Their underreporting of research is scientific misconduct. It can lead doctors who read such a report to make inappropriate treatment decisions for their patients. Sir Iain Chalmers has repeatedly warned for the dangers of underreporting of clinical research. Correction of the error is needed. However, analysis of the reported data shows that the quality of data collection in the trial is remarkably high and that careful analysis of the data permits the discovery of errors that may lead to the development of new hypotheses that may help to improve the quality of medical care for patients with severe acute pancreatitis. Let’s hope that the authors will retract and correct their publication. This was my opinion in 2009 and since then my opinion has not changed much.
Reading the recent article in the NRC I understood that the scientific misconduct of underreporting and concealment goes on and that in the lay-press the completely false and highly unscientific idea exists that 24 patients have died unnecessary. It seems to me that the time has come that full information about the state of affairs will be given.
In my opinion everybody, clinicians, patients, journalists, researchers should be informed that a major error has been discovered, and that this means that there is good reason to form a new hypothesis. Everybody should be informed that the study was not a placebo controlled trial; that abusively it became a concealed comparative trial, and that one group of patients received prebiotics and the other group received prebiotics plus probiotics. And everybody should understand that it was a comparative study on about 300 patients and that the expected mortality in the natural course of the disease, had been estimated to be about 30 patients (10%), and that the mortality found in the trial was about 33 patients (11%) patients, that is only 3 patients more than expected to occur without the trial. That the idea that 24 patients had died because of scientific errors in the study was completely false.
But we should not forget to praise the authors for the outstanding aspects of their work! Let us forgive them for the errors they have made. Let’s hope that the Lancet will act according to their own guidelines and correct the errors.
1 . Besselink, MG, van Santvoort HC, Buskenc E., et al. Probiotic prophylaxis in predicted severe acute pancreatitis. A randomised, double- blind, placebo controlled trial. Lancet 2008; 371: 651- 59.
2. Chalmers, I. Underreporting research is scientific misconduct. JAMA 1990, 263:1405-8.
3. Nath SB, Marcus SC, Druss BG. Retractions in the research literature: misconduct or mistakes? Med J Aust. 2006; 185:152-4.
4. Hoogenraad TU. BMJ-online Rapid Response on 5 September 2009 concerning the Dutch Probiotica Trial. Bmj.com/content/336/7638.
Competing interests: No competing interests