Deaths in trials should always be reportedBMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f4219 (Published 04 July 2013) Cite this as: BMJ 2013;347:f4219
- Jeppe Schroll, PhD student, Nordic Cochrane Centre, Rigshospitalet, Dept 7811, 2100 Copenhagen Ø, Denmark
Researchers generally do not publish what they planned to report in their protocols,1 and important differences can also exist between internal trial reports and published papers.2 It has been suspected that even deaths are sometimes omitted,3 but there is little direct evidence of this.
In the final stages of conducting a Cochrane review about sulfonylurea treatment for patients with type 2 diabetes,4 I realised that we had included only a few trials that considered outcomes important to patients. This was surprising given that we included several drugs that had been approved in the past 30 years, when clinical evaluation in trials was required. I searched for protocols on http://clinicaltrials.gov, but these trials were conducted before registration was mandatory.
I turned to the website of the US Food and Drug Administration (FDA) to look for reviews that might give clues about unpublished trials. Useful FDA reviews had compared the diabetes drugs repaglinide and nateglinide with sulfonylurea drugs. The sulfonylurea drugs themselves had been approved before the FDA started to publish reviews online. The FDA’s repaglinide review5 described five one-year trials. Having found only three of these trials, I contacted Novo Nordisk, and was told that the company did not share data that were not already published. After a second request, Novo Nordisk agreed to share data and sent five internal reports of six pages each.
The internal report of trial 050 described two deaths in the repaglinide arm: “a possible relation to the trial product could not be excluded by the investigator,” it read. Despite the investigator’s concern, the two deaths were not reported in the published paper,6 which said: “The safety profile of repaglinide is similar to that of glyburide [glibenclamide], and there was no difference in adverse events.” Serious adverse events—including the two deaths—were outlined in the internal report but had been omitted in the published paper.6
In trial 048, one death was reported in the internal report, this time in the comparator arm, but not in the published paper.7 In trial 049, the internal report did not describe any deaths, but the published paper reported three deaths in the repaglinide arm and one in the sulfonylurea arm.8 The published paper for trial 049 also reported 19 cardiovascular events (5%) in the repaglinide arm compared with only four (2%) in the sulfonylurea comparator—but nonetheless, the conclusion was that repaglinide was well tolerated and safe.8 One of the never published trials (trial 046) had similar outcomes, with 25 cardiovascular events (14%) in the repaglinide arm compared with four (5%) in the sulfonylurea arm. However, the difference between groups was downplayed: the internal report concluded that the “frequency of adverse event[s] was similar.” This conclusion was reached even after tolbutamide—an earlier, similar sulfonylurea drug—had been shown to increase the number of cardiovascular deaths.9
I also noted discrepancies in the number of patients with hypoglycaemia. In a published paper on trial 050, 26 (9%) patients in the repaglinide arm and 13 (9%) in the glibenclamide arm “experienced hypoglycemia.”6 But in the internal report, many more patients had a hypoglycaemic reaction (44 (16%) in the repaglinide arm; 20 (14%) in the sulfonylurea arm). Severe hypoglycaemia was defined in the published trial report, but the numbers of patients were not reported.6 In the internal report, four participants in the repaglinide arm and one patient in the sulfonylurea arm had severe hypoglycaemia.
I asked Novo Nordisk about this discrepancy. It did not consider the data unpublished because pooled data from the five trials, including deaths, were published in a review by an independent researcher.10 The company said that the reason for the few published hypoglycaemic events in trial 050 was that it only reported hypoglycaemic events in the maintenance period—but the company gave no explanation for this choice.
The review10 pooled deaths (six deaths for repaglinide), but these differed from that stated in the internal and published reports (nine for repaglinide). The review also erroneously calculated the death rate as 0.1%, when in fact it was 0.5%. It is published as a supplement, and the abstract does not explain that it contains new data. The review’s methods are not described, and only by reading the acknowledgments do you find out that the author had access to Novo Nordisk’s internal reports. Only one of the three published trials are cited, and the “independent” researcher received grants from Novo Nordisk.11
The approval of repaglinide was based on these five trials,5 which were designed to show equivalence with various sulfonylurea drugs for the unvalidated surrogate marker HbA1c (glycated haemoglobin). However, we now know that a drug could have a positive effect on HbA1c while it increases the risk of cardiovascular disease.12 Only three of five trials on repaglinide were published, and there is also a lack of published trials for other diabetes drugs.12
There have been other cases where important adverse outcomes have been omitted from published papers. Cardiovascular events were left out of a paper about rofecoxib (Vioxx),13 which has resulted in the loss of many lives. Additional cardiovascular events were also found in a study of rosiglitazone when the case reports were scrutinised by the FDA.14
The repaglinide trials show that mortality can be omitted in published papers even though the number of deaths was recorded and even though the investigators thought that it might be related to the drug. It should never be assumed that no deaths occurred just because none was reported, which is an especially important caveat for researchers conducting meta-analyses. Companies with a financial interest in downplaying harms are liable not to give an unbiased presentation of the results, which is why we need access to raw data.15 Deaths and serious adverse events should always be reported—not as a pooled analysis in a substandard secondary publication with important errors, but in the original paper.
Cite this as: BMJ 2013;347:f4219
Competing interests: I have read and understood the BMJ Group policy on declaration of interests and have no relevant interests to declare.
Provenance and peer review: Not commissioned; externally peer reviewed.
Log in using your username and password
Log in through your institution
Sign up for a free trial