Why we can’t trust clinical guidelines

Dr. Bracken's responses vary puzzlingly over time. Here, we compare his new answers to the ones in a comment thread (1) in this journal in 2008, which also cites older articles criticizing the NASCIS reports.

Both now and in 2008, Dr Bracken calculates from the CRASH data while nonetheless protesting against applying its results to NASCIS, a comparison he termed (1) "bogus."

As far as numerical conclusions from the CRASH study, in 2008 Dr. Bracken, without explaining his work, wrote (1), "If the calculation were done correctly the number [of additional deaths due to the use of MPSS in patients with acute SCI] is <500." Now, though, Dr. Bracken sees no increased mortality at all: he writes, simply, "CRASH found 6.3% v 5.8% mortality in mild head injury cases with and without steroid administration respectively, possibly a chance difference."

Curiously, this new assertion starts with new numbers: Dr. Bracken earlier reported (2), "patients with mild head injuries were not at significantly increased risk of death in CRASH (4.2% v 4.1%)." Whatever the true numbers, we pointed out (1) that the claim of a lower MPSS mortality in mild, compared to severe, injuries contradicted the CRASH trial report (3), which stated, “The relative increase in deaths due to corticosteroids did not differ by injury severity (p=0.22).”

Is it true that, as Dr. Bracken writes, comparison of CRASH safety to NASCIS is "clinically naive?" Or is it the NASCIS safety data that may be misleading?

To establish efficacy, the NASCIS sample was limited to mild trauma without head or chest injury, which are common in SCI. Dr. Bracken, not medically trained or experienced, has never made allowance that the safety results from NASCIS cannot be extrapolated to actual clinical practice, which would inevitably extend to a more diverse population, including the CRASH subjects.

Actually, Dr. Ian Roberts, CRASH Clinical Co-ordinator and perhaps a more disinterested observer, wrote (1), "The results of the MRC CRASH Trial of corticosteroids in patients with traumatic brain injury, which showed evidence of harm rather than benefit, should call into question the use of corticosteroids in patients with spinal cord injury."

Dr. Bracken writes, "No Practice Guideline should have recommended that spinal cord patients with significant head injury be administered steroids." Yet, we pointed out to him (1) that his main publications recommended MPSS enthusiastically and none expressed reservations for head injury or anything else: not the NIH fax to emergency rooms, not the NEJM NASCIS 2 report, not the JAMA NASCIS 3 report, and not his Cochrane Reviews article. (Those four reports hailed MPSS as a major breakthrough and, despite having no official status, gave practice guidelines that mandated certain regimens. But Dr. Bracken later wrote (1), "the therapy was never claimed to be other than of modest benefit.")

Dr. Bracken tasks Ms. Lenzer with missing "the real story," which he believes to be the failure of the neurosurgical community to follow up. He is correct here in his facts: there are 515 studies of SCI listed on ClinicalTrials.gov, but adding the term "methylprednisolone" shows no new or ongoing study. It is his interpretation that is wrong: there is evidently a massive consensus that MPSS in SCI is not worth pursuing.

"The real story?" Notwithstanding Dr. Bracken's newest calculation and assertions, there is too much evidence that the increased death rate from MPSS in SCI is not zero. Even by Dr. Bracken's older, unexplained and not evaluable, calculation, there may have been 500 deaths by 2008 -- and this would be a very large number of deaths. More likely, the number is much higher and is now a few thousand. This is the main point at issue here. The deaths must be weighed against a benefit that many observers, including references we cited in (1), regard as completely unproven and that even Dr. Bracken admitted (1) is no more than modest.

The subject of this article is that we cannot decide risks versus benefits because Dr. Bracken continues to issue confusing, conflicting, partial reports. He rationalizes not releasing the data to independent observers, and he did not seek the approval by the US FDA that would have meant rigorous regulatory verification, a simpler way of gaining credibility and avoiding twenty years of controversy and disbelief.

Note: In 2008 Dr. Geisler estimated (1), "5000 extra patients may have died in the US since 1990." His estimates in this thread differ because more patients have now been at risk and because he has made a conservative correction for the fraction of patients on ventilator. He has given the details of his calculation, making it possible to critique and revise independently.

References:

1. Jeanne Lenzer and Shannon Brownlee. An untold story? BMJ 2008; 336: 532-534. Most references herein are to Rapid Responses to the 2008 article: to comments by William P. Coleman (7 March 2008), by Fred H. Geisler (12 March 2008), by Michael B Bracken (14 May 2008), by Ian Roberts (28 May 2008), and by Fred H. Geisler, William P. Coleman (10 June 2008)

(2) Bracken MB. CRASH (CORTICOSTEROID RANDOMIZATION AFTER SIGNIFICANT HEAD INJURY TRIAL): Landmark and storm warning. Neurosurgery 57:1300-1302, 2005

(3) CRASH trial collaborators. Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomized placebo-controlled trial. Lancet 2004; 364: 1321-1328