Intended for healthcare professionals

Editorials

The wider role of regulatory scientists

BMJ 2017; 357 doi: https://doi.org/10.1136/bmj.j1991 (Published 27 April 2017) Cite this as: BMJ 2017;357:j1991
  1. Peter Doshi, associate editor, The BMJ,
  2. Fiona Godlee, editor in chief, The BMJ
  1. Correspondence to: P Doshi pdoshi{at}bmj.com

Why they must help us improve the evidence base

In 1990, a research scientist at the US Food and Drug Administration wrote in a letter to the New England Journal of Medicine (NEJM), “I review many clinical-trial protocols. The results of some of these trials subsequently appear in the medical literature. On occasion, the published description of the study may differ from the prospective protocol in important aspects of study design or statistical analysis—eg, study size, clinical end-points, and statistical tests used. The potential for misuse … is tremendous.”1

These comments were eerily prophetic: subsequent years brought the scandals of rofecoxib,23 celecoxib,4 paroxetine,56 and oseltamivir.7 The comments also suggest that each of these scandals could have largely been averted if journals had taken better account of what regulators—particularly the FDA—had to say about the relevant trials. We agree. Long before NEJM issued an expression of concern about data on harms missing from its report of the VIGOR trial of rofecoxib, those same data were available on the FDA’s website.2JAMA reported CLASS as a six month trial that showed celecoxib had fewer gastrointestinal side effects than non-steroidal anti-inflammatory drugs,8 but the FDA’s website told a different story.9 CLASS was actually two trials of twice that duration. At 12 months the purported safety benefits disappeared.4

The FDA releases specialist reviews on its website after a drug is approved.1011121314 These lengthy reports can and do detect errors that elude journal peer review. But few people know they are available or have the time to read them. The result may be misinformation, financial waste, and patient harm.

We believe that regulatory scientists should engage directly with journals to highlight problematic publications and help improve the medical literature. Historically, they have been reluctant to do this—we know of only three occasions when regulatory scientists have intervened. In 1980, FDA authors penned a lengthy “critique” of a trial published in the NEJM.15 In the second case, 31 years later, FDA authors included one sentence in a long article noting that published articles contained “conclusions ... not supported by the data.”16 And last year, an internal memo from the agency’s commissioner called for “formal correction or retraction” of a sponsor’s article it said had been “fundamentally debunked.”1718

Meanwhile crucial insights go unnoticed. The journal publication of Study 329 reported that paroxetine was safe and effective in adolescents with depression, but an FDA medical officer later concluded that Study 329 was a “failed study.”19 However, no letter was sent to the journal putting this information into the mainstream medical literature. Only more than a decade later, when others came to reanalyse the trial data, was the record finally corrected.5

Some will argue that it’s not a regulator’s job to “police” the literature, and that data submitted by sponsors are commercially confidential. But the alternative is to accept a world in which information known to be wrong goes uncorrected. Helping ensure the accuracy of the medical literature seems an obvious part of regulators’ wider public health mission, and echoes calls for greater engagement.20

FDA scientists contend they are unable to discuss what they know.21 But at various times the FDA has considered opening up. In 2009, it launched an agency-wide “Transparency Initiative,” which, among other things, aimed to strengthen its ability to “contribute to the scientific discussion” in response to companies that publish “an incomplete picture about the safety or efficacy of a product.”22 This initiative seems to have fizzled out.232425 But former principal deputy commissioner Joshua Sharfstein recently reiterated the call for the FDA to “correct misleading information in the market.”26

Anything that helps improve the validity of the medical literature is good for health—and could also save taxpayers’ money. By 2009, the US government had stockpiled over $1bn worth of oseltamivir with an explicit assumption that the drug reduced complications, something a manufacturer authored analysis had suggested.27282930 But nine years earlier, the FDA—following its review of the data—had warned the manufacturer that such a claim was “not supported by substantial evidence.”31 Had the FDA engaged directly with the literature, the world might have made better decisions about how to spend public money.

For any regulators reading this, here’s what such engagement might look like. Firstly, you could tell us what the problems are and where to look. You could write letters for publication or (if this is problematic) write privately to editors so they could act on your tip-offs, demand data from authors and their institutions, or issue corrections, expressions of concern, or retractions.

Secondly, when you write your reports recommending approval or rejection of a drug, you could think of your potential readers as also including journal editors, clinicians, and patients. Your reports are sometimes the only way in which “commercially confidential” trial data can get into the public domain.

The task is enormous. The evidence base is mired in a legacy of misleading reports of clinical trials published in medical journals. So where to begin? We suggest starting with drugs approved in the past three years. Many were launched with high profile journal publications. Helping to ensure these publications accurately reflect the true risks and benefits of the drug can prevent years of misinformed decision making. Regulatory scientists are uniquely poised to help.

Footnotes

  • We thank current and former FDA scientists for helpful comments.

  • Competing interests: See www.bmj.com/about-bmj/editorial-staff.

  • Provenance and peer review: Commissioned; not externally reviewed.

References