Defining antibiotic effectiveness and resistance: how a private party may soon rule judgments over susceptibility testingBMJ 2016; 352 doi: https://doi.org/10.1136/bmj.h6849 (Published 06 January 2016) Cite this as: BMJ 2016;352:h6849
- Peter Doshi, associate editor, The BMJ
When treating infections physicians routinely rely on laboratory tests to guide the selection of an effective antibiotic. These reports indicate which drugs the patient’s bacteria are susceptible to and those to which they are resistant, with implications for morbidity and survival.
Traditionally, the Food and Drug Administration has set the thresholds at which antibiotics are deemed effective based on information submitted by the manufacturer. But the US Congress is considering legislation to shift the decision making to private organizations that use other methods. This opens the potential for vested interests to adjust the laboratory thresholds (“breakpoints”) used to distinguish between sensitive and resistant for established drugs and inappropriately influence physicians to select new antibiotics over old ones without compelling evidence of superiority on patient centered outcomes. These new powers could also ultimately mislead the public and governments about the threat of resistant organisms.
Susceptibility testing explained
Bacterial isolates are generally tested for susceptibility to antibiotics using an in vitro test. One method places the specimen from the patient in multiple tubes or plates of growth media and exposes each to increasing concentrations of a given drug. The lowest concentration of drug that inhibits bacterial growth is deemed the isolate’s minimum inhibitory concentration (MIC). When the MIC falls below a predetermined breakpoint established for that organism and drug combination, the isolate is deemed susceptible to the drug. A second predetermined breakpoint differentiates between results interpreted as “intermediate” or “resistant.” The same process is repeated for each antibiotic, and a final laboratory report generally lists all relevant antibiotics to which the organism is susceptible, intermediate, and resistant.
“It’s so fundamental,” says Lisa Plymate, an internist and member of a new task force fighting the proposed law. As a clinician “you turn in your bacterial specimen to be cultured, you get back a report, and it tells you what drugs your bacteria is sensitive to and resistant to. You have faith in that. You pick your antibiotics based on that.”
“When I heard that this could be manipulated and that this did not have to be based on hard science, but drug companies could come in and change the [breakpoints] to make the old antibiotics look like they didn’t work … to think that somebody was going to use my lab data and influence my practice, it was a real ‘oh my god’ moment. That’s when I knew FDA was under a lot of pressure and we decided to form a task force.”
That task force, the National Physicians Alliance FDA Task Force, is battling legislative efforts that would take control of determining breakpoints out of the FDA’s hands. All clinical microbiology laboratories and device manufacturers which have historically followed FDA’s guidance would be affected.
The move even seems to have the support of the White House. In March, the Obama administration released an action plan to counter antimicrobial resistance. It directs the FDA to, by 2020, update breakpoints more efficiently—for example, “by adopting criteria developed by SDOs [standards development organizations].”
Draft legislation known as the 21st Century Cures Act, which passed the House of Representatives earlier this year, further details how the outsourcing could occur.1 The bill calls on the FDA to establish and maintain a website recording changes in breakpoints determined by standard development organizations. The move would break from current practice in which updating breakpoints, which are recorded on drug labels, requires a product sponsor to submit data and revised labeling for FDA review.
It would also cast the process of revising breakpoints as part of the everyday business of good antibiotic stewardship, something it arguably is not. Historically, the FDA set breakpoints at the time of drug approval and they were rarely changed, even when organisms developed resistance and their MIC rose above the threshold. But by the mid-2000s, FDA breakpoints were increasingly described as tentative and in need of periodic updating. In 2007, Congress instructed the FDA to update breakpoints periodically.2
In an interview, the FDA described the current approach as challenging and slow. The proposed 21st Century Cures legislation, the FDA asserted, “could help leverage the work that is already being done by standards development organizations to ensure more timely updating of breakpoints.” The agency maintained that it would “retain final decision authority over breakpoints.”
One potential beneficiary of the draft legislation is the Clinical and Laboratory Standards Institute (CLSI). The institute describes itself as a leader in clinical laboratory testing standards and boasts more than 1800 “active volunteers creating our standards.”
Its chief executive officer, Glen Fine, however, played down the importance of the draft legislation to his organization. The FDA, he told The BMJ in an interview, is “in no way or form … outsourcing the interpretive criteria [breakpoints] . . . Sure, it would be great if the legislation passes. The profile of CLSI is raised. But regardless whether relevant breakpoint language is in or out, we’re going to do what we do.” He stressed that it was his organization’s consensus approach that had brought the government to CLSI’s door, and not vice versa.
But it is CLSI’s extensive use of people employed by or in financial relationships with industry that has alarmed some critics, raising questions about conflict of interest and the organization’s ability to produce objective, unbiased standards. The health advocacy group Public Citizen, for example, has cited the fact that drug industry employees are on the committee that sets and revises breakpoints.
“Companies that produce and market antibiotics should not be involved in setting or adjusting MIC [minimum inhibitory concentration] breakpoints,” Plymate told an FDA advisory committee in 2013.3 She pointed to CLSI’s microbiology committee, which includes employees of drug companies and people who declare financial relations with companies that market antibiotics. Its chair, for example, is John Rex, vice-president and head of infection global medicines development at AstraZeneca.
Fine characterizes concerns over conflicts of interest as misguided. “There are elements that say that if industry touches anything, it’s poison, and that’s unfortunate.” The institute specifically requires industry presence on its writing and consensus committees, he says, along with members from the government and healthcare professions. Fine emphasizes that “meetings are open,” and last year CLSI changed its policy so that “pharmaceutical employees are no longer voting members on final decisions to publish AST [antimicrobial susceptibility testing] subcommittee standards including breakpoints.”
But a review of disclosures by The BMJ showed that all eight people on the committee that proposes breakpoints who have academic or hospital affiliations declared financial relations with the drug or device industry. Only three members declared no conflicts of interest: two from the US Centers for Disease Control and Prevention, and one from the UK Medicines and Healthcare Products Regulatory Agency. And of the committee’s 27 “advisors,” 19 declared financial interests or benefits.4
Test tube science
Since 2010, CLSI has issued around 30 new breakpoint determinations, including for some of the most successful antibiotics in history, and questions are emerging about the rigor of the institute’s methods.
Dan Sahm, a microbiologist and chief scientific officer of a company that carries out the tests used to determine breakpoints for drug companies, says that “breakpoints should not be altered unless there are significant clinical indications that there are safety and efficacy reasons for change.”
Rather than using clinical evidence of loss of efficacy to guide reconsideration of breakpoints, CLSI instead bases many decisions on pharmacokinetic and pharmacodynamic in vitro modeling studies with little to no clinical data.
John Powers, former lead medical officer for antibiotic resistance at FDA, added: “The process seems to be turned on its head now. We should start with patient outcomes then evaluate in vitro tests or modeling that correlate with those. If we ‘can’t get’ patient data how do we know there is a problem for patients?” Powers pointed to a recent government systematic review that reported a “near absence of strong evidence” for using pharmacokinetic or pharmacodynamic data to guide antibiotic treatment of hospital acquired pneumonia.5
“Nobody disagrees that that [the absence of clinical data] is the smoking gun,” said CLSI’s Fine.
But some researchers have retrospectively evaluated the effect of CLSI’s recommendations on patient outcomes. One study evaluated a 2010 CLSI recommendation for lowering the breakpoints for ceftriaxone against Enterobacteriaceae. The revised breakpoint tripled the number of children who would have been deemed to be resistant to ceftriaxone. Yet they concluded that children with supposedly resistant infections “appear to have similar clinical outcomes when prescribed ceftriaxone compared with broader spectrum β-lactam regimens.”6 The results suggested that CLSI got it wrong.
Lead author, Pranita Tamma, who subsequently joined CLSI’s committee for antimicrobial susceptibility testing, was sympathetic to the organization’s position but told The BMJ that pharmacokinetic and pharmacodynamic modeling studies should only be considered “hypothesis generating,” not confirmatory. “I think they want to use clinical data to inform their decisions, but unfortunately there’s not a lot of data out there.”
In a separate study, Tamma evaluated CLSI’s 2012 recommendation to lower the breakpoint for piperacillin against Pseudomonas aeruginosa.7 This time, her findings suggested CLSI was right to have re-evaluated a long established breakpoint.
But it remains unclear why CLSI is the right party to do the work, and why clinical data are following—rather than preceding—revised breakpoint recommendations.
Breakpoint judgments also carry important financial implications. The higher the breakpoint, the more people will be deemed to have “susceptible” infections. Conversely, lowering the breakpoint of an older drug that is off patent will encourage physicians to prescribe newer, more expensive or broader spectrum antibiotics, often with less evidence of safety and effectiveness. In addition, “you could have the perception that resistance rates are going up,” said Sahm. “So you better use the newer, brighter drug.”
Creating the problem they set out to solve?
All critics agree that the solution is simple: keep the power to set and revise breakpoints within the FDA.
“These guys [CLSI advisors] are all in business as my company is,” Sahm explained. “We provide consultative or laboratory services to generate data to help pharmaceutical companies develop all the data they need to generate an application to the FDA. However, if somebody in my company or another company is perceived as a person who can help you get the breakpoints you want, that’s a company you want to work with … anybody in my shoes who says they can be objective is fooling themselves.”
“The US FDA is the only group that has all of the capabilities and they don’t have any skin in the game.”
Congressional proponents of the 21st Century Cures Act legislation promote it as a solution to the pressing need for new treatments, including antibiotics. Yet the little known provision on outsourcing breakpoint determinations could end up creating the problem it purports to solve. As lowered breakpoints will paint new and more alarming snapshots of a world in which fewer old antibiotics are believed to work because of the natural evolution of antimicrobial resistance, perceptions of the urgency for new drugs grows. Changing breakpoints for older drugs denies patients antibiotics that are proved effective in the absence of evidence of worse clinical outcomes—a notion seemingly counter to the bill’s idea of bringing effective medications to patients.
A new approach
Some scientists view the debate over breakpoints as a symptom of a much larger problem in the field. “The practice of infectious diseases, everywhere from the physician to the infectious diseases doctor, is focused on killing the bug,” says Liise-anne Pirofski, a professor of medicine, microbiology, and immunology at the Albert Einstein College of Medicine in New York. Pirofski and her colleague Arturo Casadevall suggest that current inadequacies in antibiotic therapy often have little to do with resistance.
“You have a very good example in the fungal world,” Casadevall says. “You have examples of drugs that have great MIC and yet you cannot eradicate the infection. Many of these patients are treated for their life.
“The central assumption that is made is that what happens in vitro for an organism grown in culture will apply to the situation in vivo. There are enormous amounts of evidence now that metabolic organisms are different … that approach works very well with some antibiotics, but doesn’t work very well across the board.”
Pirofski and Casadevall contend that the focus on “bugs and drugs” is narrow and impeding the collective ability of scientists to develop better treatments because it overlooks the elephant in the room: the sick patient.8 They argue it is time to “ditch the term pathogen” and instead focus on the simultaneous analysis of microbe-host interactions. The MIC does not measure the human immune system and its ability (or not) to combat infection.
But they are not holding their breath. “The drug companies and the people developing drugs are loathe to do anything differently because the regulatory agencies use precedent,” Casadevall said.
Cite this as: BMJ 2016;352:h6849
Competing interests: I have read and understood BMJ policy on declaration of interests and declare I know and consider some of the people interviewed in this article as colleagues. I was a corecipient of a UK National Institute for Health Research grant (for the HTA update and amalgamation of two Cochrane reviews of neuraminidase inhibitors). In addition, I received support from the European Respiratory Society for travel to give a talk at the society’s 2012 annual congress in Vienna. I also received a 2015 new investigator award from the American Association of Colleges of Pharmacy to fund a PhD student to work on research on how the potential harms of statins are conveyed in drug labeling and pharmacy leaflets.
Provenance and peer review: Commissioned; externally peer reviewed.