Research versus the patientBMJ 2013; 346 doi: http://dx.doi.org/10.1136/bmj.f332 (Published 21 January 2013) Cite this as: BMJ 2013;346:f332
“If it were not for the great variability among individuals, medicine might be a science and not an art.” Sir William Osler, 1892
Tension between the science and art of healthcare has never been greater. On the one hand is the certainty of P values generated by well designed double blinded randomized controlled trials. Means and medians from these studies are used as the foundation for treatment guidelines and insurance coverage.
But physicians know from experience that patients seldom comport with the statistics of pristine trials. Patients are messy, they come with comorbidities, differing biologies, their own desires and preferences that seldom match up exactly with what is seen on the clean pages of the scientific literature.
“There is probably no issue more exciting than this scientific notion that treatments—be they pharmaceutical treatments or system level interventions—may have differential effects depending on who the patient is,” says Joe V Selby. He is executive director of the Patient-Centered Outcomes Research Institute (PCORI), the independent organization established under the Affordable Care Act (ACA).
It might be frustrating that answers for individual patients are so slow in coming, but Selby points to advances that have been made over the past decades using averages. He adds, “Treatment heterogeneity is something real that we can study and act on.”
“As you increase the number of subgroups that you are able to include in your analysis, you zero in on the individual,” according to Selby.
“There has to be an important role for observational studies in an exploratory analysis looking for treatment heterogeneity,” he says. “If there is a lot of it, and it is common across a number of diseases, we are generally not going to find it in the kind of clinical trials we can afford to put together. So, observational studies of very large databases should be one of our approaches to looking for heterogeneity.”
“The average patient isn’t the individual patient; that is the thing we are going to struggle with over and over again,” says Bobby W Dubois, chief science officer of the National Pharmaceutical Council.
“A treatment that we think is safe and effective for some people, doesn’t mean it is safe and effective for everybody. And maybe even more important is the corollary: A treatment that is not safe and not effective for some therefore is placed in a different part of the formulary and is harder to get to, or maybe it is not even on the formulary” and unavailable for use.
“Vioxx [rofecoxib] was taken off the market because it caused harm in some people,” says Dubois. “It may turn out there is a subgroup of folks that the alternative drugs, like Celebrex [celecoxib] and others, doesn’t work for them, and in fact Vioxx would be the proper solution for them.”
Considerable practical barriers loom in dealing with heterogeneity of response and subgroup analysis, says David M Kent, director of the Clinical and Translational Science MS/PhD Program at Tufts University. Post hoc analysis might be useful for generating hypotheses but by itself seldom provides definitive answers to research questions.
Meaningful subgroup analysis must be prospectively built into the study design and be adequately powered to answer those questions. The size of a study has to expand, he says, “You need, not just more patients, but orders of magnitude more patients.” Often recruitment must be targeted and oversampled to accrue adequate subgroup representation.
“That is not just an economic and practical problem for trialists, it’s also an ethical problem. Once you find the main effect, it is very difficult to continue to randomize patients in your trial because you have disturbed clinical equipoise.” Kent says this is a principal reason why it is rare to find clinical guidelines “that are driven by subgroup analysis.”
He illustrated this with the GUSTO study, which found that, in the general population, the clot busting drug streptokinase carried a higher overall risk of mortality than tissue plasminogen activator (tPA). However, “There is a subgroup that is identifiable on the basis of pre-treatment therapy” where the overall risk-benefit ratio favored use of streptokinase.”
But because the focus of the study was on the average response, which was negative, Kent said, use of the drug declined so greatly that “streptokinase is no longer available in the US.”
“We should pay some degree of homage to the average”—evidence based medicine is a statistical tool that shows a signal in the noise—but those measures also “distort the real world in a predictable way,” Kent said. “It has given us under-fit models for individual patients. The challenge is to make the models better.”
Crossover studies are the only way to find out which treatment is better for a patient, but they are seldom done, says Dubois. “For hypertension you can do that; for chemotherapy you can’t, you only get one whack at it.”
He points to the crossover CATIE trial of antipsychotic drugs sponsored by the National Institutes of Health as demonstrating that “the averages are very important, but there are people who differ from the average, not just by a little bit but by a fundamental amount that would change what you would do” for them.
Patients want to understand how the information in a study applies to them; what are the tradeoffs of potential risks and benefits that they are likely to experience, says Myrl Weinberg, president of the National Health Council, a coalition of chronic disease groups.
They have become wary of studies that focus on the disease, not the patient, she says. Those questions might be of scientific interest to researchers or of economic interest to providers and payers, but studies seldom evaluate questions of function and quality of daily life that often are the real metrics that patients use to make decisions.
Patients also are concerned with how data might be applied. They fear that study averages might be used to create barriers to obtain the “treatment that works for them.” Weinberg says, “This fear is especially real for people with a high degree of variability in response to treatment, such as mental health, neurologic, autoimmune, and rare conditions.”
Lost in translation
The hundreds of studies a year that PCORI will soon begin to generate “will give us answers at the patient level and the population level,” says Dubois, but there is no guarantee that they will be correctly applied to clinical practice.
The mainstream media will write about some of these studies, “But they are not going to be able to get into the nuances of heterogeneity,” he argues. And the medical trade press is little better; “They don’t immediately say individual patients may vary, there is no caveat in this stuff.”
Physicians acknowledge that heterogeneity can be a reason for varying patient responses to treatment says University of Arizona College of Pharmacy researcher Daniel C Malone. But in surveys and focus groups he has conducted, “They also stated more frequently, it’s a lack of adherence or compliance that led to treatment failure.” They prefer to blame the patient rather than question the relevance of a study to that individual patient.
Malone’s research on arthritis drugs documented that heterogeneity is seldom discussed in journal papers. “Prior to 2000, package inserts generally had no measure of variance around the mean. Newer package inserts tend to have much more information about variance than older package inserts,” but that means “it is difficult for physicians to make decisions when they have no data.”
“The poor doctor, who is faced with all of these results, is going to have a bit of a reductionist view,” says Dubois. It is difficult to keep up with all of the studies being published, let alone “which were hypothesis confirming and which were hypothesis generating, that I really shouldn’t take to the bank. It is very important and complicated.”
Money and ethics
“What do you do when patients respond differently and you have copay structures? You try the less expensive one and it doesn’t work for you,” Dubois explained. “By dint of biology or something you may have no control over, you may not respond to the lower tier, less expensive drug, and now may need to go on to the more intensive therapies. You have to pay more.”
He extended the example to two antidepressants; both have roughly the same efficacy at the population level but the less expensive one causes side effects like weight gain or sexual dysfunction. Is it ethical for the insurer to force the patient to pay more in order to choose “the side effect profile that may be more in keeping with what is important for that patient?”
“How do we begin to dialog around the ethical question of ‘is that appropriate?’” Dubois asks.
Patient advocates see differing coverages and copayments as a false dilemma. Forcing people to pay more “inherently isn’t right,” says Weinberg. “The idea of ‘let’s try the cheaper in two-step therapy’—for some people that is very harmful.”
“Misapplication of CER [comparative effectiveness research] can lead to a public/private coverage structure in which only those drugs and devices that serve the most patients at the least cost are accessible,” she says. “This would create significant health issues for those patients whose individual physiology and unique lifestyle needs don’t fit that average mold.”
“It is absolutely immoral to have patients, particularly cancer patients, fail multiple treatments before they get on the right treatment,” adds Ellen V Sigal, chair of Friends of Cancer Research. “By the time they get the right treatment they are likely to be dead or not respond.”
Bob Griss, executive director of the Institute of Social Medicine and Community Health, believes it might even be a violation of the Americans with Disabilities Act to charge patients more if they require a different therapeutic approach.
A good case can be made that physicians and payers should listen to patients’ concerns, argues Weinberg. When quality of life issues are addressed, patients become more engaged in their own care and “are much more likely to be adherent; people have better health outcomes and lower healthcare costs.”
Dubois believes the crucial factor in deciding what intervention to try first is the cost of being wrong. Patients can tolerate trial and error with something like an antihypertensive drug, “but where you have got a bad infection, or rapidly deteriorating MS [multiple sclerosis], or if you believe that early treatment of RA [rheumatoid arthritis] will prevent joint damage from ever occurring, you’ve got one chance.”
“If the cost of being wrong is very high [policy makers and health plans] might give more freedom to the doctor and the patient. Where the costs are lower, you have a chance to try multiple things” perhaps more discipline might be imposed.
Kent suggests that wherever possible the decision on use “should be moved down to the level where a well informed doctor and a well informed patient can decide.”
A more immediate concern for Dubois is that initial guidelines for “essential benefits,” which health plans must cover under the Affordable Care Act, only have to offer one drug per class.
Some states in establishing health insurance exchanges and expanding Medicaid coverage are setting more generous requirements for coverage, but many states are refusing to set up those programs, “which means the federal government will establish a program. Unless they change that federal structure, it will be one drug per class.”
The path to one
PCORI has begun to implement its mandate for patient centered outcomes research with the goal of getting to one, the individual patient. Selby is optimistic that many of the answers will be found in the research they are supporting, which focuses on large, longitudinal databases of typical patients that capture real work experiences and preferences.
“Averages aren’t always bad,” says Dan Leonard, president of the National Pharmaceutical Council. “It takes drilling into those averages on the fringes to get the answers for the individuals.”
The mean, the average is just a number; it is not some panacea. It might be a starting point for treating the individual patient, but it should not be a straightjacket that restricts the practice of medicine.
Cite this as: BMJ 2013;346:f332
Competing interests: None declared.
Provenance and peer review: Commissioned; externally peer reviewed.
Acknowledgement: Much of the information in this report was gathered at the conference The Myth of Averages: Why Individual Patient Differences Matter sponsored by the National Pharmaceutical Council and held in Washington, DC in late November 2012. Short video interviews with some of the speakers can be found here http://www.npcnow.org/Public/Newsroom/Video___Podcasts/Public/Newsroom/Video___Podcasts/Video___Podcasts_Main_Page.aspx?hkey=bb8c00ce-e651-4d6e-a628-04215ea8f638.