Re: Medical error—the third leading cause of death in the US
Makary and Daniel recently gained widespread publicity with their estimate that medical error constitutes the 3rd leading cause of death in the US.[1] As editors of BMJ Quality and Safety and researchers of long-standing in patient safety and quality of care, we appreciate the urge to draw attention to this area. But it is critical that the claims made to secure attention are well-founded. We worry that this estimate is not.
First, the estimate fails the plausibility test. Of around 2.5M deaths in the US each year, approximately 700,000 occur in hospital.[2] We – and many clinicians and researchers - find it very hard to believe that one in 10 of all US deaths, or a third of inpatient deaths (the 251,454 estimated by Makary and Daniel) result from “medical error”.
Second, the authors of the article do not provide any sort of formal methodology. Their estimate seems to rely on extrapolating preventable death rates from those reported in other studies. They then place the estimate derived from these heterogeneous studies in a “ranking” of causes of death in the US to make their argument that it is the third leading cause. These two steps are both precarious. The four studies on which they appear to base their estimate on use different methodologies and wildly varying definitions that Makary and Daniel collapse into their vividly-titled construct of “preventable lethal adverse event”. It is not clear how the “point estimates” they derive were calculated, but it is notable that the denominators across the studies are not comparable and no confidence intervals are reported.
The authors call for death certificates to include an extra field asking whether a preventable complication stemming from the patient’s medical care contributed to the death. The practical details of how this might be achieved are scant. Causes listed on death certificates already represent educated guesses much of the time, as not many patients die of diagnoses supported by gold standard tests during life (or autopsy results after death[3]) Moreover, the doctor who pronounces death (and thus fills out the death certificate) may be ill-placed to know whether the patient experienced a preventable complication in care. But, suppose we sidestepped these practical issues (and put aside questions of resources) to implement a system whereby at least two clinicians not based at the hospital where death occurred undertake an independent medical record review and then discuss the case in order to reach consensus about whether or not medical error had likely contributed to death.
As it turns out, this approach has been implemented in research settings on at least three occasions.[4-6] In all of these studies, the authors sampled deaths from multiple institutions and asked trained reviewers to look over the cases to identify possible quality of care problems and to make a judgment about the preventability of death. In all three studies, reviewers estimated that around 3% to 5% of deaths were ‘probably preventable’ (a greater than 50% chance that optimal care would have prevented death). The largest and most recent of these studies[5] reported that trained medical reviewers judged 3.6% of deaths to have at least a 50% probability of avoidability. Applying this rate of preventability to the total number of hospital deaths in the US each year produces an estimate of about 25,200 deaths annually that are potentially avoidable among hospitalized patients in the US—roughly 10-fold lower than the estimate advanced by Makary and Daniel.
The Makary and Daniel claim that medical error accounts for more than 250,000 deaths per year therefore stands in contrast to the results of several robust studies performed using the type of review they say is needed for medical error to be listed on a death certificate. Moreover, two of these studies [4, 6] make the point that roughly half of patients who probably had a preventable death were in their last six months of life – unlike the tens of thousands of people each year in the US who die in car accidents, shootings, or commit suicide (some of the causes of death that Makary and Daniel argue account for less mortality than does medical error).
A further problem with the estimate is more subtle. Making the field of patient safety all about death has risks. Just as most deaths do not involve medical error, most medical errors do not produce death—but they can still produce substantial morbidity, costs, suffering and distress. Drawing attention only to death as the focus of patient safety efforts risks drawing resources away from many settings of care – including almost all non-hospital environments – where death is not the most relevant outcome.
As people who care deeply about patient safety, we are troubled by at figures that produce lurid headlines but distract from areas where harm may be most amenable to interventions. As researchers, we fear for efforts to engage with clinicians when they are confronted by headline-grabbing numbers that fly in the face of their clinical experience. And, finally, as concerned citizens, we would rather not have medical care characterized as more dangerous than firearms or motor vehicles.
On one point, we agree with Marky and Daniel: they say that “sound scientific methods, beginning with an assessment of the problem, are critical to approaching any health threat to patients.” Sadly, their paper does not exemplify such sound science.
1 Makary MA, Daniel M. Medical error-the third leading cause of death in the US, BMJ 2016;353:i2139.
2 Hall MJ, Levant S, DeFrances CJ. Trends in inpatient hospital deaths: National Hospital Discharge Survey, 2000-2010, NCHS Data Brief 2013;(118):1-8.
3 Shojania KG, Burton EC. The vanishing nonforensic autopsy, N Engl J Med 2008;358:873-5.
4 Hayward RA, Hofer TP. Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer, JAMA 2001;286:415-20.
5 Hogan H, Zipfel R, Neuburger J, et al. Avoidability of hospital deaths and association with hospital-wide mortality ratios: retrospective case record review and regression analysis, BMJ 2015;351:h3239.
6 Hogan H, Healey F, Neale G, et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study, BMJ Qual Saf 2012;21:737-45.
Competing interests:
No competing interests
02 June 2016
Kaveh G. Shojania
Physician-researcher, journal editor
Mary Dixon-Woods (RAND Professor of Health Services Research, Cambridge Centre for Health Services Research)
Rapid Response:
Re: Medical error—the third leading cause of death in the US
Makary and Daniel recently gained widespread publicity with their estimate that medical error constitutes the 3rd leading cause of death in the US.[1] As editors of BMJ Quality and Safety and researchers of long-standing in patient safety and quality of care, we appreciate the urge to draw attention to this area. But it is critical that the claims made to secure attention are well-founded. We worry that this estimate is not.
First, the estimate fails the plausibility test. Of around 2.5M deaths in the US each year, approximately 700,000 occur in hospital.[2] We – and many clinicians and researchers - find it very hard to believe that one in 10 of all US deaths, or a third of inpatient deaths (the 251,454 estimated by Makary and Daniel) result from “medical error”.
Second, the authors of the article do not provide any sort of formal methodology. Their estimate seems to rely on extrapolating preventable death rates from those reported in other studies. They then place the estimate derived from these heterogeneous studies in a “ranking” of causes of death in the US to make their argument that it is the third leading cause. These two steps are both precarious. The four studies on which they appear to base their estimate on use different methodologies and wildly varying definitions that Makary and Daniel collapse into their vividly-titled construct of “preventable lethal adverse event”. It is not clear how the “point estimates” they derive were calculated, but it is notable that the denominators across the studies are not comparable and no confidence intervals are reported.
The authors call for death certificates to include an extra field asking whether a preventable complication stemming from the patient’s medical care contributed to the death. The practical details of how this might be achieved are scant. Causes listed on death certificates already represent educated guesses much of the time, as not many patients die of diagnoses supported by gold standard tests during life (or autopsy results after death[3]) Moreover, the doctor who pronounces death (and thus fills out the death certificate) may be ill-placed to know whether the patient experienced a preventable complication in care. But, suppose we sidestepped these practical issues (and put aside questions of resources) to implement a system whereby at least two clinicians not based at the hospital where death occurred undertake an independent medical record review and then discuss the case in order to reach consensus about whether or not medical error had likely contributed to death.
As it turns out, this approach has been implemented in research settings on at least three occasions.[4-6] In all of these studies, the authors sampled deaths from multiple institutions and asked trained reviewers to look over the cases to identify possible quality of care problems and to make a judgment about the preventability of death. In all three studies, reviewers estimated that around 3% to 5% of deaths were ‘probably preventable’ (a greater than 50% chance that optimal care would have prevented death). The largest and most recent of these studies[5] reported that trained medical reviewers judged 3.6% of deaths to have at least a 50% probability of avoidability. Applying this rate of preventability to the total number of hospital deaths in the US each year produces an estimate of about 25,200 deaths annually that are potentially avoidable among hospitalized patients in the US—roughly 10-fold lower than the estimate advanced by Makary and Daniel.
The Makary and Daniel claim that medical error accounts for more than 250,000 deaths per year therefore stands in contrast to the results of several robust studies performed using the type of review they say is needed for medical error to be listed on a death certificate. Moreover, two of these studies [4, 6] make the point that roughly half of patients who probably had a preventable death were in their last six months of life – unlike the tens of thousands of people each year in the US who die in car accidents, shootings, or commit suicide (some of the causes of death that Makary and Daniel argue account for less mortality than does medical error).
A further problem with the estimate is more subtle. Making the field of patient safety all about death has risks. Just as most deaths do not involve medical error, most medical errors do not produce death—but they can still produce substantial morbidity, costs, suffering and distress. Drawing attention only to death as the focus of patient safety efforts risks drawing resources away from many settings of care – including almost all non-hospital environments – where death is not the most relevant outcome.
As people who care deeply about patient safety, we are troubled by at figures that produce lurid headlines but distract from areas where harm may be most amenable to interventions. As researchers, we fear for efforts to engage with clinicians when they are confronted by headline-grabbing numbers that fly in the face of their clinical experience. And, finally, as concerned citizens, we would rather not have medical care characterized as more dangerous than firearms or motor vehicles.
On one point, we agree with Marky and Daniel: they say that “sound scientific methods, beginning with an assessment of the problem, are critical to approaching any health threat to patients.” Sadly, their paper does not exemplify such sound science.
1 Makary MA, Daniel M. Medical error-the third leading cause of death in the US, BMJ 2016;353:i2139.
2 Hall MJ, Levant S, DeFrances CJ. Trends in inpatient hospital deaths: National Hospital Discharge Survey, 2000-2010, NCHS Data Brief 2013;(118):1-8.
3 Shojania KG, Burton EC. The vanishing nonforensic autopsy, N Engl J Med 2008;358:873-5.
4 Hayward RA, Hofer TP. Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer, JAMA 2001;286:415-20.
5 Hogan H, Zipfel R, Neuburger J, et al. Avoidability of hospital deaths and association with hospital-wide mortality ratios: retrospective case record review and regression analysis, BMJ 2015;351:h3239.
6 Hogan H, Healey F, Neale G, et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study, BMJ Qual Saf 2012;21:737-45.
Competing interests: No competing interests