Persistent health disparities in the US signal need for new thinking
BMJ 2012; 345 doi: https://doi.org/10.1136/bmj.e6204 (Published 13 September 2012) Cite this as: BMJ 2012;345:e6204
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
The article by Epstein[1] reporting on the recent Institute of Medicine Workshop on health disparities [2] suggests that after more than a decade of government reports and other initiatives, there needs to be new thinking about how to deal with health disparities in the United States. But such thinking can be of little value if not informed by an understanding of how badly flawed research into health disparities has been as a result of the failure to recognize the way standard measures of differences between outcome rates tend to affected by the prevalence of an outcome.
For reasons related to the shapes of distributions of factors associated with experiencing an outcome, the rarer the outcome the greater tends to be the relative difference in experiencing it and the smaller tends to be the relative difference in avoiding it. Thus, for example, as mortality declines, relative difference in mortality tend to increase while relative differences in survival tend to decrease; as rates of appropriate healthcare increase, relative differences in receipt of appropriate care tend to decrease, while relative differences in rates of failing to receive appropriate care tend to increase. As prevalence of a health or healthcare outcome changes, absolute differences tend to change in the same direction as the smaller relative difference. And, irrespective of the shapes of the underlying distributions, when an absolute difference and a relative difference have changed in opposite directions (as will frequently be noted), the unmentioned relative difference will have changed in the opposite direction of the mentioned relative difference and in the same direction as the absolute difference.[3-6]
In 2005, the National Center for Health Statistics (NCHS) recognized the pattern by which relative differences in favorable outcomes and relative differences in adverse outcomes tend to change in opposite directions as the prevalence of an outcome changes.[7] But, rather than address the implications of such pattern with respect to the utility of either relative difference to track disparities over time without taking changes in prevalence into account, NCHS merely recommended that henceforth disparities in all health and healthcare outcomes should be measured in terms of relative differences in adverse outcomes. According to this approach, which underlies Health and Human Services’ appraisals of achievement of the health disparities reduction goals in Healthy People 2020, healthcare disparities that previously were regarded as decreasing now would be deemed to be increasing. See Table 3 of ref. 5. And where researchers relying on relative differences in favorable healthcare outcome would find dramatic decreases in disparities, NCHS would find dramatic increases in disparities. See Table 4 of ref. 5.
Meanwhile, in the National Healthcare Disparities Reports, the Agency for Healthcare Research and Quality (AHRQ) measures disparities in terms of whichever relative difference (in the favorable of the adverse outcome) is larger, without a suggestion in any of the reports of an understanding that these two measures tend to change in opposites directions.[8] The scope of the failure to understand the pattern of relative differences is perhaps best reflected in the disparities studies, particularly those addressing cancer outcomes, where observers refer to disparities in survival and disparities in mortality interchangeably, sometimes purporting to analyze one while in fact analyzing the other, and without recognizing that the two change in opposite directions as survival changes over time.[9]
Increasingly, research into healthcare disparities, often funded by AHRQ, relies on absolute differences between rates, without any recognition of the way absolute differences tend to change solely because of changes in prevalence of an outcome or that reliance on absolute differences tends to yield opposite conclusions as to changes over time from those AHRQ would reach. The failure to understand the patterns by which absolute differences between rates are affected by the prevalence of an outcome has even caused Massachusetts to include a disparities measure in its Medicaid pay-for-performance program that is more likely to increase disparities than reduce them. See pages 21-24 of ref. 6.
One of the most recent guides to disparities measurement, a joint product of Harvard Medical School and Massachusetts General Hospital,[10] is substantially superior to earlier guides in that it at least discusses the importance of reporting absolute differences and both relative differences and points out that the two relative differences can yield different conclusions as to changes over time. But, because it fails to reflect any recognition of the patterns by which the measures it discusses tend to be affected by overall prevalence, or the implications of such patterns with respect to efforts to determine whether the forces causing a disparity have increased or decreased, this guide will lead to further flawed disparities research. See pages 42-43 of ref.6
The Institute of Medicine report fails to reflect any understanding of the above measurement issues. While some of its discussion may be of value, the failure to recognize those issues materially undermines its overall utility.
References:
1. Epstein K. Persistent health disparities in the US signal for new thinking. BMJ 2012;345:e6204 doi: 10.1136/bmj.e620
2. Institute of Medicine of the National Academies. How far have we come in reducing health
disparities? Progress since 2000. www.iom.edu/Reports/2012/How-Far-Have-We-Comein-
Reducing-Health-Disparities.aspx.
3. Scanlan JP. Can we actually measure health disparities? Chance 2006:19(2):47-51. http://www.jpscanlan.com/images/Can_We_Actually_Measure_Health_Dispariti...
4. Scanlan JP. Race and mortality. Society 2000;37(2):19-35. http://www.jpscanlan.com/images/Race_and_Mortality.pdf
5. Scanlan JP. The Mismeasure of Group Differences in the Law and the Social and Medical Sciences. Applied Statistics Workshop at the Institute for Quantitative Social Science at Harvard University, Oct. 17, 2012. http://jpscanlan.com/images/Harvard_Applied_Statistic_Workshop.ppt
6. Scanlan JP. Harvard University Measurement Letter. Oct. 9, 2012. http://jpscanlan.com/images/Harvard_University_Measurement_Letter.pdf
7. Keppel K., Pamuk E., Lynch J., et al. (2005) Methodological Issues in Measuring Health Disparities. Vital Health Stat;2 (141): http://www.cdc.gov/nchs/data/series/sr_02/sr02_141.pdf
8. Scanlan JP. Measurement Problems in the National Healthcare Disparities Report, presented at American Public Health Association 135th Annual Meeting & Exposition, Washington, DC, Nov. 3-7, 2007: PowerPoint Presentation: http://www.jpscanlan.com/images/ORAL_ANNOTATED.pdf
9. Mortality and Survival Page of jpscanlan.com: http://jpscanlan.com/mortalityandsurvival2.html
10. Weinstein JS, Betancourt JR, Green AR, et al.. Commissioned Paper: Measuring Healthcare Disparities, Disparities Solution Center, Oct. 4, 2011: http://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemI...
Competing interests: No competing interests
Re: Persistent health disparities in the US signal need for new thinking
This corrects and updates my February 2013 comment [1] on an article by Epstein regarding the need for new thinking about health disparities in the US.[2] In the comment, I criticized US health and healthcare disparities research for its failure to recognize patterns by which standard measures of differences between outcome rates tend to be systematically affected by the prevalence of an outcome. The patterns include those whereby as the prevalence of an outcomes changes (a) relative (percentage) differences in favorable outcomes and relative differences in the corresponding adverse outcomes tend to change in opposite directions and (b) absolute (percentage point) differences tend to change in the same direction as the smaller relative difference. And I stated that in the yearly National Healthcare Disparities Report (NHDR) the US Agency for Healthcare Research and Quality (AHRQ) measured health and healthcare disparities in terms of whichever of the two relative differences is larger and therefore tended to reach opposite conclusions about directions of changes in disparities from those reached by AHRQ-funded researchers relying on absolute differences.
The statement that AHRQ relied on the larger of the two relative differences in the NHDR was based on statements in NHDRs from 2005 until 2009 that a disparity would be deemed important where “the relative difference is at least 10% different from the reference group when framed positively as a favorable outcome or negatively as an adverse outcome.” [3 (at 28 n.9)]. But I have since recognized that that approach was limited to determinations of the importance of a disparity. For the purpose of determining whether disparities have increased or decreased, it has been AHRQ’s intention to measure all disparities in terms of relative differences in adverse outcomes, in accord with the approach of the US National Center for Health Statistics (NCHS). AHRQ has not, however, invariably implemented that approach. As shown in Table 5 of reference 4(a), among cases that the 2012 NHDR highlights as the fastest decreasing disparities in healthcare outcomes are situations where the relative difference in the adverse outcome in fact increased (though the relative difference in the favorable outcome and the absolute difference decreased). The failure of the agency to correctly implement its intended measurement approach is less serious an issue, however, than its failure yet to understand that the measures it employs or intends to employ tend to be systematically affected by changes in the prevalence of the outcome examined, and, hence, why those measures cannot effectively indicate whether the strength of the forces causing outcome rates of disadvantaged and advantaged groups to differ has increased or decreased over time.
The earlier comment did not discuss the US Centers for Disease Control and Prevention (CDC), which had issued its own extensive 2011 Health Disparities and Inequalities Report.[5] Last December the agency issued a similarly extensive 2013 Health Disparities and Inequalities Report.[7] Like the earlier document and other disparities research of CDC, the 2013 report shows no awareness that the measures on which it relies may be affected by the prevalence of an outcome, or even that NCHS (an arm of CDC) has specifically found that determinations of directions of changes in disparities will commonly turn on whether one examines relative differences in favorable outcomes or relative differences in adverse outcomes. See pages 26 to 32 of reference 4(a) regarding the disarray of health and healthcare disparities research among the most prominent US governmental and nongovernmental institutions involved in such research (or guidance on such research) and the failure of those institutions yet to provide sound research, or sound guidance on research, regarding whether health and healthcare disparities should be deemed to have increased or decreased over time. That failure will continue until those institutions recognize and responsibly address the implications of the fact that standard measures tend to change simply because the prevalence of an outcome changes.
As reflected in a more recent comment [7] on a Marmot and Goldblatt’s editorial emphasizing the importance of monitoring health inequalities, such monitoring is no sounder in the UK than in the US.
References
1. Scanlan JP. The need for new thinking about how to measure disparities. BMJ 4 Feb 2013. http://www.BMJ.com/content/345/BMJ.e6204/rr/628910
2. Epstein K. Persistent health disparities in the US signal for new thinking. BMJ 2012;345:e6204 doi: 10.1136/BMJ.e620). http://www.bmj.com/content/345/bmj.e6204
3. Agency for Healthcare Research and Quality, 2009 National Healthcare Disparities Report. http://www.ahrq.gov/research/findings/nhqrdr/nhdr09/index.html
4. Measuring Health and Healthcare Disparities. Proceedings of the Federal Committee on Statistical Methodology 2013 Research Conference (March, 2014)
(a) Presentation: http://jpscanlan.com/images/2013_FCSM_Presentation_pdf_.pdf
(b) Paper: http://jpscanlan.com/images/2013_Fed_Comm_on_Stat_Meth_paper.pdf
5. Centers for Disease Control and Prevention. 2011. CDC Health Disparities and Inequalities Report – United States, 2011: http://www.cdc.gov/mmwr/pdf/other/su6001.pdf
6. Centers for Disease Control and Prevention. 2013. CDC Health Disparities and Inequalities Report – United States, 2013. http://www.cdc.gov/minorityhealth/CHDIReport.html#CHDIR
7. Scanlan JP. The monitoring of health inequalities has never been sound. BMJ 9 Nov 2012). http://www.bmj.com/content/347/bmj.f6576/rr/671152
8. Marmot M, Goldblatt P. Importance of monitoring health inequalities. BMJ 2013;347:f6576. http://www.bmj.com/content/347/bmj.f6576
Competing interests: No competing interests