A population health approach to reducing observational intensity bias in health risk adjustment: cross sectional analysis of insurance claimsBMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g2392 (Published 10 April 2014) Cite this as: BMJ 2014;348:g2392
- David E Wennberg, associate professor of medicine1,
- Sandra M Sharp, research analyst1,
- Gwyn Bevan, professor of policy analysis2,
- Jonathan S Skinner, James O Freedman professor of economics134,
- Daniel J Gottlieb, research analyst1,
- John E Wennberg, Peggy Y Thomson professor emeritus in the evaluative clinical sciences1
- 1The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, 35 Centerra Parkway, Lebanon, NH 03766, USA
- 2Department of Management, London School of Economics and Political Science, London, UK
- 3Department of Economics, Dartmouth College, Hanover, NH, USA
- 4National Bureau of Economic Research, Hanover, NH, USA
- Correspondence to: D E Wennberg
- Accepted 19 March 2014
Objective To compare the performance of two new approaches to risk adjustment that are free of the influence of observational intensity with methods that depend on diagnoses listed in administrative databases.
Setting Administrative data from the US Medicare program for services provided in 2007 among 306 US hospital referral regions.
Design Cross sectional analysis.
Participants 20% sample of fee for service Medicare beneficiaries residing in one of 306 hospital referral regions in the United States in 2007 (n=5 153 877).
Main outcome measures The effect of health risk adjustment on age, sex, and race adjusted mortality and spending rates among hospital referral regions using four indices: the standard Centers for Medicare and Medicaid Services—Hierarchical Condition Categories (HCC) index used by the US Medicare program (calculated from diagnoses listed in Medicare’s administrative database); a visit corrected HCC index (to reduce the effects of observational intensity on frequency of diagnoses); a poverty index (based on US census); and a population health index (calculated using data on incidence of hip fractures and strokes, and responses from a population based annual survey of health from the Centers for Disease Control and Prevention).
Results Estimated variation in age, sex, and race adjusted mortality rates across hospital referral regions was reduced using the indices based on population health, poverty, and visit corrected HCC, but increased using the standard HCC index. Most of the residual variation in age, sex, and race adjusted mortality was explained (in terms of weighted R2) by the population health index: R2=0.65. The other indices explained less: R2=0.20 for the visit corrected HCC index; 0.19 for the poverty index, and 0.02 for the standard HCC index. The residual variation in age, sex, race, and price adjusted spending per capita across the 306 hospital referral regions explained by the indices (in terms of weighted R2) were 0.50 for the standard HCC index, 0.21 for the population health index, 0.12 for the poverty index, and 0.07 for the visit corrected HCC index, implying that only a modest amount of the variation in spending can be explained by factors most closely related to mortality. Further, once the HCC index is visit corrected it accounts for almost none of the residual variation in age, sex, and race adjusted spending.
Conclusion Health risk adjustment using either the poverty index or the population health index performed substantially better in terms of explaining actual mortality than the indices that relied on diagnoses from administrative databases; the population health index explained the majority of residual variation in age, sex, and race adjusted mortality. Owing to the influence of observational intensity on diagnoses from administrative databases, the standard HCC index over-adjusts for regional differences in spending. Research to improve health risk adjustment methods should focus on developing measures of risk that do not depend on observation influenced diagnoses recorded in administrative databases.
We thank Anne Carney for her help with editing the manuscript before final submission.
Contributors: DEW and JEW conceived the work, oversaw statistical analysis, and drafted and revised the manuscript. SMS was lead research associate, oversaw cleaning, management, and analysis of data, and assisted with presentation of results in the manuscript. GB provided input into design and drafting and revisions of the manuscript. DJB acquired and cleaned key datasets and was central to several of the analytic methods. JSS provided input into methodology and statistical analyses, and participated in manuscript revisions. All authors gave approval of final version and agree to be accountable for all aspects of the work. DEW is guarantor.
Funding: This study was partially supported by the National Institute on Aging (grant PO1-AG19783) and the Robert Wood Johnson Foundation. The funders had no role in the design and conduct of the study; the collection, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on No commercial request from the corresponding author) and declare: support from the organisations described below for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work. GB is a member of the Department of Health’s Advisory Committee on Resource Allocation and its Technical Advisory Group, but has contributed to the argument of this paper in a personal capacity.
Ethical approval: Not required.
Data sharing: No additional data available.
Transparency: The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.