Observational intensity bias associated with illness adjustment: cross sectional analysis of insurance claimsBMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f549 (Published 21 February 2013) Cite this as: BMJ 2013;346:f549
- John E Wennberg, professor emeritus in the evaluative clinical sciences1,
- Douglas O Staiger, professor of economics2,
- Sandra M Sharp, research associate1,
- Daniel J Gottlieb, research associate1,
- Gwyn Bevan, professor of policy analysis, head of the department of management3,
- Klim McPherson, visiting professor of public health epidemiology and emeritus fellow4,
- H Gilbert Welch, professor of medicine, professor of The Dartmouth Institute, and professor of community and family medicine 1
- 1The Dartmouth Institute for Health Policy and Clinical Practice, The Audrey and Theodor Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA
- 2Department of Economics, Dartmouth College, Hanover, NH, USA
- 3London School of Economics and Political Science, London, UK
- 4Nuffield Department of Obstetrics and Gynaecology, New College, Oxford, UK
- Correspondence to:
- Accepted 9 January 2013
Objective To determine the bias associated with frequency of visits by physicians in adjusting for illness, using diagnoses recorded in administrative databases.
Setting Claims data from the US Medicare program for services provided in 2007 among 306 US hospital referral regions.
Design Cross sectional analysis.
Participants 20% sample of fee for service Medicare beneficiaries residing in the United States in 2007 (n=5 153 877).
Main outcome measures The effect of illness adjustment on regional mortality and spending rates using standard and visit corrected illness methods for adjustment. The standard method adjusts using comorbidity measures based on diagnoses listed in administrative databases; the modified method corrects these measures for the frequency of visits by physicians. Three conventions for measuring comorbidity are used: the Charlson comorbidity index, Iezzoni chronic conditions, and hierarchical condition categories risk scores.
Results The visit corrected Charlson comorbidity index explained more of the variation in age, sex, and race mortality across the 306 hospital referral regions than did the standard index (R2=0.21 v 0.11, P<0.001) and, compared with sex and race adjusted mortality, reduced regional variation, whereas adjustment using the standard Charlson comorbidity index increased it. Although visit corrected and age, sex, and race adjusted mortality rates were similar in hospital referral regions with the highest and lowest fifths of visits, adjustment using the standard index resulted in a rate that was 18% lower in the highest fifth (46.4 v 56.3 deaths per 1000, P<0.001). Age, sex, and race adjusted spending as well as visit corrected spending was more than 30% greater in the highest fifth of visits than in the lowest fifth, but only 12% greater after adjustment using the standard index. Similar results were obtained using the Iezzoni and the hierarchical condition categories conventions for measuring comorbidity.
Conclusion The rates of visits by physicians introduce substantial bias when regional mortality and spending rates are adjusted for illness using comorbidity measures based on the observed number of diagnoses recorded in Medicare’s administrative database. Adjusting without correction for regional variation in visit rates tends to make regions with high rates of visits seem to have lower mortality and lower costs, and vice versa. Visit corrected comorbidity measures better explain variation in age, sex, and race mortality than observed measures, and reduce observational intensity bias.
We thank Jon Deeks, Adam Steventon, and Wynand van de Ven for comments on earlier drafts of this manuscript.
Contributors: All authors jointly wrote the article and are guarantors.
Funding: This study was partially supported by the National Institute on Aging (grant PO1-AG19783) and the Robert Wood Johnson Foundation. The funders had no role in the design and conduct of the study; the collection, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: support from the organisation described below for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: No additional data available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.