Primary care

Use of risk adjustment in setting budgets and measuring performance in primary care II: advantages, disadvantages, and practicalities

Azeem Majeed, senior lecturer in general practice ^a, Andrew B Bindman, associate professor of medicine, epidemiology, and biostatistics ^b, Jonathan P Weiner, professor ^c. 

^a School of Public Policy, University College London, London WC1H 9EZ, ^b Departments of Medicine, Epidemiology, and Biostatistics, University of California, San Francisco CA 94110, USA, ^c Health Services Research and Development Center, Johns Hopkins School of Public Health, Baltimore MD 21025, USA

Correspondence to: A Majeed a.majeed{at}ucl.ac.uk

Risk adjustment could help to improve decisions about budgets as well as help measure the performance of doctors. In the first of these two articles we discussed why risk adjustment could become more important in the United Kingdom and how it works in the United States.¹ In this article we consider the benefits and problems of risk adjustment and assess how one US system would perform in the United Kingdom.

	What are the uses of risk adjustment?

Top What are the uses... What are the limitations... Could risk adjustment models... Application of risk adjustment... Conclusions References

In the United States, risk adjustment is starting to be used to adjust capitation or other types of payments to healthcare providers such as family practices, multispecialty medical groups, or consortiums of physicians and hospitals (integrated delivery systems).² For large populations (such as that of a broad geographical area), age, sex, and ecological measures may be adequate for this purpose. But for smaller populations, such as those managed by one family practice or a small consortium of physicians, risk adjustment helps ensure that providers who manage patients with more complex medical problems have their budgets adjusted to take this into account (box). The use of risk adjustment systems has also given doctors and health maintenance organisations a powerful incentive to provide more accurate and complete diagnostic data.¹

The second important use of risk adjustment is to adjust for case mix when comparing practice patterns across providers.³ For example, the NHS plan states that it "promises better performance and accountability systems to reduce variations in service across England."⁴ The performance indicators published by the Department of Health show wide variation in performance among doctors in both primary and secondary care.⁵ However, as they do not take into account differences in case mix, we do not know how valid such indicators are as measures of clinical efficiency and efficacy. Risk adjustment can help correct such variations for underlying differences in population case mix and thus could lead to fairer and more accurate performance measures for providers.

Another use of risk adjustment is to measure the health of a population.⁶ The traditional way of doing this has been to use death rates or self reported measures of chronic illness derived from censuses or surveys of the population. Because many important chronic illnesses do not result in death, mortality is not always a good measure of the health of a population. Similarly, a generic self reported measure of chronic illness may be an unreliable measure of health status. Risk adjustment models based on all conditions treated by primary and secondary care physicians can take into account the full spectrum of illnesses in a population. Hence, they include chronic illnesses such as arthritis or epilepsy, which cause considerable population morbidity but are often not recorded on death certificates, as well as illnesses such as ischaemic heart disease.

	What are the limitations of risk adjustment?

Top What are the uses... What are the limitations... Could risk adjustment models... Application of risk adjustment... Conclusions References

Risk adjustment methods are relatively new and have only recently started to be used to adjust capitation payments in the United States. Hence, we have limited knowledge about whether risk adjustment improves the equity of resource allocation or the efficiency of health services (box).⁷ When applied retrospectively, as they would be when used to compare primary care physicians, they explain 30-50% of the variation in physician performance. However, when the models are applied prospectively, as they would for capitated payments, they explain less than 10% of the variation in future healthcare costs after adjustment for age and sex.

Risk adjustment models are based on data derived from computerised patient records or from the billing records of insurance companies. They therefore suffer from problems such as incomplete or inaccurate coding of diagnostic data. The models, which have a bias to create estimates clustered around the mean, are also poor at dealing with "outlier" cases (patients with very high healthcare costs or with unexpected healthcare needs) such as patients who need organ transplantation, HIV positive patients who are prescribed expensive antiretroviral drugs, and patients who suffer serious injuries. Patients with such disorders require special funding arrangements and are usually excluded from budgets or performance reports when risk adjustment methods are applied to a general population. ^{8 9}

Another potential problem associated with risk adjustment is that of "upcoding." This arises when providers use diagnoses that result in their patients appearing to have more complicated illnesses than is really the case in order to benefit from additional resources or improve their ratings on case mix adjusted measures of performance. Furthermore, because patients in the United States frequently move between health maintenance organisations and health plans, the risk adjustment models are often based on data on use of health care for a limited period (one year in many cases).

Another caveat about risk adjustment is that it developed partly as a response to greater fragmentation and increased competition in the US healthcare industry. ^{10 11} Although risk adjustment may have a role in the United Kingdom, it is important not to repeat the mistakes made in the United States, such as the substantially greater spending on administration of health services. Hence, any introduction of risk adjustment in the United Kingdom should be driven by the need to improve the fairness of current methods of allocating resources and of improving equity within the NHS.

Despite these limitations, risk adjustment seems to do as well as other measures of health status in explaining variations in performance that may arise from differences in case mix. For example, in a study by Fowles and colleagues, one case mix adjustment method, aggregated diagnostic groups, predicted spending on health care better than information derived from the short form health status survey (SF-36), a commonly used and well validated instrument for measuring health status in individuals.¹²

	Could risk adjustment models be used in the United Kingdom?

Top What are the uses... What are the limitations... Could risk adjustment models... Application of risk adjustment... Conclusions References

The NHS in England is shifting from a health funding system based on allocating budgets to large (typically around 500 000 patients) and geographically well defined populations of health authorities to funding based on the much smaller and less well geographically defined populations of primary care trusts and general practices. Differences in the health characteristics of patients among these groups are likely to be greater than that seen among health authorities. Hence, methods of funding that take into account differences in case mix will increasingly be needed. Methods are also needed that are capable of producing measures of performance adjusted for case mix. This is particularly the case for general practices, where age, sex, and ecological measures are not sufficient to measure the clinical characteristics of their populations. This leads to, for example, large variations in prescribing rates and costs, and disputes between general practitioners and prescribing advisers about the fairness of prescribing budgets. The use of risk adjustment could help reduce such problems by providing better measures of the health status and clinical characteristics of practice populations.

US methods of population based diagnostic risk adjustment have never been applied in the United Kingdom. Hence, some research is needed to assess how well these methods can be adapted for use in the United Kingdom and how effective they are at explaining differences in use of health care or performance of providers.

One difficulty in applying risk adjustment methods in the United Kingdom is that we do not have computerised diagnostic information on the care provided to patients in all healthcare settings. Although data are available for hospital admissions, there is no information on outpatient consultations or from general practices that are not fully computerised. However, the NHS is committed to developing fully computerised medical records, integrated between primary, community, and secondary care, over the next few years.¹³

Finally, general practices in the United Kingdom generally use Read codes to classify clinical data whereas all the US risk adjustment programmes use International Classification of Diseases (ICD) codes. Hence, either the US risk adjustment programs would have to be adapted to use Read codes or the diagnostic codes in clinical systems in primary care will have to be converted to ICD codes.

	Application of risk adjustment to UK population

Top What are the uses... What are the limitations... Could risk adjustment models... Application of risk adjustment... Conclusions References

We investigated the feasibility of applying the Johns Hopkins adjusted clinical group case mix system in the United Kingdom using data from the fourth national morbidity survey.¹⁴ This was a prospective one year study of all consultations with general practitioners in 60 general practices in England and Wales. Diagnostic information on all consultations was entered directly on to computer by the participating general practitioners. The study population comprised a 1% sample of the population of England and Wales and was representative of the population for age, sex, social class, and housing tenure. A validation survey carried out at the end of the study confirmed the accuracy and completeness of the data collected. All diagnoses made during the survey were converted from Read to ICD-9 codes.

Of the patients in the survey, 72% (365 209) consulted at least once. In total, there were 1 273 051 separate ICD-9 diagnostic codes, of which just under 99% could be assigned an aggregated diagnostic group; 14 667 (1.15%) ICD-9 codes could not be assigned an aggregated diagnostic group, a similar percentage to that seen in North American studies. The mean number of aggregated diagnostic groups per patient was 2.0, and the mean number of major aggregated diagnostic groups per person was 0.2. In total, 17% of patients in the survey had at least one major aggregated diagnostic group. Patients with several aggregated diagnostic groups or one or more major diagnostic groups are likely to have an important influence on practice workload and use of health services.

The table shows the breakdown of the distribution of the total number of aggregated diagnostic groups and number of major aggregated diagnostic groups per person in the morbidity survey. Also shown are comparative data from enrolees of two large health insurance plans in the United States. One set of comparative data is from the billing database of a fee for service managed care population in a midwestern state. The second is from a commercial point of service health maintenance organisation in a northeastern state.

The distributions of aggregated diagnostic groups are similar for all three populations, although the US populations have a higher percentage of patients with 5 aggregated diagnostic groups and 3 major groups. This may reflect differences in medical practice between the two countries, information lost in the translation of Read to ICD-9 codes, or more complete recording of diagnostic data in the United States. The preliminary findings of our empirical analysis suggest that US derived risk adjustment methods may work reasonably well in the UK and that further research is warranted.

	Conclusions

Top What are the uses... What are the limitations... Could risk adjustment models... Application of risk adjustment... Conclusions References

The ultimate objective of any healthcare system is to improve the health of the public. An important task for the NHS is therefore to ensure that patients, particularly those with complex medical problems that are expensive to manage, gain access to the health services they require to help them lead longer and healthier lives. Risk adjustment models like those now being used in the United States offer the possibility of fairer methods of funding primary care trusts and general practices and fairer methods of comparing the performance of healthcare providers.

One advantage that the United Kingdom has over the United States in developing risk adjustment methods is that the UK healthcare system can provide information on patients' use of both primary and secondary care, and for a longer period, than most American health maintenance organisations or health plans can do. This may allow the United Kingdom to develop more valid methods of risk adjustment and to benefit from the substantial investment being made by the NHS in information and communication technology.

	Footnotes

   Competing interests: AM, ABB, and JPW are currently carrying out research on the feasibility of using the Johns Hopkins ACG system in the UK, supported by the Commonwealth Fund of New York. Software based on the ACG method is the copyright of the Johns Hopkins University. The university receives royalties for non-academic uses of this software. AM holds a primary care career scientist award and is funded by the NHS Research and Development Directorate.

	References

Top What are the uses... What are the limitations... Could risk adjustment models... Application of risk adjustment... Conclusions References

1.	Majeed A, Bindman AB, Weiner JP. Use of risk adjustment in setting budgets and measuring performance in primary care I: how it works. BMJ 2001; 323: 604-607[Free Full Text].
2.	Iezzoni LI, Ayanian JZ, Bates DW, Burstin HR. Paying more fairly for Medicare capitated care. N Engl J Med 1998; 339: 1933-1938[Free Full Text].
3.	Salem-Schatz S, Moore G, Rucker M, Pearson S. The case for case-mix adjustment in practice profiling: when good apples look bad. JAMA 1994; 272: 871-874[Abstract/Free Full Text].
4.	Department of Health. The NHS plan. London: DoH, 2000. (www.doh.gov.uk).
5.	NHS Executive. Quality and performance in the NHS: high level performance indicators. Leeds: NHS Executive, 1999.
6.	Reid R, MacWilliam L, Roos N, Bogdanovic B, Black C. Measuring morbidity in populations: performance of the Johns Hopkins adjusted clinical group (ACG) case-mix adjustment system in Manitoba. Manitoba: University of Manitoba, 1999. (www.umanitoba.ca/centres/mchpe/reports/pdfs/acg.pdf).
7.	Newhouse JP. Patients at risk: Health reform and risk adjustment. Health Affairs 1994; Spring: 132-146.
8.	Maguire AM, Poew NR, Starfield B, Andrews J, Weiner J, Anderson G. "Carving out" conditions from global capitation rates: protecting high-cost patients, physicians and health plans in a managed care environment. Am J Managed Care 1998; 4: 797-806[Medline].
9.	Andrews J, Anderson G, Han C, Neff J. Pediatric carveouts. The use of disease-specific conditions as risk adjusters in capitated payment systems. Arch Pediatr Adolesc Med 1997; 151: 236-242[Abstract/Free Full Text].
10.	Bodenheimer T. California's beleaguered physician groupswill they survive? N Engl J Med 2000; 342: 1064-1068[Free Full Text].
11.	Bodenheimer T. Selective chaos. Health Affairs 2000; 19: 200-205[Medline].
12.	Fowles JB, Weiner JP, Knutson D, Fowler E, Tucker AM, Ireland M. Taking health status into account when setting capitation rates. JAMA 1996; 276: 1316-1321[Abstract/Free Full Text].
13.	NHS Executive. Information for health. Leeds: NHS Executive, 1998. (www.nhsia.nhs.uk).
14.	McCormick A, Fleming D, Charlton J. Morbidity statistics from general practice. Fourth national study, 1991-1992. London: HMSO, 1995.

(Accepted 13 June 2001)