Measuring quality of care with routine data: avoiding confusion between performance indicators and health outcomes

BMJ 1999;319:94-98 ( 10 July )

General Practice

Measuring quality of care with routine data: avoiding confusion between performance indicators and health outcomes

Editorial by Jankowski

Antonio Giuffrida, research fellow, ^a Hugh Gravelle, professor of economics, ^a Martin Roland, professor of general practice. ^b

^a National Primary Care Research and Development Centre, Centre for Health Economics, University of York, Heslington, York YO10 5DD, ^b National Primary Care Research and Development Centre, Williamson Building, University of Manchester, Oxford Road, Manchester M13 9PL

Correspondence to: M Roland m.roland{at}man.ac.uk

Objective: To investigate the impact of factors outsidethe control of primary care on performance indicators proposedas measures of the quality of primarycare.
Design: Multiple regression analysis relating admissionrates standardised for age and sex for asthma, diabetes, and epilepsyto socioeconomic population characteristics and to the supplyof secondary careresources.
Setting: 90 family health services authorities in England,1989-90 to 1994-5.
Results: At health authority level socioeconomic characteristics,health status, and secondary care supply factors explained 45%of the variation in admission rates for asthma, 33% for diabetes,and 55% for epilepsy. When health authorities were ranked, onlyfour of the 10 with the highest age-sex standardised admissionrates for asthma in 1994-5 remained in the top 10 when allowancewas made for socioeconomic characteristics, health status, andsecondary care supply factors. There was also substantial yearto year variation in therates.
Conclusion: Health outcomes should relate to crude ratesof adverse events in the population. These give the best indicationof the size of a health problem. Performance indicators, however,should relate to those aspects of care which can be altered bythe staff whose performance is beingmeasured.

Key messages

The NHS executive has proposed that admission rates for asthma, diabetes, and epilepsy could be used at health authority level as indicators of the quality of primary care
There is considerable year to year variation in the ranking of health authorities by admission rates for these conditions, even when rates are aggregated. This makes it hard to interpret a single year's data: a 3 year average would be more reliable
Morbidity, socioeconomic characteristics, and secondary care supply are important confounding factors that explain between a third and a half of the variation in admission rates across health authority areas
Performance indicators should relate to aspects of care that can be controlled by decision makers. Confounding factors have a clear impact on admission rates and must therefore be taken into account if such rates are to be used as performance indicators

© BMJ 1999

CiteULike

Complore

Connotea

Del.icio.us Add to Digg

Digg

StumbleUpon Add to Technorati

Technorati What's this?

Relevant Articles

The future of the quality and outcomes framework: Helen Lester and Azeem Majeed
BMJ 2008 337: a3017. [Extract] [Full Text]

Patient factors largely explain variation in GP admission rates
BMJ 1999 319: 0. [Full Text]

What do hospital admission rates say about primary care?: Raymond Jankowski
BMJ 1999 319: 67-68. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

Lester, H., Majeed, A. (2008). The future of the quality and outcomes framework. BMJ 337: a3017-a3017 [Full text]
Groenewoud, MSc, A. S., van Exel, MSc, N. J. A., Berg, PhD, M., Huijsman, PhD, R. (2008). Building Quality Report Cards for Geriatric Care in The Netherlands: Using Concept Mapping to Identify the Appropriate "Building Blocks" From the Consumer's Perspective. Gerontologist 48: 79-92 [Abstract] [Full text]
van der Ent, C. K. (2008). Quality assessment: is the truth in the outcome?. Eur Respir J 31: 6-7 [Full text]
Marshall, M., Klazinga, N., Leatherman, S., Hardy, C., Bergmann, E., Pisco, L., Mattke, S., Mainz, J. (2006). OECD Health Care Quality Indicator Project. The expert panel on primary care prevention and health promotion. Int J Qual Health Care 18: 21-25 [Abstract] [Full text]
Nabitz, U., van Den Brink, W., Jansen, P. (2005). Using Concept Mapping to design an indicator framework for addiction treatment centres. Int J Qual Health Care 17: 193-201 [Abstract] [Full text]
Roland, M., Dusheiko, M., Gravelle, H., Parker, S. (2005). Follow up of people aged 65 and over with a history of emergency admissions: analysis of routine admission data. BMJ 330: 289-292 [Abstract] [Full text]
Love, T., Dowell, A. C, Salmond, C., Crampton, P. (2004). Quality indicators and variation in primary care: modelling GP referral patterns. Fam Pract 21: 160-165 [Abstract] [Full text]
Spies, T. H., Mokkink, H. G. A., De Vries Robbe, P. F., Grol, R. P. T. M. (2004). Which data source in clinical performance assessment? A pilot study comparing self-recording with patient records and observation. Int J Qual Health Care 16: 65-72 [Abstract] [Full text]
Arah, O. A., Klazinga, N. S., Delnoij, D. M. J., Asbroek, A. H. A. T., Custers, T. (2003). Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. Int J Qual Health Care 15: 377-398 [Abstract] [Full text]
Goldfield, N., Gnani, S., Majeed, A. (2003). Primary care in the United States: Profiling performance in primary care in the United States. BMJ 326: 744-747 [Full text]
Powell, A E, Davies, H T O, Thomson, R G (2003). Using routine comparative data to assess the quality of health care: understanding and avoiding common pitfalls. Qual Saf Health Care 12: 122-128 [Abstract] [Full text]
Hansell, A., Hollowell, J., McNiece, R., Nichols, T., Strachan, D. (2003). Validity and interpretation of mortality, health service and survey data on COPD and asthma in England. Eur Respir J 21: 279-286 [Abstract] [Full text]
Hippisley-Cox, J., Pringle, M., Coupland, C., Hammersley, V., Wilson, A. (2001). Do single handed practices offer poorer care? Cross sectional survey of processes and outcomes. BMJ 323: 320-323 [Abstract] [Full text]
O'Donnell, C. A (2000). Variation in GP referral rates: what can we learn from the literature?. Fam Pract 17: 462-471 [Abstract] [Full text]
McColl, A., Roland, M. (2000). Clinical governance in primary care: Knowledge and information for clinical governance. BMJ 321: 871-874 [Full text]
Kamps, G., Stewart, R., van der Werf, G., Schuling, J, Jong, B M.-d. (2000). Adherence to the guidelines of a regional formulary. Fam Pract 17: 254-260 [Abstract] [Full text]
Jankowski, R. (1999). What do hospital admission rates say about primary care?. BMJ 319: 67-68 [Full text]