Intended for healthcare professionals

Editorials

Measuring the quality of healthcare systems using composites

BMJ 2008; 337 doi: https://doi.org/10.1136/bmj.a639 (Published 14 August 2008) Cite this as: BMJ 2008;337:a639
  1. Bruce Guthrie, professor of primary care medicine
  1. 1University of Dundee, Community Health Sciences, Dundee DD2 4BF
  1. b.guthrie{at}chs.dundee.ac.uk

    Is useful for communication, but lacks the detail to improve care locally

    Composite measures are increasingly being used to assess the quality of health care. Recent examples include English star ratings, the quality and outcomes framework points score in the United Kingdom, and national and state scorecards in the United States.1 2 3 4 Composites summarise data from many quality indicators in one more easily understood number or rating.

    This is particularly important for patients and the public.5 Ease of interpretation is also important for policymakers, who need broad overviews of a system’s performance.1 Composites may therefore improve communication with the public (although the evidence that the public uses such data is weak, irrespective of how it is presented), increase accountability to payers, and help ensure that quality of care remains a priority in policy.

    In the linked study (doi: 10.1136/bmj.a957), Steel and colleagues report findings from a rigorously designed and implemented representative survey of older people in England, and they present various composites based on 32 clinical and seven “patient centred” quality indicators.6 They build on work done in the US to develop indicators of care for older people,7 and they use composites similar to those reported in an influential study of the quality of US health care.3

    The composite chosen is the overall percentage of recommended care received, defined as “all the care received by the population” divided by “all the care recommended for the population.” Overall, 62% of the care recommended for older adults was actually received. The authors rightly conclude that although the quality of health care has generally improved in many areas, much remains to be done. However, the study also highlights some of the key uncertainties and problems relating to the use of composites.

    Firstly, any composite is only as good as its underlying measures. The measures used here were modified from the US based Assessing Care of Vulnerable Elders (ACOVE) programme,8 but several of them—such as offering antihypertensive drugs to all patients with stroke, rather than treating blood pressure to target—differ greatly from UK practice. Additionally, the measures used are all derived from a patient survey, but clinical quality is usually better measured by directly examining clinical records.3 One consequence of using survey methods is that the processes measured are often not that tightly linked to patient outcomes, because of difficulties in accurately measuring intermediate clinical outcomes. Although excellent process is a crucial first step, measuring glycated haemoglobin or offering blood pressure treatment by itself makes no difference to patient outcomes unless the control of intermediate outcomes improves.9

    Related to this, truly summarising “quality” requires the underlying measures to provide comprehensive coverage of all dimensions of quality. Similar work in the US used several hundred measures rather than the 39 used here.3 7 The “patient centred care” domain in this study is particularly narrow—it focuses almost entirely on information and support for self management of selected conditions.

    Finally, as the authors mention, all composites are bedevilled by questions of weighting. In the composite used here every opportunity to deliver recommended care is weighted equally. This means that rare conditions do not contribute much to the overall score. Hence, monitoring of the international normalised ratio for people on warfarin forms 0.2% of the total score, whereas explaining the meaning of high cholesterol to patients forms 8.7%.6

    All weighting systems are arbitrary, and equal weighting is transparent even if it is indefensible at the margins. However, with equal weighting, quality measured by a composite could seem high even if rare but critical processes were lethally unsafe. It is questionable whether such different kinds of measure should be included in a single composite. Given the increasing use of composites, research is needed to define which indicators can legitimately be made into composites, to develop and test clear rationales for weighting schemes, and to build understanding of how changing weights alters conclusions about performance.1

    However, debate about the important technicalities of composite measures should not overshadow Steel and colleagues’ conclusion that the quality of health care for older adults has important deficiencies, even given the relative narrowness of the measures used. This applies particularly to “geriatric” conditions that cause high morbidity, like deafness and osteoarthritis, and that are currently excluded from routine measurement and incentivisation. This finding reinforces evidence that performance management of particular measures risks creating tunnel vision and crowding out improvement work for other care.10 11 12

    Steel and colleagues’ use of composites makes this message clearer for the public and policymakers, but composites are less practically useful for turning findings into interventions. From a clinician’s and a manager’s perspective the devil is always in the detail when it comes to improving quality. The challenge this study poses is to make the shift from identifying the problem using a national composite, to local measurement of the problem, and finally to local intervention to improve care.

    Notes

    Cite this as: BMJ 2008;337:a639

    Footnotes

    References

    View Abstract