Future of quality measurementBMJ 2007; 335 doi: https://doi.org/10.1136/bmj.39385.406192.AD (Published 29 November 2007) Cite this as: BMJ 2007;335:1130
- Helen Lester, professor of primary care,
- Martin Roland, director
- Correspondence to: H Lester
In the past decade there has been sustained international interest in measuring quality of care. In the United Kingdom, quality indicators with financial incentives to reward good care were introduced as a result of increasing awareness of variable quality in primary care, the technical feasibility of introducing evidence based indicators within information technology systems, and a resolve by political negotiators to use improved quality to secure additional investment in primary care.1 Similar but less comprehensive initiatives have been introduced in the United States, Europe, Australia, and New Zealand. However, as this series has shown, the use of quality measures has also created controversy. Our view is that using incentives to improve quality of care has been beneficial. We look at what needs to be done to ensure those benefits remain in the future.
Options for developing quality measures
The quality and outcomes framework, which forms the basis of quality measurement in UK primary care, could be developed in several different ways:
Leave indicators unchanged and expect higher achievement each year—This means restricting the potential benefits of quality measures to a limited number of areas2
Add new indicators or conditions regularly—This could lead to a vast and unmanageable set of measures
Build a larger set of evidence based measures that are all monitored and pay for performance against a subset of these
Remove measures once a predetermined and agreed level of achievement has been reached—Although this would allow new measures to be introduced without making the scheme unmanageable, it would require robust information about the effect of removing measures on performance in terms of both patient care and practice income and agreement over reintroduction of measures if performance worsens
Rotate measures regularly, enabling a potential improvement across a range of conditions and areas—This would be our preferred option, although it would need to be carefully piloted to look for and guard against unintended consequences to patient care or practice morale.
The quality measures introduced into general practice in 2004 were mainly drawn from existing national guidelines. They reflected widely accepted standards of clinical care, and there was little direct criticism of the indicators themselves. The first major revision, in 2006, included evidence based indicators that changed clinical practice in ways that were unfamiliar to many general practitioners—for example, encouraging use of validated structured questionnaires as part of the assessment of patients with depression and more active management of chronic kidney disease. Indicators that seek to extend existing practice will always be more controversial than those reinforcing established practice. In our view, indicators that aim to change standard practice should be particularly carefully evaluated, both before and during their introduction. Piloting of new measures for at least 12 months would highlight any professional concerns, education and training needs, and information technology problems.
Interpersonal aspects of care
A common criticism of quality measures is that they look at only limited areas of clinical practice and ignore, and hence may devalue, some core aspects of general practice. These aspects include care for people with multiple complex problems, care for people for whom continuity of care makes a real difference to satisfaction, and the quality of interpersonal care itself. Current UK quality measures may discourage continuity of care, for example by fragmenting care between doctors and nurses, and there is certainly an argument that something in the organisation and financing of general practice should encourage continuity of care. Indeed, there is urgency here, since it may not be long before the trainers within primary care are doctors who have grown up in a climate that prizes the easily measurable and financially rewarded above the less measurable and definable aspects of care.
We also need to place greater value on patients' views of care when measuring the quality of interpersonal care. However, as Elwyn and colleagues highlighted, it is less clear whether doctors should be paid according to the results of patient questionnaires.3 Such an approach would cause major problems in practices where patient turnover was high or where patients were unfamiliar with the health system. Research is needed to determine the effect of paying practices against patient evaluation scores and to develop more innovative and meaningful ways of involving patients from different social and ethnic backgrounds in their health care.
Effect of measurement on health inequality
Quality improvement measures targeted at high risk patients should, in theory, reduce health inequalities. When cervical cytology and immunisation targets were introduced in 1990, practices in affluent areas rapidly reached near maximum performance but deprived areas caught up during the next few years. This led to a substantial overall reduction in inequality,4 5 an example of the inverse equity hypothesis.6
In the quality and outcomes framework, affluent areas achieved higher scores and reported more exceptions than poorer areas, but the differences were small.7 Overall, the financial incentives seem to have reached areas of high need relatively effectively for most targets. The effect of incentive structures needs to be constantly reviewed to ensure that they deliver health benefits across all communities. An important subsidiary message is the need to take a long term view when interpreting the effects of quality measures on health inequalities.
Learning from beyond medicine
Measuring aspects of quality is now part of 21st century life, and we may be able to learn from other disciplines. Should we, for example, be looking at the business sector's 15 years of experience from implementing standards such as Investors in People?8 This voluntary standard has been achieved by over 32 000 organisations employing over 27% of the UK workforce. However, when the standard was introduced, its effect was less than predicted since many firms used it to gain recognition for existing good practice.9 This is perhaps analogous to the many practices in 2004-5 that achieved their quality and outcomes framework targets through accurate recording of existing practice. Would this knowledge have influenced Department of Health predictions about achievement levels in primary care and helped shape the associated financial payments? Such commonalities highlight the importance of looking beyond the immediate and obvious comparators if we want to broaden our understanding of the potential and problems of introducing and developing quality measures.
Financial incentives are, of course, not the only way of improving the quality of care. In the five years before the quality and outcomes framework was introduced, major improvements occurred in the quality of management of chronic disease in general practice.10 We need to continue to use a mix of professional, financial, and managerial approaches and experiment to find the mix that gives the NHS best value for money and patients the best care.
Improvements will not be maintained if quality measures remain static
Measures could be rotated, enabling potential improvement across a range of conditions and areas
Indicators that aim to change standard practice should be especially carefully evaluated and piloted
The effect of quality measures needs to be constantly reviewed to ensure that they deliver health benefits across all communities
This is the last article in a series looking at use of performance indicators in the UK and elsewhere.
This series is edited by Azeem Majeed, professor of primary care, Imperial College London(email@example.com)and Helen Lester,professor of primary care, University of Manchester(firstname.lastname@example.org).
Contributors and sources: HL has written about pay for performance and has a longstanding research interest in health quality and inequalities. MR is a general practitioner whose research over the past 10 years includes developing ways of measuring and improving quality of care.
Competing interests: MR provided academic advice to the BMA and employers' negotiating teams on the development of the quality and outcome framework in 2001 and 2002. HL provides this advice to the current negotiating teams.
Provenance and peer review: Commissioned; externally peer reviewed.