Measuring performance and missing the point?BMJ 2007; 335 doi: https://doi.org/10.1136/bmj.39377.387373.AD (Published 22 November 2007) Cite this as: BMJ 2007;335:1075
- Iona Heath, general practitioner1,
- Julia Hippisley-Cox, professor of clinical epidemiology and general practice2,
- Liam Smeeth, professor of clinical epidemiology3
- 1Caversham Group Practice, London NW5 2UP
- 2Nottingham University, Nottingham NG7 2RD
- 3London School of Hygiene and Tropical Medicine, London
- Correspondence to: I Heath
Since April 2004, the performance of general practitioners has been measured and remunerated against a limited, although increasing, number of easily measurable clinical activities. This initiative is unique worldwide and is attracting considerable international interest. It therefore behoves us to think critically about what we are doing.
General practitioners have responded by systematically recording the listed activities, apparently performing well above expectation, and the whole exercise has been hailed as a success for quality of care. Despite evidence that these sorts of incentives improve the quality of documentation while having a much more limited effect on underlying standards of care,1 there have undoubtedly been useful achievements. Of these, probably the most substantial are improvements in diabetic control and innovations in computer prompting systems. However, the system is in danger of missing the point of both quality and general practice.
Treatment by numbers
The clinical activities that are measured and rewarded by the quality and outcomes framework are largely evidence based. Nevertheless, almost all interventions cause some harm, and even when effective treatments are applied to a series of patients in clinical practice some will be harmed (although more will benefit). The risks of harm tend to increase with age, as does the potential for benefit. The stakes therefore become higher as the evidence becomes more tenuous because many trials focus on younger patients.
Evidence based care was never meant to be a substitute for clinical judgment but, combined with the inducements of the quality and outcomes framework, it becomes so. Mechanistic blanket management strategies, embedded into computer software, become fixed and static with the danger that innovation will be stifled. Interventions become routine, and practitioners are no longer required to grapple with the innate uncertainty of each different clinical situation. Most randomised trials systematically exclude patients' symptoms, functional status, comorbidity, severity of illness, ideas, and preferences. Yet these are the factors which should fundamentally affect decisions about appropriate treatment.2 Within large study populations, there will be smaller populations sharing different characteristics whose response to a given treatment will differ from that of the larger group. Such groups could be systematically harmed by the intervention, and there are currently no robust systems in place to measure or monitor this.3
The quality and outcomes framework diminishes the responsibility of doctors to think, to the potential detriment of patients, and encourages a focus on points scored, threshold met, and income generated. To give just one example, the failure to make any allowance for age means that doctors are encouraged to overtreat hypertension in old people4 with the danger of causing fainting, falls, and fractures.
The whole initiative is based on reductive linear reasoning that views the body as a machine and assumes that a standardised treatment will produce an equally standard unit of beneficial outcome. However, any practising clinician knows that the same treatment applied to two people with the same diagnosis can produce very different outcomes. Complexity theory suggests that the body is more usefully regarded as a complex adaptive system, characterised by rich interactions between multiple components that produce unpredictable outcomes. This analogy makes much more sense of clinical experience. Psychological states and social contexts exert measurable effects on the functioning of the body. Standardised treatments ignore all of this.
Best way to reach goals?
The quality and outcomes framework necessarily concentrates on clinical activities that are easily measured. Clearly, one of the easiest is the issuing of prescriptions. This leads to a situation where epidemiological research, with its tendency to extrapolate from short term studies over the whole of the remaining life span, combined with computer driven surveillance of whole populations, is driving hugely increased prescribing of some drugs. Almost all of this is designed to prevent future events rather than alleviate present suffering. Prevention has its merits, but was this an intention of the framework and what are the opportunity costs for other healthcare interventions?
One of the aims of the framework is to tackle health inequalities, but it has the potential to work in the opposite direction. Most fundamentally, it encourages the illusion that health inequalities can be solved by the health service and allows policy makers to ignore the extent to which health inequalities are a symptom of socioeconomic inequalities that continue to widen. The framework situates health inequalities at the level of the individual and ignores the political, social, and cultural context within which people find themselves. It is based on the astonishing assumption that everyone wants to live as long as possible whereas, in reality, some people seek to end their lives prematurely. Others adopt more chronically self harming behaviours in the full awareness that they are likely to shorten their lives, clearly prioritising coping in the immediate future above the uncertainties of the long term.
As there is a socioeconomic gradient for the prevalence of almost every disease, the poor are much more likely to experience comorbidity. The most marginalised people, who have a combination of physical and mental illnesses often compounded by drug or alcohol dependence, are difficult to engage in health care and even more difficult to coerce into the framework's unitary care pathways. Patients living in adverse social circumstances are also more likely to be taking maximal tolerated therapy without achieving the desired outcome. Not surprisingly, such people are much more likely to be reported as exceptions and, once given that status, are at risk of receiving proportionately less attention.5
Finally, as people living in deprived areas are sicker, more therapeutic effort will be needed for doctors to reach fixed targets. This means that doctors working in these areas have to work harder to achieve the same remuneration.6 The increased payments for higher disease prevalence do not take severity and complexity into account. Working in poorer areas becomes less desirable, further reducing quality of care or even making care difficult to find (figure⇓). Furthermore, there is evidence that payment for performance systems reward already high achievers and penalise low achievers and so exacerbate inequalities.7
What's not measured
Three quarters of the population do not have any of the diseases included in the quality and outcomes framework,8 and ever since its introduction, special interest groups have been lining up to have their particular priority included among the targets. However, worthy attempts to include depression have only made matters worse as the imposition of standardised questionnaires and scoring systems serves simply to reify and medicalise distress and unhappiness. Authentic dialogue between doctor and patient is disrupted and many doctors feel fundamentally compromised.9
None of the framework measures estimate clinically important outcomes. What they assess is treatment processes that are supposed to lead to improved outcomes. A marked discrepancy exists between the likely effect on health and the level of monetary reward, and there seems to have been no attempt to align the two.10 Until the undoubted and now well documented increase in process is translated into tangible outcomes such as diabetes complication rates, renal failure in hypertension, or incidence of myocardial infarction or smoking related deaths, the benefits and cost effectiveness of the exercise cannot be estimated. Outcomes are much more difficult to measure than processes, especially at the level of individual practices, but the heightened emphasis on process brought about by the framework should not be allowed to distract from the fundamental aims of medical care.
Measurable differences do not necessarily translate into meaningful differences in patients' lives
Clinical care needs to be tailored to individual patients rather than using a mechanistic approach
The quality and outcomes framework needs to include clinically important outcome measures
It should also include mechanisms to measure and monitor potential harms
This is the fourth article in a series looking at use of performance indicators in the UK and elsewhere.
This series is edited by Azeem Majeed, professor of primary care, Imperial College London (email@example.com), and Helen Lester, professor of primary care, University of Manchester (firstname.lastname@example.org).
Competing interests: All the authors work as NHS general practitioners and are therefore subject to the quality and outcomes framework. JH-C has been commissioned by the Department of Health, National Audit Office, and Disability Rights Commission and the Information Centre to do analysis to inform the development of the quality and outcomes framework. This article reflects her views rather than the views of the commissioning bodies.
Contributors and sources: LS is supported by a Wellcome Trust senior fellowship in clinical science. The core ideas in this article arose from discussions between all three authors. IH wrote the first draft and all authors contributed to serial drafts and agreed the final submission. IH is guarantor.
Provenance and peer review: Commissioned; externally peer reviewed.