Quality in primary care

The United Kingdom has led the world in several national policy initiatives to improve the quality of primary care, notably by implementing both nationwide surveys of patient experience and the largest healthcare pay for performance scheme in the world—the Quality and Outcomes Framework. This makes the United Kingdom fertile ground for examining the many uncertainties surrounding these policy initiatives, two of which are examined in the linked papers (doi:10.1136/bmj.g6034, doi:10.1136/bmj.g6423).1 2

Roberts and colleagues used multilevel modelling to explore whether variation in patient experience was due more to individual doctors or to the practices within which they worked.1 The doctors were responsible for 3.5 times more variation than the practices in patients’ experience of communication in the consultation, and 6.5 times more variation in whether or not patients trusted their doctor. In contrast, much more variation existed between practices than between the doctors for four indicators measuring aspects of patient experience that are less under the control of individual practitioners, such as cleanliness of buildings or ease of access.1 This is consistent with the few other studies that have rigorously examined variation in outcome at both individual and institutional level,3 suggesting, for example, that blood pressure measurement for patients with diabetes varies more between doctors but organisationally complex processes such as eye screening vary more between hospitals.4

The study also found that doctors’ scores for communication varied more in practices where the average score was low, leading to the conclusion that “higher performing practices usually contain only higher performing doctors” whereas “lower performing practices may contain doctors with a wide range of communication scores.”1 This conclusion is perhaps contentious since the paper does not report whether any of the practices or doctors were statistically significantly above or below average in their performance. This makes it difficult to judge the meaning of the variation, and it remains unclear whether or how patients could use doctor level performance data to choose their doctor, even if such data became publicly available in the future.3 The current English general practitioner patient survey is designed to measure satisfaction with a practice, not with individual doctors.

In the second linked paper, Harrison and colleagues examined whether or not pay for performance improves patient outcomes.2 They carefully examined whether the introduction of the Quality and Outcomes Framework reduced emergency hospital admissions for ambulatory care sensitive conditions (ACSCs, conditions where better primary care is expected to reduce the need for admission).2 The study compared admissions for ACSCs incentivised by the Quality and Outcomes Framework, ACSCs not incentivised by the scheme, and conditions not thought to be ambulatory care sensitive. It found that subsequent to the year that the framework was introduced, admissions for ACSCs incentivised by the scheme decreased in both absolute terms (a 10% reduction in the actual admission rate) and relative to the non-incentivised comparators, where admissions rose steadily.

However, as the authors discuss, the large observed reductions in admissions are hard to account for given the small changes in process measures associated with the Quality and Outcomes Framework. The findings might be partly or wholly due to other quality improvement activities in the period running up to the Quality and Outcomes Framework, although these would not be expected to cause a large change in admissions in the year that the scheme was introduced. The authors make the speculative though plausible suggestion that introducing pay for performance may lead to important changes in the organisation of care beyond the intended effect on incentivised processes, and that these associated changes may influence outcomes (for example, regular incentivised review of people with coronary heart disease might lead to earlier identification and treatment of people with poorly controlled angina even in the absence of incentives specifically to do this, reducing admissions for unstable angina).

The two papers help to narrow gaps in our understanding of how to measure and improve the quality of primary care. Roberts and colleagues’ study is a rare example of the practical value of measuring variation carefully. An important achievement of the movement to improve quality and safety has been to focus attention on the importance of systems and context, encouraging a shift away from blaming individual professionals for the failings of the systems within which they work.5 However, Roberts and colleagues’ study shows that individuals still matter,6 7 and it is sobering to note that 3% of Australian doctors account for 49% of complaints escalated to regional or federal ombudsmen, highlighting that individuals can be an important determinant of system performance.8 It remains unclear how systems should respond to individuals whose performance looks worse than that of their peers, or how best to balance supportive interventions to improve performance with rarer summative action to remove from the system those professionals with persistently unacceptable performance.9

Harrison and colleagues’ study2 contributes substantially to the small evidence base evaluating whether pay for performance improves longer term outcomes for patients.10 11 12 13 14 Although their analysis found significant reductions in related emergency hospital admissions, the mechanism by which this might have occurred is uncertain. The authors’ therefore concluded that pay for performance is no more of a “magic bullet” than any other quality improvement method, and that long term improvement is likely to require the persistent use of many different types of intervention.2 The broader lessons are that pay for performance remains an improvement method where hope continues to exceed evidence. Large investments in any improvement method, especially those with hoped for magic bullet properties, need to be matched by investment in formal evaluation.14