Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework
BMJ 2011; 342 doi: https://doi.org/10.1136/bmj.d3590 (Published 28 June 2011) Cite this as: BMJ 2011;342:d3590All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
We are pleased that Wharam, Serumaga and Soumerai share the concerns
that we addressed in the paper. Universal programmes such as the Quality
and Outcomes Framework eliminate control groups and preclude the use of
certain study designs, and we agree that for a programme as revolutionary
as the QOF, trialling would have been preferable to universal
implementation. In the absence of control groups, however, an Interrupted
Time Series design is the best analytic approach - indeed it is the
approach Serumaga and Soumerai used themselves [1]. Attentive readers will
note that our model was log-linear rather than linear, and as such it did
in fact gradually plateau as achievement scores increased. In the pre-
intervention period this model was an excellent fit to the indicator
trends. Post-intervention, the trends for incentivized indicators
increased above those predicted by this model, whereas the trends for non-
incentivised indicators declined.
It is possible that changes in recording activity contributed to our
findings. However, the results of laboratory tests are mostly
electronically downloaded to practices' clinical computing systems, so
documentation of these activities is not affected by physician behavior.
With respect to prioritisation, every activity we selected - whether
incentivised or non-incentivised - was recommended in guidelines.
Activities were excluded from the QOF for a variety of political and
pragmatic reasons, not necessarily because they were deemed less
important, and some controversial activities were included in the scheme.
This is reflected in the pre-QOF achievement rates - for example: the four
activities with the highest achievement rates in 2002/3, which were
presumably valued by practices, were not subsequently included in the QOF.
Re-prioritisation can occur at any time, therefore if prioritisation and
incentivisation were unrelated it would be an extraordinary coincidence if
- as we found - clinicians started ranking incentivised activities as
higher priority just at the point when incentives were introduced.
[1] Serumaga B, Ross-Degnan D, Avery A, Elliott R, Majumdar S, Zhang
F, Soumerai S (2011). Has pay-for-performance improved the management and
outcomes of hypertension in the United Kingdom? An interrupted time-series
study. BMJ 2011;342:d108 doi:10.1136/bmj.d108
Competing interests: No competing interests
Doran and colleagues conclude that "Improvements associated with
[physician] financial incentives seem to have been achieved at the expense
of small detrimental effects on aspects of care that were not
incentivised." We believe that three major study limitations call this
conclusion into question.
The authors used a suboptimal research design with only three
baseline data points and no control group, basing all conclusions on a
presumed linear increase in achievement of indicators. A non-linear trend
that gradually plateaus is almost certainly more likely. Without better
knowledge of how physicians would have acted in absence of incentives, it
is nearly impossible to judge the impact of financial incentives.
Furthermore, most clinicians would likely rank the incentivized
measures higher in priority than the non-incentivized measures independent
of financial incentives. It is therefore quite possible that over time
physicians would have increasingly recognized the relative importance of
incentivized and non-incentivized measures even in the absence of
incentives.
There is also a reasonable likelihood that practices would document
measured processes more often than care that did not improve their
finances. The authors counter by noting that they found no differences in
documentation of measures assessed by physician groups (such as body mass
index) and third parties (such as laboratory tests). However, as far as
we can tell, the authors did not have independent data from such third
parties, but still relied on documentation by the practices themselves.
Thus, increases in incentivized measures may well be due to better
documentation and/or a natural increase in physician-preferred care. The
lack of increase in non-incentivized measures could be due to less
documentation or the expected trend for lower priority care. Finally,
despite huge public expenditures, this and previous investigations in the
journal [1] have failed to demonstrate any sustained health benefit
resulting from incentive payments.
[1] Serumaga B, Ross-Degnan D, Avery AJ, Elliott RA, Majumdar SR,
Zhang F, Soumerai SB (2011). Has pay-for-performance improved the
management and outcomes of hypertension in the United Kingdom? An
interrupted time-series study. BMJ 2011;342:d108
doi:10.1136/bmj.d108
Competing interests: No competing interests
Tim Doran and colleagues are to be congratulated on a first class
piece of health services research into the effects of the QOF, the UK's
pioneering "pay for performance" scheme in primary care.
An important
worry about schemes of this kind is that: "what doesn't get incentivised
gets marginalised". However, Doran and colleagues find that the non-incentivised aspects of primary care quality that they were able to
measure were only slightly neglected. Does that mean policy makers can
relax and stop worrying about the unintended consequences of "pay for
performance" schemes? Unfortunately not. This particular "pay for
performance" scheme was implemented in a resource-rich climate of
accelerating health expenditure growth. In this favourable financial
climate, primary care providers had the capacity to hire new staff and do
more QOF activity without having to do less non-QOF activity. The iron
law of opportunity cost still held true in the "golden age" of NHS
spending growth in the 2000s - its just that the opportunities forgone
were the unmeasured and largely unnoticed opportunities to new things
other than QOF activity. No-one needs reminding that the financial
climate has changed. Resources are tight; budget constraints are hard;
opportunities foregone are not just bright ideas for doing new things with
extra resources. In these more austere times, what gets marginalised may
get noticed.
Competing interests: No competing interests
Unjustified conclusions - Qof is good for patients.
I am afraid this is not a 'rapid' response as I have just received my
paper copy of the BMJ in Vanuatu.
I am surprised that there are no responses from General Practitioners
who were out there implementing the Qof.
Of course a lot of the increase in achievement rate in the 2001-2004
was improving our IT systems and getting our activity on the computer in a
searchable way.
The major breakthrough came when we were able to generate lists of
patients that did not have incentivised markers recorded and we could
actively seek to either do the activity or have the already performed
activity recorded. Often this was delegated to the nurses.
So Qof not only paid us for what we had previously been doing but
also gave a system for doing it even better. The incentivised markers were
all extensively discussed and I had no problems about any of them. They
all passed the test of wanting them done on myself or a relative if
appropriate.
What I do have a problem with is the measly mouthed criticism from
academic departments that some rather messy non incentivised indicators
did not increase as fast as they would like. The conclusion 'Quality of
care improved under UK Qof for incentivised activities but fell
significently for non-incentivised activities' is totally unfounded. It is
based on a statement that nonincentivised achievements were significently
'below rates projected from preincentive trends.' Why should this rate
follow any particular line? Who determines where this line should be ? Are
these activities something an average general practitioner should be
concerned about?
It is only when the nonincentivised activities are looked at that one
understands maybe why there was not a similar increase as the incentivised
activities.(this is the problem with the synopsis of papers)
Take BP recording in people aged 22-44. As usual young men aged 22-44
are usually absent from a general practice waiting room. Why should I take
their BP when they come for a musculoskeletal injury? Where is the
evidence and the balance of good/bad. Running late as I usually did I
concentrate my activity where I think it can offer more benefit like
advice about manual handling (not recorded, not searchable).
I did a thyroid function test on a depressed woman over 50 occasionally
but not routinely. Where is the evidence that this is an essential
activity? The same could be said for weighing patients with osteoathritis
when a good doctor can make a reasonable eye estimation.(the evidence that
people with obesity manage to lose weight even with severe OA is not
good). There is controversy about asthma and betablockers especially in
management of heart failure and I have used then cautiously. As to cough
and antibiotic use at the same consultation then please let me be the
first to say that despite preaching against overuse of antibiotics that I
have frequently resorted to this activity.There should have been no change
in my activity on this marker over this period.
At the end of the day the nonincentivised activities are a motley
collection of things, some of importance but some of minor importance and
none approach the importance of the Qof indicators. Maybe we should
surprised that they increased at all or was it just that this activity was
better recorded.
What this paper adds is that financial incentives lead to improvement
in quality of care (where quality is determined by the Qof makers).
Competing interests: I was and am greatly in favour of the Qof as it gave us financial reward for greater activity and I am convinced that it benefited our patients though this is still to be shown