Analysis

When financial incentives do more good than harm: a checklist

BMJ 2012; 345 doi: http://dx.doi.org/10.1136/bmj.e5047 (Published 14 August 2012) Cite this as: BMJ 2012;345:e5047
  1. Paul P Glasziou, professor1,
  2. Heather Buchan, director of implementation support2,
  3. Chris Del Mar, professor1,
  4. Jenny Doust, professor 1,
  5. Mark Harris, professor 3,
  6. Rosemary Knight, associate professor3,
  7. Anthony Scott, professorial fellow4,
  8. Ian A Scott, director of internal medicine5,
  9. Alexis Stockwell, research programme manager6
  1. 1Centre for Research in Evidence-Based Practice, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Queensland 4229, Australia
  2. 2Australian Commission on Safety and Quality in Health Care, Sydney, Australia
  3. 3University of New South Wales, Sydney
  4. 4University of Melbourne, Victoria, Australia
  5. 5University of Queensland, Brisbane, Australia
  6. 6Institute of Health and Biomedical Innovation, Queensland University of Technology, Australia
  1. Correspondence to: P P Glasziou Paul_Glasziou{at}bond.edu.au
  • Accepted 1 July 2012

Financial incentives can sometimes improve the quality of clinical practice, but they may also be an expensive distraction. Paul Glasziou and colleagues have devised a checklist to help prevent their premature or inappropriate implementation

Financial incentives (pay for performance) for clinicians are an intuitively reasonable solution to the well documented gaps between evidence based best practice and routine care.1 They were fundamental to the 2004 Quality and Outcomes Framework (QOF), which paid primary care physicians in England up to 25% of their income for achieving 147 performance indicators, including 76 clinical targets (such as recording smoking behaviour, keeping blood pressure and cholesterol levels below targets, and spirometry in patients with asthma).2 Whether the cost (around an extra £1bn (€1.3bn; $1.6bn) annually) was justified has been contested.3 Similar attempts include over 170 initiatives in public and private US hospitals,4 and Australia’s Medicare Practice Incentives Program, which targets quality in primary care.5

To aid those making the difficult and costly decision of whether and how to use a financial incentive, we reviewed the evidence on the positive and negative effects of financial incentives and developed a simple checklist.

Synopsis of the evidence

Current evidence on the effectiveness of financial incentives is modest and inconsistent. Outside healthcare, early research suggested that financial incentives improved employee motivation and performance, but a meta-analysis found this was not always true for complex systems, where careful design and integration within the organisation was needed.6 An overview of four systematic reviews in healthcare found none had examined the effect on patient outcomes.7 Financial incentives had mixed effect on consultation or visit rates (improving 10 of 17 outcomes from three studies) and generally improved processes of care (41 of 57 outcomes from 19 studies) and referrals and admissions (11 of 16 outcomes from 11 studies) as well as reducing prescribing costs (28 of 34 outcomes from 10 studies).7 However, they were ineffective in improving compliance with guidelines (improving five of 17 outcomes from five studies). A Cochrane review of seven eligible studies in primary care found that financial incentives were effective for some outcomes in some settings but concluded that there was “insufficient evidence to support or not support the use of financial incentives to improve the quality of primary health care.”8

The studies paid insufficient attention to effect modifiers such as the nature and complexity of the target behaviour, the size and method of the incentive, the health professional group being targeted, and the organisational environment. While incentives for individuals have been extensively examined, group rewards are less well understood.4 Theoretically, the costs of the incentives might be repaid by reduced costs, but evidence on cost effectiveness is limited.4 Finally, and most crucially, most studies gathered few data on potential unintended consequences, such as attention shift, gaming, and loss of motivation.

The checklist

If all the answers are “yes” to the first six questions (part A)—which help decide whether a financial incentive should be considered at all—then the three questions in part B help with the design of the potential incentive programme. If the answer to any of the first six questions is no, then financial incentives should be postponed while further information is sought, or abandoned if there is no likely benefit.

Part A: Is a financial incentive appropriate?

1. Does the desired clinical action improve patient outcomes?

Any incentive targeting change in clinician behaviour requires strong evidence that the desired clinical action will improve patient outcomes (at the level of “strong recommendations” set out by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) group; www.gradeworkinggroup.org). For treatments this will generally mean consistent results from high quality randomised trials or a systematic review. Guideline recommendations alone are usually insufficient: they may be based on weak evidence or may not have been developed using an evidence based process (even guidelines from the World Health Organization9 and American College of Cardiology and American Heart Association10).

This problem is shown by the financial incentives to achieve a glycated haemoglobin target of <7% in diabetic patients. The target is based on prognostic data from cohort studies and the UK Prospective Diabetes Study in the 1990s, even though three concurrent large trials subsequently showed no benefit compared with more liberal targets. One large trial was stopped early because of higher mortality with the lower target (7%),11 leaving the “correct” target uncertain because most intervention patients did not achieve even the 7% target.

2. Will undesirable clinical behaviour persist without intervention?

Before contemplating a change we should obtain data on the extent of the problem and trends suggesting the problem is not resolving (from audits, surveys, and registry studies). The types of undesirable behaviour (box) are underuse of clearly indicated interventions in eligible patients, overuse of ineffective or harmful interventions, and misuse (improper or inefficient administration of indicated care to eligible patients or care given to ineligible patients).

Examples of undesirable clinical behaviours

Underuse
  • Warfarin prescribing in patients with atrial fibrillation

  • Use of β blockers in heart failure

  • Secondary prevention (statins, angiotensin converting enzyme inhibitors) after acute myocardial infarction

  • Immunisation, metformin in obese diabetic patients

  • Prophylaxis for deep vein thrombosis in orthopaedic surgery

Overuse
  • Unnecessary investigation of low risk patients with atypical chest pain (troponin, stress testing, computed tomography of coronary arteries)

  • Repeated cholesterol testing

  • Use of antibiotics in viral respiratory tract infections

  • Prolonged use of nebulised bronchodilators in exacerbations of chronic obstructive pulmonary disease

Misuse
  • Inappropriate use of psychotropic drugs in elderly patients

  • Mistimed antibiotic prophylaxis in surgical patients

Thresholds for underuse or overuse of care should vary according to the effect of the proposed change on patient outcomes, patient eligibility, and the complexities and costs of providing that care. Thus an effective, easily administered, safe, and inexpensive treatment should have a low threshold for underuse, and vice versa.

3. Are there valid, reliable, and practical measures of the desired clinical behaviour?

Measures for incentives need to be valid (measuring what is intended) and reliable (precise and reproducible) and preferably capable of independent verification and routine collection. Surrogate measures can be dangerous if “improvement” does not translate into patient outcomes. For example, clinics randomised to a financial incentive showed improved recording of smoking status and provision of advice compared with control clinics but without any difference in the proportion of patients who tried to stop smoking.12 The measure should be practical, and the cost of collecting the data must be outweighed by the benefits to patients. We found no studies on the cost of collecting clinical indicators.

4. Have the barriers and enablers to improving clinical behaviour been assessed?

Several barriers or enablers—including the nature of the intervention, its mode of dissemination, the clinicians, their environment, and the attitudes of patients—interact to determine whether, and how quickly, any change in clinical practice happens and whether it is sustained.

These barriers and enablers need to be understood before the change technique is designed. This may need a literature review, surveys, focus groups, or a combined approach.13 Subsequently, methods can be devised to overcome identified barriers, although little is known about the most effective approaches to tailoring incentives according to desired change in delivery of care.14 Thus, financial incentives may compensate for the additional costs of providing a service, act as a stimulus for behaviour change, or motivate practitioners or organisations with the economic reward.

5. Will financial incentives work, and better than other interventions to change behaviour, and why?

Clinical behaviour can be changed by many interventions, including education, audit and feedback, opinion leaders, reminders, collaborative quality improvement, regulatory approaches, public reporting of performance, and financial incentives. Systematic reviews reach common conclusions: much of the evidence is weak, reporting of interventions is incomplete, no strategy is consistently effective, and individual studies show effect sizes that are small to modest—around 5-10%—with unpredictable variation.13 Rarely are sustainability and cost reported. All this makes deciding when financial incentives are a better choice than other methods challenging.

The large psychology literature shows the importance of alternative drivers to clinician behaviour, especially intrinsic motivation.15 Worse, external rewards such as targets and incentives can actually reduce intrinsic motivation. Although reduced motivation was not reported among general practitioners in England after the introduction of QOF payments, it was seen among practice nurses.16

6. Will benefits clearly outweigh any unintended harmful effects, and at an acceptable cost?

Financial incentives may have unintended consequences.17 Possible downsides include the following:

  • Attention shift—Incentives in one practice area may decrease activity in another. The introduction of targets for waiting times in English emergency departments resulted in staff being recruited from other areas in the hospital and the cancellation of elective surgery lists.18 General practitioners in the UK decreased their average time in clinical teaching after QOF was introduced.19

  • Gaming is behaviour aimed at obtaining strategic advantage without necessarily realising a patient benefit. For example, 16% of English emergency departments reported directly manipulating the data to appear that they had met the four hour target for discharging or transferring patients.18 A related practice is “upcoding”—stretching the classification of clinical episodes to obtain maximum remuneration.

  • Harm to the patient-clinician relationship was found when consultations became more centred on achieving QOF targets than on what the patient wanted.9 Clinicians also became reluctant to attend patients for whom QOF targets would have been difficult to achieve,19 and informed consent was bypassed in the imperative to attain targets.19

  • Reduction in equity—Inequalities in the achievement of targets by age, sex, and ethnic group persisted after the introduction of QOF payments despite their improvement overall.20

Part B: Implementation

7. Are systems and structures needed for the change in place?

Realisation of the desired behaviour may depend on overcoming barriers to change. For example, although an Australian immunisation incentive paid to primary care practices was effective in most areas, it did not work in some remote areas because there was no reliable cold chain supply.21 Funding for necessary infrastructure should precede funding for the incentive.

Some common requirements are a communication strategy about the changes; information systems for monitoring achievement of milestones; endorsement from key bodies (particularly health professional groups); and support from key opinion leaders with examples of excellent practice.

8. How much should be paid, to whom, and for how long?

Poor design of financial incentives may explain their low effectiveness.4 They may be more successful if clinicians are involved in their design and operation; performance targets and measures are seen to be valid, precise, up to date, and attainable; and the opportunity costs in monitoring data collection are acceptable.4

The largest improvements come from the payment of individuals and teams rather than organisations.4 Distributing payment to clinical team members equitably retains team cohesion and cooperation.4

Common sense dictates that the size of the financial incentive should relate to the effort required to attain the desired behaviour, but this relation is unclear from empirical studies. More frequent payments may be more effective if the drivers are the same as for the feedback of performance. To reduce the risk of budget overspend, as occurred with QOF payments in England, capped payments should be considered.

If targets are set too high then poor performers may be discouraged; if set too low, high performers attain the targets so easily that funds are wasted. Perhaps the best returns on investment come from targeting people with the poorest performance and rewarding the extent to which they narrow the gap between desired and current performance.22

Finally, some incentives may lead to long term structural changes that persist, such as the setting up of computer systems, but many behavioural changes seem to revert when incentives are withdrawn.23

9. How will the financial incentives be delivered?

The measure used to ascertain the financial rewards (question 3 above) must set out the practical processes, with financial incentives closely aligned. The rules for this must be explicit, while retaining flexibility.

Discussion

We have assumed that the clinical changes and costs would be monitored, and hence have not included a possible tenth question: “What mechanisms are in place to review effects?”

The table gives some examples of applying the checklist. All of these programmes had at least one “no” in the first six questions, which may be acceptable in some contexts. However, a “no” for questions 1 and 2 is a particular concern. Few programmes seem to have examined barriers to change or compared the effects and costs with those of alternative ways of changing behaviour.

Application of financial incentives checklist to some real examples

View this table:

Policy makers have recognised the uncertainties and downsides of financial incentives, reflected by the large scale evaluations of the major UK and US programmes that have contributed to the evidence behind our checklist. However, a decision to implement an incentive should include a critical assessment beforehand. Our checklist could help in that assessment. In particular items 1-3 should have a “yes” on a first pass. Items with a “no” or “unclear” answer suggest the need for further background checks or pilot studies, and if a decision is still made to proceed then monitoring and evaluation seem warranted.

Financial incentives assume that paying more for a service will lead to better quality or additional capacity, or both. However, because money is only one of many internal and external influences on clinical behaviour, many factors will moderate the size and direction of any response. The evidence on whether financial incentives are more effective than other interventions is often weak and poorly reported. As in the QOF, new incentive programmes should include research to examine the impact, downsides, and cost effectiveness of incentives, and this should include evaluation of the comparative effectiveness of different strategies in different contexts. Such research should also include long term follow-up, since behaviour may revert when incentives are withdrawn.23 24

While some commentators and policy makers believe financial incentives can reduce the delay between new evidence and changes to clinical practice, there are many pitfalls. The proposed checklist is aimed at guiding implementers of financial incentives past some of these pitfalls.

Notes

Cite this as: BMJ 2012;345:e5047

Footnotes

  • Contributors and sources: We assembled a multidisciplinary writing group with experience of, and an interest in, the use of financial incentives for a one day conference that was open to the public. The group included clinicians from primary (JD, CDM, MH) and specialist care (IS), and experts in health economics (A Scott, A Stockwell), health policy (RK), quality improvement (HB), and clinical epidemiology (PG, JD, IS). A draft checklist was circulated to attendees several months before the meeting. Literature searches by group members were presented to the conference; comments and criticisms were discussed and recorded, and used by the authoring group to refine the checklist. PG conceived the idea; all authors presented at the conference, contributed to the checklist development, and assisted with the writing and editing of the paper. PG is guarantor.

  • Funding: This work was in part supported by an NHMRC Australia Fellowship grant number 0527500.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, no other relationships or activities that could appear to have influenced the submitted work .

  • Provenance and peer review: Not commissioned; externally peer reviewed.

References