Intended for healthcare professionals

Education And Debate

Guidelines, enthusiasms, uncertainty, and the limits to purchasing

BMJ 1995; 310 doi: https://doi.org/10.1136/bmj.310.6972.101 (Published 14 January 1995) Cite this as: BMJ 1995;310:101
  1. Martin McKee, senior lecturera,
  2. Aileen Clarke, senior lecturera
  1. a Health Services Research Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT
  1. Correspondence to: Dr McKee.
  • Accepted 8 November 1994

Recently government ministers have set out their vision of the future of purchasing. Ineffective treatments will be discarded and purchasing will be based on guidelines or protocols rather than activity. But have the advocates of this approach considered all the issues? This paper examines the challenges of balancing the desire for protocol based uniformity with the needs of individual patients, explores the extent to which existing purchasing structures can support this process, and questions whether such moves will actually lead to reduced costs. In each case it is concluded that oversimplistic analyses are likely to be misleading and that much of the current debate fails to recognise the complexity of health care.

A health service in which decisions are rational and based on evidence has obvious benefits. Some current treatments are ineffective and others that are effective are underused. Much effort is being devoted to developing clinical guidelines and protocols. Amid continuing disillusion with audit some politicians, managers, and health serviceresearchers see the purchasing process as the way forward. There have been several initiatives to increase the effectiveness of health care through the purchasing process, described in recent executive letters,1 2 3 including the “Effective Healthcare” bulletins, the “Outcomes Clearing House,” and the changing focus of the research and development programme. These seek to gather together the best available evidence. But it is less clear how this information should be translated into practice.4

This vision sees all interventions progressively being evaluated by randomised controlled trials and purchasers undertaking comprehensive assessments of need that will indicate which services should be purchased: locally adapted guidelines will be implemented by all providers; purchasers will buy not activity but guidelines or protocols5; change will come about through the “incredibly powerful lever of contracting”6; and Dr Mawhinney's seven steps to better purchasing will have been achieved.7

But not everyone welcomes the imposition of guidelines through purchasing. Some see it as a political move to reduce professional power, which is an obstacle to the working of the market.8 Others see it as a crude means of reducing costs, as ministers, citing evidence of widespread variations, argue that much care is ineffective.9 Finally, some people oppose “cookbook medicine,”4 in which overrigorous application of guidelines does not permit sufficient recognition of the needs of individual patients. Have those who advocate using purchasing as a means to achieving completely rational, systematic, and evidence based health care really considered how feasible this is? Three questions require answers. What is the role of clinical judgment in the face of inadequate research evidence and legitimate physician and patient preferences? Are there constraints on the extent to which purchasing can be used as a mechanism to improve clinical practice? Can guidelines really reduce costs? Overcoming clinical uncertainty

The most enthusiastic advocates for the purchasing of guidelines and protocols may have paid insufficient attention to the uncertainty inherent in clinical practice, with the imposition of a spurious rationality on a sometimes inherently irrational process. The purchasing of guidelines or protocols is seen as bringing order into the chaos of widespread variations. Purchasers will stop paying for care that is ineffective. This may be easy when the research evidence of ineffectiveness is clear cut, when a test has low predictive power and adds nothing to other diagnostic information, or when a treatment brings no improvement in health. The history of medicine is full of such examples of treatments once popular but now known to be valueless10—for example, gastric freezing or treatments for “night time starvation” or the “wandering womb.”

It is also possible, though difficult, to stop purchasing one treatment or diagnostic technique when two are of equal effectiveness but different cost; when less effective treatment is more expensive11; or when a treatment is being offered in circumstances in which the outcome is known to be poor, such as a hospital performing a small number of certain procedures.12 Unfortunately, real life is even more complicated. Most decisions are affected by many different factors, including characteristics of the doctor and the patient and their interaction.

One problem is that treatment guidelines or protocols are useful only once an accurate diagnosis has been made. This depends not only on the clinical ability of the doctor but also critically on the health beliefs and illness behaviour of the patient and the quality of the doctor-patient relationship.

Diagnosis is also subject to bias because of availability error, in which the probability of a diagnosis seems more likely if a doctor has just seen a series of patients with a particular disease and then sees another with similar symptoms13 or a patient who knows someone witha set of symptoms of a particular serious disease is concerned that he or she has it too. This has stimulated the development of diagnostic guidelines, including computer based expert systems. Unfortunately, these also have their problems.14 It was recently noted that “diagnostic computer programs have come a long way, but they still have a long, long way to go.”15

Once a diagnosis has been made, factors affecting the clinical decision, and thus the role that treatment guidelines or protocols can play, fall into three categories. These are the probability of particular outcomes, the valuation of the different outcomes that may result, and willingness to accept or live with a degree of risk. Only the first of these is addressed directly by the effectiveness agenda.

Estimates of the risks and benefits associated with various courses of action are often inaccurate16 but research based guidelines rarely provide all the answers. Panels examining appropriateness, using formal consensus methods, frequently identify many indications where there is uncertainty, partly because of other, often unquantifiable factors.17

A further problem is that outcomes can be expressed prospectively only as probabilities applying to populations. Consequently, studies cannot predict with certainty the outcome for an individual. Though surgery for glue ear is often unnecessary, and is more likely to be ineffective if the hearing loss is less than 25 dB, some children with lower levels of hearing loss do benefit.18 Prostatectomy is much less likely to relieve mild than severe symptoms but, again, some men with mild symptoms benefit.19

VALUING OUTCOMES

Valuation of outcomes and dealing with risk are still more difficult. Even if the probability of a particular outcome can be ascribed to individual patients with reasonable certainty, each outcome may be valued differently. Two 40 year old pregnant women facing the decision about amniocentesis and, if the fetus has Down's syndrome, subsequent termination of pregnancy may be in possession of accurate information on the risks of all of the possible outcomes but may make different decisions because of the differing values that they place on each outcome. This is true of many clinical situations.20 21 When considering diagnostic tests, individuals place different values on information. This may be impossible to predict as it may be dependent on the result. In one study patients with possible multiple sclerosis who were found to have the disease derived benefit from possession of the information in terms of quality of life whereas those who remained without a firm diagnosis were worse off.22

Economists have developed methods of measuring utilities ascribed by patients to particular outcomes, but though it is possible to obtain aggregate values for groups of people, individuals' values vary widely, both with regard to the value placed on particular situations and with regard to the timing at which those states occur. Utilities, particularly at the point of accepting or rejecting an intervention, “cannot be averaged across individuals,”23 Also the result obtained is sensitive to the method used and the questions asked.24

Finally, even with accurate knowledge of risk and agreement on the valuation of outcomes there may be differing views on what constitutes an acceptable risk. People tend to be more averse to taking a risk when a potential gain is involved but become risk takers when a loss is possible,25 and specialists and general practitioners have different time horizons when assessing risk.26 However, other factors, many of them individual, are important, such as previous experience of treatment.27

Research based protocols and guidelines can identify the probability of various outcomes, facilitating informed choice,28 and can also stimulate debate on the value of each outcome. They cannot include individuals' values of different outcomes or change their willingness to accept risk.

Guidelines should

therefore only ever by used as guidelines. Purchasersseeking to convert them into restrictive protocols in contracts will have much to do to incorporate patient choice in a way that is distinguishable from the unjustifiable variation in clinical judgment that they seek to combat. This raises the question of monitoring. If purchsers are to move from purchasing activity to purchasing protocols,4 then accountability for public money requires them to decide how to measure what they are buying.3 This may be possible in an all or nothing situation—for example, stopping diagnostic dilatation and curettage in all women under 4029—even allowing for the limited discriminant power of routine coding systems30 and the opportunities for gaming through shifts in definitions of diagnoses and other variables.31 But it is much more difficult when there is legitimate variation. It is for these reasons that Wennberg concluded that “micromanagement” of the doctor-patient relationship is impossible.32

Purchasing mechanism

Even if the view is accepted that guidelines should not be imposed too rigorously and they are used to educate rather than compel, current purchasing structures provide insufficient support for this process. On the one hand, there is recognition that the purchasing function is underdeveloped.33 34 35 On the other, those trying to develop purchasing are faced with continuing environmental turbulence, including mergers of health authorities, mergers with family health services authorities, mergers with subsequent abolition of regions, and, most recently, substantial increases in the coverage of general practice fundholding.36 At the same time the Department of Health is seeking to reduce management costs further in a system that by its nature requires greater investment in management than the one it is replacing, if there is to be a sustained attempt to promote effective and equitable care.

Many would argue that the purchasing process can never make a major contribution to increasing the effectiveness of health care. Some of themost compelling reasons for this view include, firstly, the built in flaws in the functioning of any internal market in health care and the scope for opportunism that this affords.37 Secondly, the quantity and extent of “quality police” who would need to be employed to ensure adequate continuing external quality assessment38; and, thirdly, the fact that most available evidence suggests that locally acceptable activities perceived as internal to an organisation or a group of clinicians (for example, education by local opinion leaders) are the most effective at changing practice.39

Many staff in purchasing organisations have little formal training in the concepts of effectiveness and appropriateness, though groups in at least two English regions are actively seeking to rectify this. Contracts managers often deal with a wide variety of contracts covering a multiplicity of specialties. Medical mystification can be used extremely effectively by clinicians seeking to diminish the ability of sometimes rather junior contracts mangers to have an impact on their practice. Public health can make an important contribution, but attempting to keep up to date with the minutiae of change in clinical practice in several different specialties may not represent the most efficient use of public health resources. Purchasing by fundholding general practitioners may overcome some of these problems but introduces others, particularly in relation to diseconomies of scale, transaction costs, or the relative powerlessness of small fundholders in the face of major providers.

Saving costs

Purchasing can be viewed as a means of either saving costs or increasing the effectiveness of care. For many people it is seen as a combination, though with differing degrees of emphasis on one or the other. Arguments for using guidelines as a means of reducing costs are based on the awareness of extensive variation in health care intervention rates. This variation exists at all levels of the health care system and hastraditionally been ascribed to clinical uncertainty and, more recently, to clinical enthusiasm,40 doctors doing more of what they enjoy and less of what they do not.

There is an implicit belief that high intervention rates can be equated with a degree of ineffective care, a view that underpins the comparative approach to assessing need.41 But there is little evidence that this is true. Studies of high and low use areas in the United States found similar levels of inappropriate interventions.42 Those who intervened more did so equally on patients with appropriate and inappropriate indications. Guidelines can be seen as a means of reducing the intervention rate of the more enthusiastic doctors, a view supported by most published studies that have focused on means of reducing unnecessary investigations.43

The view that the purchasing of guidelines should be used to increase the amount of effective care and that this is congruent with reducing costs stems from this view. Guidelines may, however, increase the volume of activity and thus cost. There is growing evidence of a large number of people in the community with symptoms that could be relieved effectively but who have not sought treatment.44 45 Calculations of costs and benefits should include the cost of developing, implementing, and monitoring the guidelines. Experience with medical audit46 in the United Kingdom and professional review or the work of the Agency for Health Care Policy and Research in the United States suggests that this may be considerable.

The work entailed in developing and implementing guidelines, then, should not be underestimated. Guidelines need to be developed locally and disseminated within the context of an educational programme.37 47 These processes take time, effort, skills, and money. And the continuing lack of appropriate research based evidence in many areas is a major problem. Poorly developed or insufficiently comprehensive protocols or guidelines can be inappropriate, unhelpful, or evenpotentially misleading.48

The way forward

At one level the issue of implementing guidelines through purchasing reflects a wider debate about the validity of applying models based on rational decisions and perfect information to complex adaptive human systems.49 Dogmatic approaches based on these models seek to avoid market failure by increasing the quality of information used to inform transactions, though the advocates of these policies often ignore the risk that in doing so they may precipitate market failure through increased administrative costs or failure to value such qualities as altruism. These are fundamental issues that need to be addressed explicitly by health policy makers, though they may be seen as somewhat esoteric by those charged with implementing government policies.

With respect to more practical issues, this review has highlighted some limitations of the purchasing process as a means of increasing the effectiveness of health care that is delivered. This should not be interpreted as reflecting either complacency that all is well or that nothing can be done. Purchasing clearly has a contribution to make, though it may be more limited than originally anticipated. Large areas of clinical practice will remain where attempts to purchase by means of protocols will be impossible. In these areas more appropriate approaches including increased reliance on education and clinical audit to improve quality of care are likely to offer the best way forward.37

Implementation must be sensitive to the concerns of both health care professionals and their patients. There must be recognition that cooperation achieves better results than conflict and coercion.50 Given the importance of the “doctor as medicine,” forcing doctors to abide by guidelines that they do not believe inespecially if this is obvious to the patient—is likely to have an adverse effect on outcome. Ways of reconciling patient choice with research based evidence are needed when they conflict.

Perhaps the most difficult issue for all of us concerned in purchasing or providing care is the need for an increased acceptance that a degree of uncertainty is integral to health services. When Heisenberg introduced the uncertainty principle over 60 years ago,51 this did not halt progress in theoretical physics. Rather, by explicitly identifying the limitations to knowledge it enabled scientists to pursue the achievable rather than the impossible.

Certainly a more explicit recognition of the limits to and costs of implementing guidelines and protocols through purchasing may inject a greater degree of reality into some of the current debates.

We are grateful to Colin Sanderson, Naomi Fulop, Nick Black, Nick Mays, and Jenny Roberts for helpful comments.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
View Abstract