Education and debate

Decision analysis and the implementation of research findings

R J Lilford, professor of health services research, ^a S G Pauker, professor, ^b D A Braunholtz, senior research fellow, ^a Jiri Chard, research associate. ^c

^a Department of Public Health and Epidemiology, University of Birmingham, Birmingham B15 2TT, ^b Department of Medicine, New England Medical Centre, Tufts University, 750 Washington Street, Boston, MA 02111, USA, ^c MRC Health Services Research Collaboration, University of Bristol, Bristol BS8 2PR

Correspondence to: Professor Lilford rlilford{at}doh.gov.uk

Series editors: Andrew Haines and Anna Donald

Evidence based medicine is more than just reading the results of research and applying those results to patients because patients have particular features that may make them different from the "average" patient studied in a clinical trial.¹ There are two types of differences. The first type of differences comprise those that affect probability (for example, the probability that treatments will have the same absolute or relative effects as those measured in the trial). The second type of differences comprise those values (or utilities) that affect how much of a side effect a person is prepared to trade off against the positive advantages of treatment.

Thus it is necessary for doctors to relate the results from a trial to their particular patient. Health professionals usually do this intuitively, but formal decision analysis provides an intellectual framework for developing an explicit decision making algorithm which can be criticised and improved. Although, currently, time constraints make it unrealistic to conduct a separate decision analysis for each patient, computer programs may soon help overcome this problem. It is, however, feasible for decision analyses to be done for categories of patients with similar clinical features and personal utilities. The results of such generic decision analyses provide a sound basis for developing clinical guidelines. Decision analysis thus provides a rational means of allowing health professionals to move from finding evidence to implementing it.

	An example of decision analysis

Decision analysis is described in detail elsewhere,^2-5 but we will illustrate it with an example. Megatrials have shown that thrombolytic drugs save lives in cases of suspected myocardial infarction.⁶ However, these drugs can cause stroke, which may leave the patient severely incapacitated. Also, there is a choice of drugs; the genetically engineered accelerated tissue plasminogen activator seems more effective than streptokinase in preventing death from myocardial infarction, but it has a higher chance of causing stroke. The risk of causing a stroke does not depend on when treatment is given. However, the probability of preventing death from myocardial infarction does depend on how soon treatment begins after the onset of symptoms and on whether the patient has actually had a myocardial infarction and on their risk of dying if they have had one. To further complicate the picture, the relative advantage of tissue plasminogen activator over streptokinase in preventing cardiac death dissipates after about six hours, and thrombolytic drugs can cause other complications (such as haemorrhage and anaphylaxis).

How can the clinician account for these factors? We base our example on the work of Kellett and Clarke, who did a systematic review and then modelled all of these variables using decision analysis (fig 1).⁷ Probabilities of the main outcomes according to the treatment given are presented in table 1. Specimen utilities are used for the various outcomes, a value of 1 for healthy survival and 0 for death. About half of patients who have a stroke in these circumstances will survive, but often with some degree of impairment; the mean utility of existence after stroke is 0.5.⁸ The results of running the base case model (that is, for a 55 year old man with chest pain of recent onset and classical changes on electrocardiography, typical of participants in trials of thrombolysis) are shown in table 2. Clearly, there is much expected utility to be gained by using thrombolytic drugs, and, moreover, tissue plasminogen activator is the drug of choice. Even if we assume a passionate desire to avoid the disability associated with stroke, giving it a utility of 1 (for example, a healthy person who would equate a 20% risk of death with a 10% risk of stroke), the thrombolytic treatments remain optimal (data not shown). However, we get very different results as we move away from the base case. For example, chest pain in a 55 year old man with a normal electrocardiogram is associated with only a 17% risk of myocardial infarction, and thrombolytic drugs would lower the expected utility in these circumstances. The same man with a normal ST wave but an abnormal T wave has about a 24% risk of myocardial infarction; thrombolysis is advantageous, but only slightly and it would be disadvantageous if he was younger (his risk of dying if he had a myocardial infarction would drop to 5% at age 45), if he presented late (after 6 hours), or if he was particularly strongly averse to residual morbidity from stroke.

Key definitions in decision analysis Utility A utility represents a patient's preference for one outcome over others. A utility is given a numerical value which is then used in the decision analysis. Utilities (or values) are quantified on a scale (usually from 0 to 1) that allows meaningful comparison between alternative outcomes. Probability Decision analysis is based on bayesian statistics. Probabilities of clinical outcomes are the doctor's best guess, based on indirect evidence such as laboratory studies, updated with data from relevant clinical studies by Bayes's theorem. Decision tree The various decision points and their consequences are mapped (with their associated probabilities and utilities) to form a decision treea visual representation of the decisions available. The aim of decision analysis is the logical reduction of a decision process into its individual decision points. Probabilities and utilities are assembled logically to determine the optimum treatment (that is, the treatment with the highest expected utility). Sensitivity analysis Sensitivity analysis is used to determine the robustness of a choice made using decision analysis. By varying the utilities and outcome probabilities it is possible to see how easily a decision would changethat is, how sensitive it is. This makes it possible to produce guidelines so that treatment can be tailored to groups of patients with similar characteristics; the final choice is, of course, the patient's.

View larger version (36K): [in this window] [in a new window]   Fig 1.   Decision tree for choice of treatment in case of suspected myocardial infarction (MI) of aspirin only v aspirin and streptokinase v aspirin and tissue plasminogen activator. P stands for probability. For example, P2a is the probability of surviving without impairment for someone who has had an MI and who has been treated with aspirin only. The formulas for expected utilities are aspirin, P1{(1*P2a) + (0*P3a) + (0.5*P4a) + (0*P5a) + (0*P6a)} + (1P1){(1*P7a) + (0*P8a) + (0.5*P9a) + (0*P10a) + (0*P11a)}; streptokinase, P1{(1*P2b) + (0*P3b) + (0.5*P4b) + (0*P5b) + (0*P6b)} + (1P1){(1*P7b) + (0*P8b) + (0.5*P9b) + (0*P10b) + (0*P11b)}; tissue plasminogen activator, P1{(1*P2c) + (0*P3c) + (0.5*P4c) + (0*P5c) + (0*P6c)} + (1P1){(1*P7c) + (0*P8c) + (0.5*P9c) + (0*P10c) + (0*P11c)}. Figures on the far right are values

	Probability

Effective care in general, and decision analysis in particular, are underpinned by probabilities. Much of clinical research is concerned with providing the necessary probabilistic information.

Epidemiological studies provide baseline estimates of risk (for example, the risk of death from myocardial infarction) and data for revising these risks on the basis of test results (here test denotes any information about a patient, not only results from a laboratory). The ratio of the probability of an observed test result if the patient is or is not affected is known as the likelihood ratio. Given the patient's prior odds of being affected (odds are simply a ratio of probabilities) and the likelihood ratio for the observed test result, the revised (posterior) odds, and hence the probability that the patient is affected by a condition, can be calculated by multiplying the prior odds by the likelihood ratio. For example, a young woman whose only brother has Duchenne's muscular dystrophy has prior odds of being a carrier of the disease of 1:2. A raised concentration of creatine kinase (likelihood ratio 28) gives posterior odds of 14:1 (a probability of 93%). Likelihood ratios of multiple tests can be multiplied together, provided they are relatively independent.

Interventional studies such as clinical trials provide data on the effects of treatments. These studies can give two kinds of probability: conventional and bayesian. Conventional (frequentist) statistical analyses give P values and confidence intervals based on the probability of seeing the observed result (or a more extreme result) given a particular state of the world, typically that different treatments are equally effective. However, decision analysis, and bedside decisions generally, requires not the probability of already observed results given some assumed particular treatment effect, but rather the posterior probabilities of particular differences in the effects, given the observed data.⁹

If a trial comparing treatments X and Y finds an improvement in survival of 10 percentage points with treatment Y, a patient who is similar in relevant characteristics to those in the trial does not want to know that this observed improvement had only a 2.5% chance of occurring if the treatments are equivalent. The patient needs to know, for example, what the probability is that survival with treatment Y really is better than with treatment X. Probabilities that describe beliefs about the size of true effects are known as bayesian, and their calculation requires that a prior belief, expressed as a probability distribution, is updated according to the results of research. Because bayesian probabilities relate to the probability of the true state of the world they are the rational basis for the implementation of a finding and for calculating parameters, such as the number of patients who must receive a new treatment to help (or harm) one patient. When studies are very large, as in trials of thrombolytic drugs, the two types of statistics will give similar results.

In our example of a myocardial infarction we considered two kinds of patient variables. We first considered features that affect absolute risk, such as age, but not relative treatment effects, such as the 25% improvement in cardiac mortality that occurs over a wide range of ages when patients are treated with thrombolytic drugs. The second type of patient variable has an influence on relative treatment effects (for example, the duration of symptoms affects the effectiveness of thrombolysis). Of course, if trials were infinitely large we could look up the precise relative treatment effect for any given category of patient. However, even when overall effects are measured precisely the effects in subgroups of patients (strata) are typically imprecise.

Should clinicians take the overall effect and apply it to the subgroup, or should they use the imprecise measurement made in the subgroups? For example, the second international study of infarct survival (ISIS 2) trial of thrombolytic drugs was analysed in subgroups. Unsurprisingly, this showed a null effect for people who had had their pain for a long time, but it also unexpectedly found a null effect for those born under the star sign Gemini. On what basis can we believe one analysis of a subgroup and not the other? In a bayesian analysis of subgroups we must state our prior beliefs for how the effect in the subgroup may relate to the effect in the remainder of the group. ^{10 11} This prior belief would be that there is little or no difference between Geminis and non-Geminis. The observed difference will therefore fail to shift our prior belief, and our posterior belief will remain that Geminis and non-Geminis benefit similarly. Our prior belief in the difference between patients with prolonged pain and others would be less precise and would reflect our belief that those with prolonged pain will benefit less than patients with a shorter duration of pain; for example, from our knowledge of drugs and infarcts, we expect the benefits to be largest when these drugs are administered quickly. In this case the data reinforce our prior belief and enable us to be more precise about how benefit is reduced as delay increases.

	Values (utilities)

The great strength of decision analysis is that it is based not just on probabilities but also on the value placed on various outcomes. It therefore represents a method for synthesising both medical facts (probabilities) and human values (utilities), which together determine the best course of actionthat is, the course that stands to maximise expected utility.¹² Decision analysis reconciles evidence based medicine with patients' preferences.

There is debate about the best way to obtain these utilities. Utilities imply a trade off: the extent to which the disadvantages of one outcome can be offset by the advantages of another. For example, for certain patients with cancer of the larynx survival is better for those who have radical surgery than for those treated with radiotherapy. However, radical surgery limits the ability to speak, at least in the short term. There is then a trade off between survival (maximised by surgery) and the ability to communicate (which is retained to a much better degree with radiotherapy). If a patient would run a 10% chance of dying to avoid losing the power of speech then the patient values life with this impediment at 0.9 on a scale from 1.0 (healthy life) to 0 (death). The subject of utilities and how they can be elicited is discussed in more detail elsewhere.^13-16

	Sensitivity analysis, generic decision analysis, and the individual patient

When consulting with individual patients it is important to elicit their personal values or at least to get a sense of them. However, it is not essential to redo an analysis for every patient in a busy clinic. Decision analysis may also be done outside the consulting room using a selection of different probability and utility figures within a reasonable range; this is known as sensitivity analysis. We used this technique to see how the expected utility of thrombolytic drugs might vary according to a patient's medical and psychological characteristics to produce the guidelines in table 2. The sequence of events followed in performing a decision analysis, developing a guideline based on that analysis, and implementing the guideline are shown in figure 2.

View larger version (52K): [in this window] [in a new window]   Fig 2.   The sequence of events followed in performing a decision analysis, developing a clinical guideline based on the analysis, and implementing the guideline

Information on short term outcomes is often available from clinical trials but long term outcomes must be derived from observational studies. Since long term outcomes are often more important to the patient and third party payer, these should be modelled by decision analysis. For example, modelling was required to extrapolate the results of a trial evaluating the short term effects of different types of angioplasty from the information collected in the trial.¹⁷ Decision analysis is also useful when a clinical problem requires input from more than one set of study results; the effects of hormone replacement therapy have been analysed in many different studies, each concerned with different outcomes and with each outcome being valued differently.¹⁸ Furthermore, observational studies have shown that women have different risks at baseline (for example, thin women are at higher risk of fractures). Decision analysis has shown how these factors may be integrated to optimise individual treatment.¹⁹

Decision analysis is used to determine how to maximise an individual's expected utilities. By obtaining the median values of utilities from a large number of people the methodology can also be used to derive expected utilities for the community. When the costs of various options are included this is called a cost utility analysis. However, using decision analysis to make decisions for groups of patients creates some thorny ethical issues, especially when there is a conflict between maximising utility and maximising equity.

	Conclusions

Decision analysis depends on probabilities and values, neither of which can be measured with certainty. These problems are not lessened when health professionals approach them intuitively; decision analysis makes these uncertainties explicit. The attempt to make complex decisions intuitively inevitably results in gross oversimplifications because it is impossible to incorporate and consider several components of a decision simultaneously. There is a large amount of empirical literature on the limitations of intuitive reasoning that is summarised by Dawes et al.²⁰

Most research findings are applied unsystematically and intuitively. If evidence based medicine is to be seen through to its logical conclusion and if both empirical evidence and human values are to be incorporated into decision making, then this duality (the explicit collection of data v its implicit use) must be addressed. ^{21 22}

	References

1. Glasziou PP, Irwig LM. An evidence based approach to individualising treatment. BMJ 1995; 311: 1356-1359[Free Full Text].
2. Keeney RL, Raiffa H. Decisions with multiple objectives. London: Wiley , 1976.
3. Weinstein M, Fineberg HV. Clinical decision analysis. London: Saunders , 1980.
4. French S. Readings in decision analysis. London: Chapman and Hall , 1989.
5. Sox HC, Blatt MA, Higgins MC, Marton KI. Medical decision making. Boston: Butterworth Heinemann , 1988.
6. Second International Study of Infarct Survival (ISIS 2) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected myocardial infarction: ISIS 2. Lancet 1988; ii: 349-360.
7. Kellett J, Clarke J. Comparison of accelerated tissue plasminogen activator with streptokinase for treatment of suspected myocardial infarction. Med Decis Making 1995; 15: 297-310[Abstract/Free Full Text].
8. Tsevat J, Goldman L, Lamas GA. Functional status versus utilities in survivors of myocardial infarction. Med Care 1991; 29: 1153-1159[Medline].
9. Lilford RJ, Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue. BMJ 1996; 313: 603-607[Free Full Text].
10. Donner A. A bayesian approach to the interpretation of sub-group results in clinical trials. J Chronic Dis 1982; 34: 429-435.
11. Oxman AD, Guyatt GH. A consumer's guide to sub-group analyses. Ann Intern Med 1992; 116: 78-84.
12. Swales J. Science in a health service. Lancet 1997; 349: 1319-1321[Medline].
13. Lilford RJ. Trade-off between gestational age and miscarriage risk of prenatal testing: does it vary according to genetic risk? Lancet 1990; 336: 1303-1305[Medline].
14. Johnson N, Lilford R, Jones SE, McKenzie L, Billingsley P, Songane FF. Decision analysis to calculate optimum treatment for microinvasive cervical cancer. Br J Cancer 1992; 65: 717-722[Medline].
15. Lilford RJ, Jackson J. Equipoise and the ethics of randomisation. J R Soc Med 1995; 88: 552-559[Medline].
16. Pauker SP, Pauker SG. Prenatal diagnosis: a directive approach to genetic counselling using decision analysis. Yale J Biol Med 1977; 50: 275-289[Medline].
17. Sculpher M, Michaels J, McKenna M, Minor J. A cost-utility analysis of laser-assisted angioplasty for peripheral arterial occlusions. Int J Technol Assess Health Care 1996; 12: 104-125[Medline].
18. Johnson N, Lilford RJ, Mayers D, Johnson GG, Johnson JM. Do healthy asymptomatic post menopausal women want routine cyclical hormone replacement? A utility analysis. J Obstet Gynecol 1994; 14: 35-39.
19. Col NF, Eckman MH, Kara RH, Pauker SG, Goldberg RJ, Ross EM, et al. Patient-specific decisions about hormone replacement therapy in postmenopausal women. JAMA 1991; 277: 1140-1147.
20. Dawes R, Faust D, Meehl PE. Clinical versus actuarial judgement. Science 1989; 243: 1668-1674[Abstract/Free Full Text].
21. Dowie J. "Evidence-based," "cost-effective" and "preference-driven" medicine: decision analysis based medical decision making is the pre-requisite. J Health Serv Res Policy 1996; 1: 104-113.[Medline]
22. Dowie J. The research-practice gap and the role of decision analysis in closing it. Health Care Analysis 1996; 4: 5-18. [Medline]