An evidence based approach to individualising treatment
BMJ 1995; 311 doi: https://doi.org/10.1136/bmj.311.7016.1356 (Published 18 November 1995) Cite this as: BMJ 1995;311:1356- aDepartment of Social and Preventive Medicine, Medical School, Herston, Queensland, Australia 4006
- bDepartment of Public Health and Community Medicine, Building A27, University of Sydney, Sydney, New South Wales, Australia 2006
- Correspondence to: Dr Glasziou.
- Accepted 20 August 1995
To which groups of patients can the results of clinical trials be applied? This question is often inappropriately answered by reference to the trial entry criteria. Instead, the benefit and harm (adverse events, discomfort of treatment, etc) of treatment could be assessed separately for individual patients. Patients at greatest risk of a disease will have the greatest net benefit as benefit to patients usually increases with risk while harm remains comparatively fixed. To assess net benefit, the relative risks should come from (a meta-analysis of) randomised trials; the risk in individual patients should come from multivariate risk equations derived from cohort studies. However, before making firm conclusions, the assumptions of fixed adverse effects and constant reduction in relative risk need to be checked.
Should all patients with acute myocardial infarct receive streptokinase? Should all patients with non-valvar atrial fibrillation receive warfarin? Such questions are best answered by assessing benefits and risks in each patient rather than focusing on the inclusion and exclusion criteria of the trial. For example, patients with a history of peptic ulcer were often excluded from thrombolytic trials because it would be unethical to put such patients at risk of major bleeding episodes, given the lack of proved benefit. However, now that we know the size of the benefit--about three deaths saved for every 100 patients treated1--clinicians must make an informed decision that weighs this benefit against the potential for harm in patients with ulcers.
Relying on the eligibility criteria for clinical trials is both erroneous and limiting. A too restrictive generalisation needs to be guarded against, and we are advised to ask, “Are the patients in this study so different from my patients that I could not apply the study results?”2 This is good advice, but how then do we decide when a patient is too different to benefit from treatment? The search for differences should be based on features of the disease process or risk rather than differences in sociodemographic characteristics.3
The converse of this question is, “Can the study results be generalised to all patients who would be eligible for the trial?” The answer might seem to be obviously “yes.” However, we will show that this is also incorrect after we have developed a general approach.
The basic model: separating benefit and harm
Lubsen and Tijssen proposed a separate assessment of the benefit and harm of treatment.4 As shown in figure 1, their model suggests patient benefit increases with risk from the disease--those most at risk have most to gain--but that harm or rates of adverse event will remain comparatively fixed. Thus at some low level of risk the benefits will only just balance the harm and we should refrain from treatment.5 6
This model works by converting the reduction in relative risk, which is useful for assessing the strength of the intervention, to a reduction in absolute risk, which is useful for assessing the clinical worth of the intervention. The relative risk is the ratio of clinical events in the treated group relative to the control group. For example, if the control group had a death rate of 12% and the treated group a death rate of 9% the relative risk is 9/12, or 75%, which implies a 25% reduction in relative risk with treatment. However, the reduction in absolute risk is 12%-9%, or 3%.
How do we extrapolate for patients at different risk? The reduction in absolute risk of 3% is unlikely to apply universally. For example, if the control group had a death rate of only 2% instead of 12% a reduction in absolute risk of 3% would give the impossible death rate of 2%-3%, or −1%. It is usually more reasonable to assume that the reduction in relative risk stays constant, which would suggest that the treated group would have a death rate of 1.5% (still a 25% reduction in relative risk but only a 0.5% reduction in absolute risk).
The model of Lubsen and Tijssen extends the above calculations to incorporate a fixed harm. This is illustrated in figure 2 and may be expressed in the following equation:
Net benefit=riskxreduction in relative risk-harm.
To apply this model requires four steps.
(1) Estimate benefit and harm
A randomised trial, or a meta-analysis of randomised trials, is the most appropriate method to estimate the reduction in relative risk with the intervention for various outcomes relevant to patients. Methods to identify, appraise, and combine trials are described elsewhere.7 Such an analysis is vital for proving an effect and estimating its comparative size. This is, however, only a first step in deciding which patients would expect to benefit from a treatment.
EXAMPLE: ANTICOAGULANTS AND NON-RHEUMATIC ATRIAL FIBRILLATION
There have been six randomised controlled trials of low dose warfarin in a total of 4269 patients with non-rheumatic atrial fibrillation. Five of these trials have recently been reviewed by Singer, who found an overall reduction in relative risk of about 73%.8 The table describes these five trials, as summarised by Singer, plus the more recent European atrial fibrillation trial.9
The reduction in the risk of stroke is dramatic. However, clinicians have been cautious because of the incidence of bleeding induced by warfarin, particularly the risk of intracranial haemorrhage.
(2) Check assumptions of relative benefit and absolute harm
The model assumes that the reduction in relative risk for benefit stays the same for all risks. This will not always be true. Hence a necessary check is a meta-analytic review of all trial data, with an examination of whether the reduction in relative risk varies with risk. Only if this seems to be constant should we calculate a combined estimate. Similarly, we need to check that the absolute harm is independent of risk.
Several factors may cause the reduction in relative risk to vary. Some diseases are more than one disease process, each of which may respond differently to the treatment. For example, a stroke can be embolic, thrombotic, or haemorrhagic. If stroke is considered to be a single entity, preventive treatment such as anticoagulant or antiplatelet drugs may benefit one group (with few haemorrhagic strokes) but cause net harm to another group (with mostly haemorrhagic strokes), although the risk is similar.
EXAMPLE
The reductions in relative risk in the table are reasonably constant for stroke rates in the control groups. Figure 2 plots the reduction in absolute risk for both the benefit--that is, the reduction in embolic strokes--and the harm--that is, the rates of intracranial haemorrhage. This shows that the reduction in absolute risk of thromboembolic strokes rises linearly with the risk of stroke, suggesting that the model of reduction in relative risk is appropriate. In addition, the rate of intracranial haemorrhage seems to be stable across varying risks of stroke. Thus both of the necessary assumptions seem to be fulfilled.
The assumption of a constant reduction in relative risk may sometimes be violated when the intervention has both positive and negative effects on one outcome. For example, Boissel et al have performed a meta-analysis on data from 13 published trials of class I antiarrhythmic agents after myocardial infarction.10 The reduction in relative risk clearly varied with the mortality in the control group. Patients at low risk had a relative increase in mortality, presumably because of the proarrhythmic effects of these agents. However, patients at high risk (mortality greater than 15% per year) showed a net beneficial effect. Thus the assumption that the reduction in relative risk is constant is not fulfilled. Boissel et al then modelled this as a constant proarrhythmic harm and a relative antiarrhythmic benefit. Here the harm and benefit cannot easily be separated because clinical outcome is the same for both, and the only way to disentangle the benefit and harm is to do a very large trial(s) with a wide range of risk groups.10
Factors that may modify the relative reduction in risk achieved by an intervention, such as the intensity of the intervention, the timing and mode of administration, and the risk, should be checked before moving on to the next step. This can be done graphically, as we have done for risk in figure 2, and statistically.11 12 13
(3) Weigh up benefit and harm
If the assumptions of relative risk reduction and constant harm are fulfilled the predicted benefit then needs to be weighed up against the potential harm.
EXAMPLE
If we now use the estimates of the reduction in relative risk for thromboembolic stroke8 and the risk for fatal haemorrhage14 from meta-analyses in the equation for net benefit we obtain figure 3. To choose the point where benefit outweighs harm we need an equivalence between the thromboembolic strokes and fatal intracranial haemorrhages (within the six randomised trials all patients with an intracranial haemorrhage who took warfarin died). Thus to balance the benefit and harm, we can ask how many thromboembolic strokes are equivalent to one death. Recent measurements suggest that the average quality of life after thromboembolic stroke is between 0.7 and 0.8 on a scale of 0 (death) to 1 (normal good health),15 and thus the ratio is about 4 to 1. This is the higher of the two lines for harm in figure 3. Thus when the risk of stroke is somewhere between 2% and 3% per year, the harm and benefit are about equal. For higher rates than this the benefits would start to outweigh the potential harm induced by anticoagulant treatment.
The precise threshold clearly depends on the relative value patients place on thromboembolic stroke compared with death. We have used an average value, but individual preferences may need to be considered.
(4) Predict patient's risk
To identify patients who should expect benefit to be greater than harm, we need to predict each patient's risk. This requires identification of the major risk factors, and ideally their joint estimation to establish risk--that is, multivariate risk prediction. The major requirement for such prognostic studies is that a large inception cohort has been followed up for a sufficient time to predict accurately the joint effects of the risk factors.16 This information may come from population based cohort studies or from the controlled trials themselves. However, because the eligibility criteria for trials often narrow the range of risk and the consent process may result in a somewhat different average risk, population based cohort studies are preferable.
EXAMPLE
Factors that influence the risk of stroke in atrial fibrillation include age and clinical and echocardiographic evidence of coexisting cardiovascular disease. The combined effect of these factors was examined by the Stroke Prevention in Atrial Fibrillation Investigators in a multivariate risk model, which suggested that three clinical features--hypertension, recent congestive cardiac failure, and previous thromboembolism--and two echocardiographic features--left ventricular dysfunction and atrial size--are important.17 18 The arrows in figure 3 show the resulting rates of thromboembolic stroke for the clinical features (bottom axis) and the combined clinical and echocardiographic features (top axis). For patients with none of the three clinical risk factors (no risk factors on bottom axis) benefit and harm are about equal and hence we should decide not to treat. This would include about 42% of the patients in the stroke prevention in atrial fibrillation trial. For patients with none of either of the clinical or echocardiographic risk factors (no risk factors on top axis) the benefit clearly does not outweigh the potential harm. This would still include about 26% of patients in the stroke prevention in atrial fibrillation trial, who, in retrospect, we would suggest were not gaining a net benefit, despite being eligible for a trial with a strongly positive outcome.
In a similar vein, we might look at the predictors of harm--that is, the risk of intracranial haemorrhage with warfarin. However, the strongest predictor is the degree of stability of the prothrombin time.19 Since this can be known only after warfarin treatment has been started we have not included it in the model, but in principle, predictors of harm can and should also be routinely incorporated.
Discussion
With current interest in evidence based medicine, more clinical decisions will be based on the results of clinical trials. A clinician deciding whether a trial's results apply to a particular patient should not focus on the inclusion and exclusion criteria of the trial--which are usually designed for improving the power of the study or maximising safety--but should try to predict whether each patient would benefit.3 As shown in figure 4, the decision entails piecing together three types of information about benefit and harm: the reduction in relative risk, the risk, and the relative valuation of the outcomes. Though our example has used only two outcomes--stroke and death--the equation of figure 4 generalises this to multiple outcomes.
This method suggests that the application of trial results need not be confined by eligibility criteria or the trial's setting. Primary care settings, when compared with secondary or tertiary referral centres, are likely to have a larger proportion of patients with few or no risk factors, who would therefore enjoy little or no net benefit. In making this decision, however, the patient's specific characteristics, rather than the setting, are important.
Some readers may prefer to recast this process using the number needed to treat,20 which is the inverse of the reduction in absolute risk, or 1/reduction in absolute risk. This would transform the vertical scale in figures 1 to 3, but the concepts and steps are unchanged.
We have examined only models in which outcome is the risk of an event. Clearly similar models could be developed for conditions in which severity is the major concern. In all cases benefit should be weighed against harm rather than relying on the eligibility criteria or setting of trials.
Acknowledgments
We thank Professor David Sackett for helpful comments.