Influence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentrationBMJ 1994; 309 doi: http://dx.doi.org/10.1136/bmj.309.6957.761 (Published 24 September 1994) Cite this as: BMJ 1994;309:761
- H C Bucher,
- M Weinbacher,
- K Gyr
- Correspondence to: Dr H C Bucher, McMaster University, Department of Clinical Epidemiology and Biostatistics, 1200 Main Street West, Hamilton, Ontario, Canada L8N 3Z5.
- Accepted 12 July 1994
Objective: To determine whether the reporting of study results by using reductions in relative or absolute risk and the number needed to treat affects the views of physicians about the effectiveness of drugs to lower lipid concentrations and decisions about treatment.
Design: Random allocation of two questionnaires presenting the results of three end points of the Helsinki heart study as results from separate trials by using reduction in either relative or absolute20risk. In both questionnaires one end point was also presented by showing person years of treatment20needed to prevent one myocardial infarction. The effectiveness of lipid lowering drugs was assessed for all end points on an 11 point scale. For each study result the likelihood to treat hypercholesterolaemia of 7.5 mmol/l in a healthy man had to be indicated on a seven point scale.
Subjects: Random sample of 802 internists and general practioners representative of providers of primary care in Switzerland. Results - The response rate was 69.6% (558). For the prevention of fatal and non-fatal myocardial infarction the mean ratings of effectiveness of lipid lowering drugs were 0.45 (95% confidence interval 0.21 to 0.69) and 1.39 (1.09 to 1.68) scale points lower when the reduction of absolute risk or number needed to treat were reported instead of the relative risk reduction (both P<0.001). Physicians receiving trial results for identical end points in form of absolute reduction of risk or number needed to treat were less inclined to treat hypercholesterolaemia (both P<0.001).
Conclusions: Physicians' views of the effectiveness of lipid lowering drugs and the decision to prescribe such drugs is affected by the predominant use of reduction of relative risk in trial reports and advertisements.
Study results of intervention procedures are commonly reported by referring to the relative risk reduction
Alternative measures to describe treatment effects are the absolute risk reduction and the number of people who need to be treated to prevent one adverse event (the reciprocal of the absolute risk reduction)
In this study physicians were randomly assigned to two questionnaires that gave the results from a primary prevention study for the treatment of high serum lipid concentrations as relative risks or absolute risks; they gave higher ratings for the effectiveness of the drug and were more inclined to prescribe lipid lowering drugs when relative risks were given
Both reduction in relative risk and reduction in absolute risk should be reported in medical papers because exclusive emphasis on the reduction in relative risk may overstate the effectiveness of a treatment
The description of effects of treatment by reduction in relative risks is the common way of reporting study results. Although technically accurate, such descriptions may operate subliminally to magnify the effect of an intervention for the reader. For example, a reduction in relative risk of 25% can be observed whether the absolute events observed in a given trial are 20% and 15% or 2% and 1.5%.
Alternative ways of reporting results from clinical trials are the reduction in absolute risk (also referred to as the attributable risk reduction) and the number of people who need to be treated to prevent one adverse event, which is the equivalent of the reciprocal of the absolute risk reduction.1 Several authors have emphasised that both measures - reductions in relative and absolute risk - should be reported to provide the reader with more precise information about the clinical benefit of an intervention.2 This is particularly important if the incidence of events is low as is often the case in primary prevention studies. Reporting relative risk, with the focus on the small proportion of people who had morbid events, neglects the large denominator of study participants having no profit from an intervention. The advantage of the reduction in absolute risk is the additional information of the consequences of giving no treatment.
We conducted a representative study in the German speaking part of Switzerland to examine the perceptions of physicians of the effectiveness of drug treatment to lower cholesterol concentration and their judgment as to when to start such treatment in primary prevention of coronary heart disease.
The investigation focused exclusively on drugs to lower cholesterol concentration because the prescription rate of these drugs has increased considerably in recent years in various countries.3 The study was restricted to primary care physicians because most decisions about the treatment of hypercholesterolaemia are made in primary care.
Subjects and methods Study population
A random sample of 802 internists and general practitioners was drawn from the computerised register of the Swiss Medical Association, which includes all licensed physicians. It was representative of the 4016 primary care physicians from the German speaking part of Switzerland according to the distribution of specialties, age, and time since graduation from medical school.
Displayed data in the questionnaires for the ratings of treatment effects were all derived from the Helsinki heart study,4 but physicians were blinded to this fact. The Helsinki heart study is a randomised controlled trial of men aged 40 to 55 with hypercholesterolaemia and without overt coronary heart disease. Subjects in this trial were assigned to treatment with gemfibrozil (n=2046) or placebo (n=2035). This trial was chosen because it was extensively cited in the Swiss recommendations for the prevention of coronary heart disease and treatment of high blood lipid concentrations.5
We developed two questionnaires on the treatment of hypercholesterolaemia (see appendix). Questionnaire A reported the effects of treatment as relative risk reductions, and questionnaire B reported the effects of treatment as absolute risk reductions. Physicians were randomly allocated to questionnaire A or B.
Both questionnaires showed treatment effects for three different end points: fatal and non-fatal myocardial infarction, fatal myocardial infarction, and total mortality. In addition, both questionnaires displayed the end point of fatal and non-fatal myocardial infarction as number needed to be treated to prevent one adverse event. Thus, for each participant it was possible to make one paired comparison of the given responses according to whether an end point was presented in the number needed to be treated and relative or absolute risk reduction. The effectiveness of drug treatment to lower cholesterol concentration was rated on an 11 point scale. Equally spaced numbers were displayed from −5 to 5. The right end of the scale was labelled “beneficial effect” and the left end “adverse effect.” The midpoint at 0 was labelled “no effect.”
In addition, after each end point the physicians rated the likelihood of whether according to the particular “trial result” they were inclined to start drug treatment in a 55 year old man with confirmed hypercholesterolaemia of 7.5 mmol/l and unsuccessful dietary measures. The threshold of 7.5 mmol/l was chosen because in Switzerland drug treatment is recommended for men without overt coronary heart disease and confirmed hypercholesterolaemia of 7.5 mmol/l and above. Rating had to be performed on a seven point equal distance scale. The left end was labelled “not inclined to treat” and the right end “inclined to treat.”
The questionnaires were pretested on residents at our hospital. In the pretest questionnaire we presented 95% confidence intervals for each end point. Because most residents were not familiar with confidence intervals we dropped this information in the final version and reported only whether the study result was significant or not.
Two mailings were carried out at an interval of four weeks to 802 physicians. Questionnaires were returned by 558 (69.6%); of these, 59 were not completed, leaving 499. Information available from the register showed that non-responding physicians did not differ from responders with regard to age and specialty (data not shown).
Data analyses were performed with SPSS-PC software.6 Differences in characteristics of respondents between the two randomised groups of physicians were tested with X2 and t tests. For summary estimates presented as relative or absolute risk reduction, unpaired analysis of variance was carried out with the Mann-Whitney test. Rating differences were also tested by two way repeated analysis of variance. Rating differences for the likelihood to start drug treatment were evaluated likewise. Paired analysis by the Wilcoxon signed rank test was used to compare the ratings for the same outcome displayed as number needed to be treated and relative and absolute risk reduction. Similarly, repeated ratings from the same physicians for the likelihood to start drug treatment to lower cholesterol concentration were analysed with the Wilcoxon signed rank test.
Characteristics of study respondents
Table I displays the characteristics of the respondent groups. Respondents receiving either form of questionnaire did not differ according to specialty, place of practice, time since graduation from medical school, and time devoted to reading medical journals.
Comparison of relative and absolute risk reduction
Table II and figure 1 show unpaired comparisons of ratings according to relative or absolute risk reduction.
FIG 1 - Mean ratings (95% confidence intervals) for effectiveness of drugs to lower cholesterol concentration according to presentation of study results for different outcomes in either relative or absolute risk reduction: unpaired comparison
**FIGURE NUMBER OMITTED**
Perceived effectiveness of lowering cholesterol concentration
For fatal and non-fatal myocardial infarction mean ratings of absolute risk reduction in comparison with relative risk reduction were 0.45 (95% confidence interval 0.21 to 0.69) scale points lower and for fatal myocardial infarction 1.15 (0.89 to 1.41) scale points lower (P<0.001 for both). For all cause mortality the ratings were in the opposite direction, but the difference of 0.19 on the rating scale was not significant. Thus, presentation of study results in terms of relative risks increased the perception of effects of treatment both in terms of benefit and harm.
Likelihood of starting treatment with drugs to lower lipid concentrations
Respondents with questionnaires that displayed relative risk reduction had higher mean ratings in favour of treatment in comparison with respondents with questionnaires that displayed absolute risk reduction (table II and fig 2). For fatal and non-fatal myocardial infarction the mean difference on the rating scale was 0.54 (0.20 to 0.87) and for fatal myocardial infarction alone 0.93 (0.60 to 1.26) (P<0.001 for both). For all cause mortality no difference in the rating was found.
FIG 2 - Mean ratings (95% confidence intervals) for effectiveness of prevention of fatal and non-fatal myocardial infarction by drugs to lower cholesterol concentration according to presentation of study results by relative or absolute risk reduction or number needed to be treated: paired comparison
**FIGURE NUMBER OMITTED**
Comparison of number needed to be treated with risk reduction
Perceived effectiveness of lowering cholesterol concentration - In questionnaires A and B the end point of fatal and non-fatal myocardial infarction was displayed twice by giving percentage risk reductions as well as the number that needed to be treated. For fatal and non-fatal myocardial infarction ratings of effectiveness were significantly higher when relative or absolute risk instead of the number that needed to be treated were reported (P<0.001 by Wilcoxon signed rank test for either form). The mean difference in ratings between the number that needed to be treated and relative risk reduction was 1.39 (1.09 to 1.68) scale points and between the number that needed to be treated and absolute risk reduction 0.70 (0.39 to 1.00) scale points (table II and fig 2). Thus presenting study data in terms of number that needed to be treated reduced the mean ratings of effectiveness by about one scale point.
Likelihood to start treatment with drugs to lower lipid concentrations - The ratings in favour of drug treatment were lower for both questionnaires when the number that needed to be treated was reported. This was by 0.92 (0.67 to 1.22) scale points with respect to the rating for relative risk reduction. As expected almost identical ratings for the likelihood to treat were given by physicians in both groups when the number that needed to be treated was presented.
Our study illustrates that the method of summarising results of trials of drugs for lowering cholesterol concentration influences the perception of physicians of the effectiveness of such a treatment and, in addition, leads to differences in their judgment about a treatment with drugs for lowering lipid concentrations. Ratings of the effectiveness of drugs to lower cholesterol concentration were lower when the absolute rather than the relative risk reduction was reported. An even larger shift towards a more conservative estimate of the effectiveness of such drugs was observed when the number needed to be treated was reported.
We believe that the most likely explanation of these findings is that the reporting of results in terms of absolute risk reduction and number needed to be treated offers physicians more concrete information about an intervention because it expresses efficacy in a way which incorporates both the magnitude of the reduction of risk and the baseline risk without treatment. Our study also shows that physicians who received the questionnaires with absolute risk reduction gave more conservative estimates on the likelihood of beginning drug treatment for lowering cholesterol concentration in primary prevention of coronary heart disease. No inference can be drawn from these data, however, on whether the observed differences in ratings reflect an important difference in the initiation of treatment.
Clinical decision making is certainly more complex than reflected in our questionnaire. Physicians have to integrate several other aspects in addition to the interpretation of the effectiveness of a given drug before starting treatment. To keep the questionnaire as simple as possible we chose not to develop a more elaborate study that would more adequately reflect the broad spectrum of clinical decision making for treatment to lower cholesterol concentration. Nevertheless, we believe that our study offers relevant information on the perception of physicians about the effectiveness of drugs to lower lipid concentration and decision making in the primary prevention of coronary heart disease.
Our findings are in accordance with the results from two similar studies which examined the perception of physicians of the effectiveness of cholesterol lowering and antihypertensive drugs. 7,8 Our study, however, offers some additional methodological strength because it was based on a representative sample. In addition, former studies did not clarify the relation between the perception of the effectiveness and the likelihood of initiating drug treatment.
This study highlights the necessity for medical literature to provide information from clinical trials in such a manner to eliminate inconsistencies in clinical decision making resulting from differences in the presentation of estimate of risk reduction. Because the exclusive reporting of relative risk may overstate the effectiveness of a treatment, actual event rates and absolute changes in risk should be reported. Improved training of physicians in critical appraisal of the medical literature and in techniques to tailor information from clinical trials into meaningful information for the management of individual patients is essential. 9,10
We believe that the need for the implementation of such policies is well grounded and is best illustrated with the example of drugs to lower cholesterol concentration. The usefulness of such drugs in primary prevention of hypercholesterolaemia has been questioned because several meta-analyses of trials have found an increase in non-cardiac mortality, 11,12 and overall mortality has not been affected by drug treatment. 13,14 In several European countries the prescription of such drugs has increased, 3,13 however, and some of the drugs to lower lipid concentrations rank among the five most sold drugs.15
As there are no definite data to resolve the controversy about cholesterol and the concern about excess deaths from non-cardiac causes in randomised control trials, the decision to start drug treatment depends on the physician's interpretation of the available evidence. Our results suggest that more has to be done to facilitate the interpretation of this evidence
We thank all the physicians who gave their time to participate in this study; our nurses and Christine Tovena, who helped us with the mailing and with controlling data; and Dr J G Schmidt and Dr C Boyle for reviewing the manuscript.
The benefit of cholesterol lowering drugs has been investigated in many trials. This questionnaire displays results from large randomised controlled trials. How do you interpret the study results of these trials with regard to effectiveness and side effects?
Questionnaire A on relative risk reduction
A cholesterol lowering drug treatment reduces the relative risk of a fatal and non-fatal myocardial infarction by 34%. This result is significant.
A cholesterol lowering drug treatment reduces the relative risk of a fatal myocardial infarction by 26%. This result is not significant.
A cholesterol lowering drug treatment increases the total mortality by 5.7%. This result is not significant.
During a cholesterol lowering drug treatment, 71 patients have to be treated for five years to prevent one fatal or non-fatal myocardial infarction. This result is significant.
Questionnaire B on absolute risk reduction
A cholesterol lowering drug treatment reduces the incidence of fatal and non-fatal myocardial infarction by 14 per 1000 patients and five years of treatment. This result is significant.
A cholesterol lowering drug treatment decreases the incidence of fatal myocardial infarction by one per 1000 patients and five years of treatment. This result is not significant.
A cholesterol lowering drug treatment increases the total mortality by 1.2 deaths per 1000 patients and five years of treatment. This result is not significant.
During a cholesterol lowering drug treatment 71 patients have to be treated for five years to prevent one fatal or non-fatal myocardial infarction. This result is significant.
Each question in questionnaires A and B was followed by a second question: According to this study result are you inclined to treat a 55 year old male patient with a confirmed hypercholesterolaemia of 7.5 mmol/l whose attempts to lower his cholesterol concentration through changes in his diet, have failed?