Seeing what you want to see in randomised controlled trials: versions and perversions of UKPDS data

Marketing the UK prospective diabetes study results

Until 1998, type 2 diabetes had been treated for over 25 years with drugs such as the sulphonylureas, insulin, and metformin. Only one well designed, prospective clinical trial had evaluated the effect of these drugs on the development of microvascular and macrovascular disease. This was the university group diabetes program study, the results of which created considerable controversy because they showed an increased risk of death from cardiovascular disease in the group that received sulphonylureas.1 Perhaps because of this controversy the results had little impact. The fact that the trial was never repeated, and that no further randomised controlled trials were published for another 25 years may surprise many clinicians. In September 1998, the long awaited results from the UK prospective diabetes study were presented in the BMJ, Lancet, and elsewhere.2 3 The 20 year study was conducted in 23 centres in the United Kingdom. More than 5000 patients with type 2 diabetes were recruited. The aim of the study was to determine the effect of intensive blood glucose control with sulphonylureas, insulin, or metformin on 21 predetermined clinical end points.

Despite some of the methodological limitations (the study was unblinded, the trial was continued when differences were not seen at the initial evaluation, and patients in the diet group received drug treatment if their fasting plasma glucose concentration was greater than 15 mmol/l), the papers have some important messages for clinicians and patients.4^–5 Indeed, it is unlikely that any trials in the near future will provide us with more information about the effect of glucose lowering drug treatment on the microvascular and macrovascular complications of type 2 diabetes. In general, the reporting of the results of the trial was positive. Reviewers stated that:

• Clear and consistent evidence now exists that hyperglycaemia in diabetes is a continuous, modifiable risk factor for clinically important outcomes and that reduction in glucose is the key to improving outcomes 6

• We now have convincing evidence that tight blood glucose control is an important goal for type 2 diabetes. Unless patients are seriously ill or have a short life expectancy, the long term benefits of intensive therapy clearly outweigh the few risks 6

• The main translatable finding is that intensive treatment of type 2 diabetes is beneficial.7

Despite these widely disseminated conclusions, scrutiny of the published data seems to show that the sulphonylureas and insulin have no impact on clinically important outcomes.2 3 In this paper we present the raw data and invite readers to come to their own conclusions and recommendations.

What did the data show?

Table 1 summarises the 10 year results of the UK prospective diabetes study 33,2 which evaluated drug treatment in 2505 non-obese and 1362 obese participants referred to hospital clinics with newly diagnosed type 2 diabetes. We have expressed the data as percentages rather than events per 1000 patient-years so that we can give absolute reductions and numbers needed to treat over a specific period. This allows comparison with the results of other trials that have been presented in this standardised way.8 We realise that there are advantages, disadvantages, and assumptions that have to be made when presenting the results either way.9

The primary outcome for these trials was a reduction in the number of patients with an aggregate of clinical end points (table 1) or diabetes related deaths. Over the 10 years of the study, there was a 3.2% absolute reduction in the occurence of one of the aggregated end points. Most of this benefit was due to a 2.7% absolute reduction in retinal photocoagulation, which was assessed by ophthalmologists independent of the study.

However, closer evaluation of the results showed that the use of glibenclamide, chlorpropamide, or insulin to lower blood glucose concentrations produced no significant benefit on any single macrovascular end point. A 2.4% absolute difference was seen for microvascular end points, and, again, most of the benefit was due to the reduction in retinal photocoagulation. Differences were detected in the surrogate end points of progression of retinopathy and albuminuria, but there were no differences in blindness, visual acuity, or renal failure.

Nevertheless, this trial has shown that the use of sulphonylureas probably does not increase the risk of death or serious disease events, which was a potential concern suggested by the results of the university group diabetes program study.1 It seems, therefore, that clinicians can be confident in prescribing these drugs to control the symptoms of hyperglycaemia in patients whose glucose concentrations are not adequately controlled by diet, exercise, and other oral drugs.

The UK prospective diabetes study 33 suggests that the drugs used were well tolerated, although only hypoglycaemic events and weight gain were reported. Nevertheless, participants in the sulphonylurea and insulin groups gained a mean of 3.1 kg more weight than the diet alone group. Major hypoglycaemic episodes (those requiring third party help) occurred in 0.1%, 0.6%, 0.6%, and 2.3% of participants per year in the diet, chlorpropamide, glibenclamide, and insulin groups respectively (note that benefit was expressed over 10 years). The incidence of minor hypoglycaemic events was 1%, 11%, 18%, and 37% per year, respectively.

In contrast to the above results, the UK prospective diabetes study 34, which focused on 1704 obese newly diagnosed type 2 diabetic patients, found several clinically important differences in macrovascular disease end points with 10 years of treatment with metformin (table 2).3 In particular, the absolute risk reduction for the aggregate end points was more than 10%, and for overall mortality was 7%, giving numbers needed to treat of 10 and 14 respectively over 10 years. Furthermore, in these patients, metformin was not associated with increased weight gain or hypoglycaemic episodes compared with diet alone. Metformin reduced progression of retinopathy compared with dietary advice alone, but there were no differences in other surrogate markers between the treatment groups.

Contrary to expectations, treatment with sulphonylureas and insulin had no significant benefit on the occurrence of microvascular or macrovascular end points over 10 years in this obese population (table 2). Metformin also produced significant reductions in the aggregated diabetes related end points, all cause mortality, and stroke compared with the sulphonylureas and insulin.

With regard to the results of these two trials, one message seems to have been lost from many of the commentaries on the UK prospective diabetes study. That is, patients with type 2 diabetes seem to benefit not so much from the overall control of glucose but rather from taking metformin.

The study also raises an interesting point about haemoglobin A_1c, which to our knowledge has not been discussed in any detail. Haemoglobin A_1c concentration has been used for some years as a surrogate marker. Although it is a good marker of overall blood glucose control, it is not known whether reducing the haemoglobin A_1c concentration in patients with type 2 diabetes leads to an improved outcome. To establish a causal relation between a surrogate marker and a clinical outcome, it must be shown that a dose-response relation exists—that is, that a consistent, progressive clinical benefit is seen with progressive reductions in the surrogate marker.10 In the UK prospective diabetes study, changes in haemoglobin A_1c produced by drug treatment did not seem to correlate with treatment outcomes.

In study 33 an absolute reduction of 1% in haemoglobin A1c concentration was observed with chlorpropamide, glibenclamide, or insulin over 10 years compared with diet alone; yet there was virtually no significant reduction in macrovascular outcomes.2 In study 34, all the drugs given (metformin, chlorpropamide, glibenclamide, insulin) produced similar mean absolute differences in haemoglobin A1c concentrations (about 0.6%) over the 10 years compared with diet alone, but only metformin produced significant reductions in clinically important macrovascular events.3 Not only did metformin reduce clinically important events compared with diet alone, it also produced reductions in some outcomes compared with other glucose lowering drugs. This shows that the studies in question were large enough, and of sufficient duration, to show macrovascular benefits. Clinicians and patients need to be aware of this and consider that either metformin may be conferring benefit independent of, or in addition to, blood glucose reduction, or that sulphonylureas and insulin may have an adverse effect on overall risk.11 Further analysis of the study's findings may shed more light on this question.

Who inserts “spin” and why?

In summary, in contrast to the positive spin about overall glucose control applied by many editorialists, the data show that sulphonylureas and insulin produced only a small (3.2% absolute difference) reduction in an aggregate of clinical end points. In addition, these drugs produced no significant benefit on individual macrovascular end points in non-obese and obese patients with type 2 diabetes and no benefit at all in obese patients. However, metformin, which provided a similar level of glucose control to that of sulphonylureas and insulin in obese patients with type 2 diabetes, produced important (5-10%) absolute reductions or delays in clinically important end points (death, strokes, and myocardial infarctions).

Why were the results from these studies presented with such a positive spin on tight blood glucose control when the results seem to show a benefit of metformin over sulphonylureas and insulin? Are we so reluctant to give up old beliefs? A similar spin was found with the captopril prevention project, in which captopril was compared with diuretics and β blockers for the treatment of hypertension.12 Although in general there were no differences in cardiovascular outcomes between the groups, patients taking diuretics and β blockers had a lower incidence of stroke (0.8% absolute difference) despite similar blood pressure reduction. If the reverse had been seen it is possible the researchers would have said something like “these results show that angiotensin converting enzyme inhibitors provide a unique benefit over other blood pressure lowering agents.” Instead, authors have tried to explain away the difference as being due to differences in baseline characteristics.

We believe that these cases illustrate the principle that interpretations of clinical trial results are often neither objective nor value-free. Rather, researchers, authors, and editors are highly susceptible to interpretive biases, including:

“We've shown something here” bias—that is, the researchers' enthusiasm for a positive result. It took 20 years to collect and analyse the UK prospective diabetes study data. To suggest that two of the three classes of drug used had little or no effect would have been a distinct anticlimax.

“The result we've all been waiting for” bias—that is, the clinical and scientific communities' prior expectations. It was widely believed in the 1980s and early ‘90s that the strict control of blood glucose concentrations was the raison d'être of the diabetologist and should be the principal objective of every well behaved patient.

“Just keep taking the tablets” bias—that is, the tendency of clinicians to overestimate the benefits and underestimate the harms of drug treatment. All the primary reports of the UK prospective diabetes study gave a relatively low emphasis to side effects (limited to hypoglycaemia and weight gain, with little discussion of the effect these had on patients and no mention of other adverse events). Side effects were presented as events per year, although the purported benefits were presented over 10 years.

“What the hell can we tell the public?” bias—that is, the political need for regular, high impact medical breakthroughs. Pressure from the press and patient support groups arguably drew staff from the British Diabetic Association, and perhaps even the trials' authors, into producing soundbites with a positive spin.

“If enough people say it, it becomes true” bias—that is, the subconscious tendency of reviewers and editorial committees to “back a winner.” The UK prospective diabetes study results were published in high quality, peer reviewed journals and were probably seen before publication by at least a dozen independent experts in either diabetes or research methodology. The writing—that the study was about to cause a sensation—was probably already on the wall, so it would have taken a brave and rebellious individual to be the first to jump off the bandwagon.

Looking back with the benefit of hindsight at how the UK prospective diabetes study results were presented and received at the time, we believe that this is a good example of the hidden biases inherent in the interpretation of randomised controlled trials. The relatively uncritical reception of the study by conference audiences, editorial committees, and the wider scientific community, could be an example of mass “groupthink”—a well described psychological phenomenon in which a group makes an overconfident and perhaps even irrational decision which it then defends fiercely against dissenting members, whose comments are subconsciously perceived as a threat to the group's own cohesion.13

We put it to the editors of medical journals that they should, in the interests of minimising interpretation bias, require investigators initially to present the results of clinical trials with a minimum of discussion so that individual clinicians and patients can decide if the results are clinically important. In addition, we suggest that editors should continue to provide space for readers to enter a discourse about the meaning and clinical importance of those results, and indeed they should actively stimulate discussion, perhaps by encouraging publication of dissenting views. Furthermore, when new evidence challenges old beliefs—let it.