BMJ  2004;328:E280-E282 (13 March), doi:10.1136/bmj.328.7440.E280

BMJ USA: Editorial

Blowing the whistle on review articles

What should we know about the treatment of type 2 diabetes?

Ask the expert.

Whether we are trying to choose the best storm door at Home Depot or the best treatment for patients with type 2 diabetes, most of us turn to experts to help us make the decision, or, in some cases, to make the decision for us.¹ Experts are easy to use, and almost by definition are not subject to questioning. After all, they are "the experts."

In a recent analysis of review articles on the treatment of type 2 diabetes, we found that the experts writing them often did not tell us—the readers—the most important research evidence published in the past 25 years in the treatment of patients with type 2 diabetes.² It's time to explore the value of experts and examine our reliance—or even addiction—to the pronouncements of experts.

The evidence—the United Kingdom Prospective Diabetes Study

Started in 1977, the United Kingdom Prospective Diabetes Study (UKPDS) was designed to determine whether tight glycemic control decreases diabetes-related complications and increases life expectancy.^3,4 A second arm of the study investigated the role of tighter control of blood pressure in patients with both diabetes and hypertension.⁵

The investigators enrolled about 4000 patients with new-onset type 2 diabetes. These patients were assigned to receive either conventional treatment aimed at maintaining fasting blood glucose levels below 270 mg/dL or more intensive treatment with a goal of less than 110 mg/dL. Patients were studied for almost 11 years. Other than the University Group Diabetes Project, published in the 1970s, and a small study conducted in Japan, this is the only study to look at the long-term outcomes in patients with type 2 diabetes. Here are some of the most important results:

• Tight glucose control did not prevent premature mortality or significantly reduce any clinically important diabetes-related complications.
  
• In overweight patients, regardless of the degree of glucose control, treatment with metformin decreased mortality, both diabetes-related or due to any cause, and decreased diabetes-related complications.
  
• In overweight patients treated with either insulin or sulfonylureas, there was no effect on diabetes-related complications, regardless of the degree of glucose control.
  
• Tight blood pressure control decreased diabetes-related mortality and diabetes-related complications.
  
• Control of blood pressure produced a greater effect on complications than blood glucose control.
  

Why haven't you heard this evidence?

Don't be frustrated if these results are new to you. In our analysis of 35 review articles written at least two years after the publication of the UKPDS, we found that the authors of these reviews generally failed to point out these striking results, discounted them, or, in many cases, focused on other end points of the study that matter less.²

The main problem with experts is that they are, obviously, human, and thus full of biases and preconceived ideas.... Basing clinical practice on expert opinion is not necessarily harmful unless it contradicts evidence from valid POEMs.

For example, only 17% of 35 reviewers told readers that tight glucose control did not affect mortality, while 34 (97%) failed to mention the lack of proven benefit for tight glucose control in reducing clinically important diabetes-related complications. Only 7 (20%) of the 35 reviewers mentioned the role of metformin in decreasing mortality, and just 35% highlighted metformin's role in decreasing diabetes-related complications. Most—86%—did not tell readers the greater benefit of blood pressure control, even though it is touted as being the most important treatment of patients with type 2 diabetes (if their blood pressure is high).

Why experts go astray

There are several reasons, it appears, why reviews written by experts are not accurate. Foremost is the over-reliance on pathophysiologic reasoning rather than patient-oriented evidence that matters (POEMs).⁶ Patient-oriented evidence tells clinicians, directly and without the need for extrapolation, that a diagnostic, therapeutic, or preventive maneuver helps patients live longer or live better. This information matters when it requires a change in practice of a clinician.

In contrast, disease-oriented evidence is research focusing on either intermediate or surrogate outcomes.⁷ Numerous examples exist of medical practice based on disease-oriented evidence that have been shown to be not only ineffective but harmful (see table).⁸

View this table: [in this window] [in a new window]   Examples where patient-oriented evidence does not confirm disease-oriented (surrogate) end points8

 

Experts often reject new, valid POEMs when they don't make sense or conflict with disease-oriented thinking. This tendency has been called the tomato effect. For centuries, tomatoes were thought to be poisonous simply because they were related to the lethal plants belladonna and mandrake.⁹ Just as with pudding, though, the proof is in the eating.

Where to turn

The main problem with experts is that they are, obviously, human, and thus full of biases and preconceived ideas.¹⁰ Review articles written by experts are likely to express these biases. Basing clinical practice on expert opinion is not necessarily harmful unless it contradicts evidence from valid POEMs.

A new approach to writing review articles has been developed in which the reviews are prepared not unlike a research study. To perform these "systematic reviews," the authors collect all of the relevant research, and in a systematic way evaluate and summarize the current research in their review. Researchers qualified in the techniques of systematic review writing—not by experts in the field—write better reviews because they do not bring into the process a set of preconceptions and biases.¹¹

Clinicians wanting correct answers to the questions they encounter in practice should look for systematic reviews or meta-analyses (a type of systematic review that combines results from several studies into one large study for analysis). Another good source is evidence-based reviews listing key recommendations backed by acceptable levels of evidence ratings.¹² When reading an expert-based review, listening to a continuing education presentation, or talking with a local expert in the hallway, ask yourself the following question: "This information is interesting. How do I know it's true?" To paraphrase Will Rogers, if it's not based on a valid POEM, it ain't necessarily so.

Pinnacle Health System Harrisburg, PA Ashaughnessy{at}PinnacleHealth.org

Department of Family Medicine University of Virginia Charlottesville, VA

---

Competing interests: None declared.

References

1. Smith R. What clinical information do doctors need? BMJ 1996;313: 1062-1068.[Free Full Text]
2. Shaughnessy AF, Slawson DC. What happened to the valid POEMs? A survey of review articles on the treatment of type 2 diabetes. BMJ 2003;327: 266-269.[Abstract/Free Full Text]
3. UK Prospective Diabetes Study Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 1998;352: 837-853.[CrossRef][ISI][Medline]
4. UK Prospective Diabetes Study Group. Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34). Lancet 1998;352: 854-865.[CrossRef][ISI][Medline]
5. UK Prospective Diabetes Study Group. Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38. BMJ 1998;317: 703-713.[Abstract/Free Full Text]
6. Shaughnessy AF, Slawson DC, Bennett JH. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract 1994;39: 489-499.[ISI][Medline]
7. Temple R. Are surrogate markers adequate to assess cardiovascular disease drugs? JAMA 1999;282: 790-795.[Free Full Text]
8. Fleming TR, DeMets DL. Surrogate end points in clinical trials: Are we being mislead? Ann Intern Med 1996;125: 605-613.[Abstract/Free Full Text]
9. Goodwin JS, Goodwin JM. The tomato effect. Rejection of highly efficacious therapies. JAMA 1984;251: 2387-2390.[CrossRef][ISI][Medline]
10. Berger JO, Berry DA. Statistical analysis and the illusion of objectivity. Am Scientist 1988;76: 159-165.
11. Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci 1993;703: 125-133.[ISI][Medline]
12. Siwek J, Gourlay ML, Slawson DC, Shaughnessy AF. How to write an evidence-based clinical review article. Am Fam Physician 2002;65: 251-258.[ISI][Medline]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Rapid Responses:

Read all Rapid Responses

Evidence of absence or absence of evidence?

Matthew P Doogue
bmj.com, 12 Mar 2004 [Full text]

More on the UKPDS debate

Trisha Greenhalgh
bmj.com, 12 Mar 2004 [Full text]

Have we been all so absent-minded while reading the original UKPDS publications?

Peter Boucek
bmj.com, 19 Mar 2004 [Full text]

New acronym

Andrew D Martin
bmj.com, 19 Mar 2004 [Full text]