BMJ  2007;334:956 (5 May), doi:10.1136/bmj.39202.421910.59

Views & reviews

From the frontline

Knowing what you don't know

The light flashed on my buzzer. I paused for dramatic effect before saying, "Alexander Fleming." The crowd of parents went wild with clapping. I beamed—we had won, and I had captained our team in the primary school quarter finals of the local "Top of the Form" quiz. I had the sort of memory that retained facts easily and that passes as a type of intelligence. I am like most doctors.

A decade ago I was an evidence based medicine (EBM) "fact" groupie. I loved the wild swirling data, the mind expanding NNTs (numbers needed to treat), and the mop top geeks who thumped out PowerPoint presentations and effectively smashed up the instruments of the old medical establishment. We got drunk on the power of EBM and how it would change the world. Occasional recreational use wasn't enough—I was hooked. Night after night my shadow was cast in the light of a VDU screen. I pored over the classics of Cochrane and Bandolier and then sank further into Medline and PubMed. Facts, facts, facts . . . numbers, absolute risks, subgroup analyses, intention to treat. I wanted a David Sackett poster for my bedroom wall. With practice I got pretty good, and that's when the problems started.

I skipped discussion sections and went straight to the data tables. I started seeing confounding factors throughout papers. I lost sleep to intrusive concerns over study populations, study length, publication bias, surrogate end points, and a whole new concern, "commissioning bias," which gives disproportionate weight to drug interventions: no research means no evidence.

But worst of all is the poor quality of the epidemiological foundation of all our facts. Simply put, the natural history of many conditions isn't known.

Take a current topic, the demand for screening for chlamydia. What is the lifetime incidence of chlamydia if the point prevalence is 10%? Could the current observed increase merely reflect more and better testing? In the 1970s, when condoms and sexual health services were less available, surely chlamydia must have been more prevalent? What percentage of infections progress to pelvic inflammatory disease? Without this knowledge how can anyone suggest screening? Basic questions remain unanswered in all aspects of our work.

By our very nature we doctors enjoy the comfort and power that facts bring, but unfortunately we apply them with an absolute certainty that they do not deserve. EBM runs the real risk of becoming just as restrictive and conservative as the medical establishment that it replaced. The future challenge is to establish robust prospective epidemiological data for common medical conditions and to focus on what we don't know rather what we think we know.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Rapid Responses:

Read all Rapid Responses

Which EBM?

Alfred P J Lake
bmj.com, 9 May 2007 [Full text]

the limitations of trading precise answers to the wrong questions fo approximate answers to the right questions

oscar,m jolobe
bmj.com, 10 May 2007 [Full text]

What is intelligence?

Hugh Mann
bmj.com, 17 Feb 2008 [Full text]