Evidence based medicine: what it is and what it isn't

BMJ 1996; 312 doi: http://dx.doi.org/10.1136/bmj.312.7023.71 (Published 13 January 1996)
Cite this as: BMJ 1996;312:71

Recent rapid responses

Rapid responses are electronic letters to the editor. They enable our users to debate issues raised in articles published on bmj.com. Although a selection of rapid responses will be included as edited readers' letters in the weekly print issue of the BMJ, their first appearance online means that they are published articles. If you need the url (web address) of an individual response, perhaps for citation purposes, simply click on the response headline and copy the url from the browser window.

Displaying 1-9 out of 9 published

A strange and quite unfounded view. Can the authors explain then, why empiricism and positivism have such strong roots in Austria, which is a traditionally Catholic country?

Competing interests: None declared

Viktor Weichbold, Staff Scientist

HSS Innsbruck

Click to like:

The rapid response of Abt H Mat Sain importantly reminds us of the dual pillars which David Sackett described as the foundation of Evidence Based Medicine. While there may be some argument about what constitutes the best external evidence, approaching this is infinitely easier than clearly defining the second pillar on which the concept of Evidence Based Medicine rests; what David Sackett called "individual clinical expertise". He describes this as reflecting the proficiency and judgment that physicians acquire through clinical practice.

There is a qualitative aspect to these attributes, but we have not articulated criteria that might help us to decide who is to be relied upon to provide the soundest expert clinical judgement. Because of this (and some other factors), Evidence Based Medicine is in danger of becoming the very thing it sought to avoid. That is to escape from recommendations based on authority.

Today, all too often, those whose opinions are sought are experts by virtue of their elevation to this rank by the marketing departments of drug manufacturers. These might be the "key opinion leaders" or KOLs whose expert opinions cannot be regraded as free from influence by evident conflicts of interest. Promotional material from educational entities funded by industry, often tell us that their recommendations are evidence based. But it is not beyond conjecture that an expert has been created expressly to justify such a claim.

We may have no way to assess the proficiency of the expert whose opinion is sought. An association with an academic institution is really not enough and a long list of publications may also be irrelevant, but the number of years in clinical practice might be helpful. As long as what constitutes individual clinical expertise remains unquestioned and poorly defined, clinical recommendations may indeed be made by the kind of authority that Evidence Based Medicine sought to avoid.

Experts on various clinical guidelines committees in the US declare conflicts of interest, usually related to receiving payments from drug manufacturers. They provide the clinical expertise, that together with a consideration of external evidence justifies the claim that the recommendations they help to formulate are evidence based. We are far from assured that these individuals necessarily have the requisite clinical experience to provide the clinical judgment that David Sackett stated was necessary to be intergrated with the best external evidence.

In this way, there is a danger that the current application of Evidence Based Medicine may succeed in creating the very condtion it was meant to avoid.

Competing interests: None declared

Competing interests: None declared

Joseph A Sonnabend, retired physician

London NW8 9UG

Click to like:

An evidence-based medicine that prizes meta-analyses and randomised- controlled trials can help establish the most appropriate treatment for large populations with the same clearly defined clinical condition. However, such treatments will inevitably fail some patients whilst inferior comparator groups will not completely lack success.

Individuals are not populations. Symptoms might not be singularly ascribed to clearly-defined clinical conditions. Many aspects of the doctor-patient relationship have powerful therapeutic impacts in their own right. The treatment with the best evidence-base is useless for a particular patient if it fails them and, in the management of acute symptoms, must be replaced by plan B or beyond. Such ideas typically confound contemporary EBM. Some medical disciplines require an EBM that understands the uniqueness of each situation; accommodates a bit more diagnostic uncertainty; exploits the importance of the doctor-patient relationship; and suggests other options when the “best” choice fails. This potentially legitimises academic conjecture and therapeutic empiricism. It even keeps the door open to such things as case histories. Whilst contemporary EBM is vitally important, so is much of the material that it deliberately excludes. This might help explain the diversity of publication types found in most thriving medical journals.

Competing interests: None declared

Competing interests: None declared

John C Chambers, Macmillan Consultant and Medical Director

Katharine House Hospice, East End, Adderbury, Oxon, OX17 3NL

Click to like:

Are not the criteria here rather selective?

Competing interests: None declared

Competing interests: None declared

John D Stone, None

London N22

Click to like:

Evidence-based Medicine: Why in Genuinely Protestant Countries?

In a fundamental contribution on how to practise and teach Evidence- based Medicine (EbM) Sackett and his co-authors trace EbM back to mid- nineteenth century France. Is that view valid? Is medical theory as established by Bichat (1771-1802), Magendie (1783-1855), and Louis (1787- 1872) really the major root of EbM? Do the three represent a homogeneous school of medical thinking? And why has France never since been a center of EbM?

These questions call for an archeological search. The mere reference to medical thinking in Paris cannot explain the rise of EbM in late twentieth century in the face of a gap of more than 150 years. Additionally, it has to be stated that Bichat and Magendie as the founders of modern histology and experimental physiology respectively stand in another epistemiological tradition than the founder of the “numerical method”, Louis. Granted that the call for external evidence is a necessary condition for EbM, this, however, cannot be held as a sufficient property to differentiate it from conventional medicine. Relying on external evidence provided by pathology and even epidemiology is also essential to conventional medicine. Thus, it is not so much the quest for external evidence, but rather the ways of both detecting respectively creating and handling evidence that genuinely characterises EbM. So if we look at the countries and the universities that cherished EbM right from the beginning in the early 90s of the last century, it is mainly North America, the UK, Scandinavia and the Netherlands. Thereupon it seems more than likely that EbM, albeit also minted by empiricist thinking, can be figured as an offspring of Protestant exegesis practised in the aforementioned countries.

EbM could only thrive in Protestant countries as it obviously applies principles of “scientific” exegesis to the growing body of medical “tradition”. As Catholicism could base decisions on conflicting traditions on a dogma-based principle of the “Pre” of the eminence, Protestant theology lacking a single authority to settle theologic disputes had to develop other solutions to come to terms with contradicting passages from the scriptures and to clarify the relationship of the scriptures to other sources of tradition such as the Fathers of the Church. Dogma or mere negligence of whole parts of the Bible could not convince the believers any more. Thus, scientific ways of reading the scriptures had to be developed. Modern text criticism, literary criticism, for instance, emerged, and methods evolved how to weigh different sources of tradition. Researchers developed elaborate tools to dissect texts in order to come to terms with contradictions. Yet, in order to avoid the danger of being entangled in a mingle of loose ends of deconstructed texts, some kind of hierarchy had to be reintroduced, some kind of principle that could guide exegesis. The search for the canon in the canon began as a hermeneutical exegetical challenge. Thus, even within the Scriptures texts were sorted out into different levels of “importance” in a kind of similar way as “tradition” of medical research is assorted to grades of evidence.

This, however, does not suffice as an explanation for the rise of EbM in certain countries since Germany as one of the centers of Protestant exegesis hasn’t intensively been promoting EbM. As a second condition sine qua non serves a democratically shaped relationship among physicians since the new methodology does not rely on the “Pre” of hierarchical eminence and age per se, but dwells on the honest and transparent search for the actual and best “substantiated” knowledge. This again is a Protestant notion: Every lay person is entitled to read the Bible as any cleric is.

Thus, we are convinced that EbM is deeply indebted to or even embedded in the grounds of historical critical exegesis arising from the Protestant way of reading the Bible: 1. Everybody has got access to the „text“ without the preconditions of a certain position. 2. Inconsistencies are neither smoothed out by dogma nor by negligence nor by dictum of a final authority, but 3. review and appraisal follow criteria that are never absolutely set. These criteria are constantly struggled for in a hermeneutical process such as the refinement of EbM- grades.

In conclusion, the merits of EbM are not so much that it favors just one form of methodology as the gold standard such as randomised control trials, but its eminently new and challenging property is its kind of “reformatory” way of reading the tradition according to principles that everybody could intellectually follow. EbM both in its historical setting of late twentieth century and in its self-conception as a method of dealing with tradition is rooted in Protestant exegesis. Thus, it represents a hermeneutical turn in medicine. Furthermore, both the prerequisites and the consequences of EbM are in so far identical as a democratically shaped interphysician relationship is supportive to EbM and at the sime time evolves out of EbM.

Competing interests: None declared

Competing interests: None declared

Andreas U Gerber, research fellow

Markus Luengen, Karl W. Lauterbach

IGKE, U of Cologne, Gleueler Str. 176-178, 50935 Koeln, Germany

Click to like:

Sir

The unprecendented capability of the open-access online publication such as this in the British Medical Journal heralds a new interactive forms of fora and publications not experienced before the introduction of the Internet technology. The almost unlimited storage capacity of the cyberspace in handling huge amount of data, textual and images make rooms and spaces in the library rather redundant. Old articles published years ago can be retrieved anytime anywhere as need arises. Issues that are relevant now as they were before could be appraised and re-appraised expeditiously without limits. Articles written years ago could be retrieved and interactive feedbacks given anew based on fresh new ideas and insights.

Evidence-Based Medicine (EBM) is one such issue. The apparent moratorial paper published by the authors in 1996 seems relevant now as it was in the past (1). The long list of associated papers and books on the same homepage indicates the magnitude of importance given to the issue of Evidence-Based Medicine by many researchers, writers and practitioners alike. However, despite the enormous numbers of written books and articles on EBM, much debates, misunderstanding and confusions still abound. The degree of positive or negative passionate association with and pleas for EBM varies between different authors. Much confusions and misunderstandings arise from the extremely passionate argument to imbibe EBM in clinical appraisal and practice.

The other major flaw in the exhortation of subscribing to EBM is the overemphasis of Randomized Controlled Trials (RCT) as the panaceal standard bearer of evidence within EBM argument. This line of debate seems to overlook the dual components of EBM as vividly described by the authors of this article, namely the internal evidence and the external evidence (1). The explicit codified data of any RCT or even the meta-analyses of the RCTs hailed as the Holy-Grail of therapeutic clinical evidence by the extreme protagonists only addresses the external evidence within the whole concept/philosophy of EBM.

The internal clinical evidence is depicted by the individual clinician expertise and experience. This concept of clinician’s expertise and experience contributing to the EBM is a very important conceptual admission on the background of enormous amount of clinical information at the clinician’s disposal to use. Furthermore, this idea would relate EBM to the whole realm of clinical training and education. With the rapid growth of scientific knowledge and the avid application of technology in current clinical practice, advanced and ongoing clinical training and education need serious attention to ensure that practicing clinicians are fully equipped with the current internal clinical evidence.

It seems that the development of internal clinical evidence is a much more intensive undertakings from the perspectives of both capital and labour. This evidence is developed and based on the dynamism and plasticity of human intellect. The tacit information and knowledge acquired by clinicians through years of training and apprenticeship is hence given due credence in the whole context EBM. The ultimate utility of the codified external clinical evidence only depends on the appropriate appraisal by the internal clinical evidence.

It is hoped that clinicians will not anymore be skewedly exhorted to only use level I evidence from the scale of external evidence to base their responsible clinical decisions (2). The title of this article is an example of the wayward direction of the debates on EBM(2). The whole discussion about evidence is for its appropriate utility in the practice of Medicine rather than applied to the wider enterprise of exploration and experimentation of science.

It is also hoped that expert technical surgeons will not be inappropriately overtly ridiculed by the passionate proponents of external evidence of EBM for their active exploration of the therapeutic horizon in using new technology to effect state-of-the-art interventional therapy especially on uncommon diseases(3).

References

1. David L Sackett, William M C Rosenberg, J A Muir Gray, R Brian Haynes, and W Scott Richardson Evidence based medicine: what it is and what it isn't BMJ, Jan 1996; 312: 71 - 72.

2. J. Arya, H. Wolford, and A. H. Harken Evidence-Based Science: A Worthwhile Mode of Surgical Inquiry Arch Surg, November 1, 2002; 137(11): 1301 - 1303.

3. Horton R. Surgical research or comic opera: questions, but few answers. Lancet 1996;347:984-5.

Competing interests: None declared

Competing interests: None declared

Abd H Mat Sain, Deputy Director

Advanced Medical and Dental Institute, Universiti Sains Malaysia, 11800 Penang, Malaysia

Click to like:

5 December 2000

In a previous e-response [1], Peter Morrell “ask(s) Joseph Watine to provide the evidence and the observations for the existence of EBM and colo-rectal cancer”.

May I suggest Peter Morrell that he would better ask one of the thousands of patients who are currently suffering from colo-rectal cancer if they prefer to be treated using current medical knowledge (i.e. EBM) or if they prefer to be treated by homeopathy or by any other religious belief? When he has got the answer to this question, Peter Morrell might perhaps compare the survival of the patients who were treated using the EBM approach with that of the patients who were treated using the homeopathic or religious approach.

This might provide us with a very interesting and useful study.

[1] http://www.bmj.com/cgi/eletters?lookup=by_date&days=1#312/7023/71/EL2

Competing interests: None declared

Joseph Watine, Eur Clin Chem

France

Click to like:

5 December 2000

Sir,

In a previous email [1] I am rightly accused by Joseph Watine of calling EBM little more than a religious belief. This viewpoint can be approached from two directions in order to show that it might be valid. As everybody knows, anything that cannot be seen, felt, touched or detected in some way can only be assumed to exist. Science quite strictly demands that only those things that can be proven or demonstrated, are permitted to enjoy the status of being real. Those conceptual or abstract things we regularly assume to exist fall subject to similar requirements, and if they fail this test then they are not considered to be real, but merely convenient assumptions or beliefs.

In religion, such things as God and soul are certainly beliefs of this type, because they cannot be unequivocally demonstrated. Religions, therefore, might be distinguished from science in not enforcing this requirement that the claims of beliefs must be demonstrable. The ‘truths’ of religions, like the arts and humanities, tend to be evaluated in our life experience rather than through the material proof of hard science. Such beliefs tend to be measured according to how happy they make you feel rather than whether they can be proved.

In certain forms of medicine, we find conceptual entities that are believed to exist. Thus, the vital force and potency energy in homeopathy, the Qi and meridians in acupuncture, the toxins in nature cure and the subluxations or joint misalignments in osteopathy - these are all conventionally thought to fall into the same category of pure beliefs. However, in every case there exists a body of evidence, within each specialty, that confirms the conceptual entity, underpins it and validates a continued belief in it. Otherwise, clearly, such beliefs would have died out long ago. While this does not actually conclusively prove the existence of such entities, as science demands, it is sufficient to maintain an ongoing practical acceptance that such entities are there - they can be accepted as ‘working hypotheses’ and valid models.

Even in science itself similar concepts abound - things like light- year, electron, force field, proton, quark, etc dwell in a similar borderland of being assumed to exist but not being solidly proven. At the practical level, they can be mathematically or inferentially demonstrated but as no-one can see or touch them, they can never be completely proven as real. Science is not composed solely of ‘facts’ but also contains a surprising range of such unproven conceptual entities that add to the smooth conceptual fabric of the discipline. Whether that fact in any way influences the philosophical certainty of science is an open question. As with vital force and toxins [etc], there is a mass of evidence within science to sustain a continued belief in these concepts, but beliefs they certainly are. Thus, when we come to look at EBM, we must demand of it some definitions and observations that support the provisional conceptual ‘reality’ that people choose to invest in it. If such evidence cannot be adduced, clearly and for all to see, then we are correct to conclude that it is no more real than the fairies. I contend that there is no such evidence and thus EBM is just a transient fashion in medicine, an invention and a conception idly plucked from the air, which some people have latched onto as a new saviour in medicine.

The second approach involves tracking those 'conceptual errors' that collectively lead one to suppose that EBM is a real entity. These 'conceptual errors' are those through which modern medicine views the organism: what we might call ‘conceptual spectacles’ through which it habitually and unwittingly views the functioning of the organism, and with which it explains life phenomena and how it arrives at 'medical truth'. If it can be demonstrated that these ‘conceptual spectacles’ only lead to a certain viewpoint, which, upon closer inspection, does not accord with the actual functioning of the human organism, then clearly one is then entitled to conclude that it is only a partial picture, rather than the full truth. Scientific hypotheses cannot stand supreme over observations; those that conflict with data must be amended or abandoned - in science, the real world is king.

Having chosen to follow a certain track since the 1850s, the conceptual fabric of modern medicine inevitably faces a future crisis. Any sober analysis of its approach to the organism most certainly does reveal that it possesses conceptual spectacles and only views organism functioning through them. If it were willing to change those spectacles, then of course it could obtain very different views of the organism and hence very different ideas of what medicine is about. It is precisely in an arena of mature well-reasoned debate, tolerant respect and philosophical inquiry, that the various claims and evidences of the different healing modalities, should be brought and evaluated patiently, neutrally and without prejudice. If such a task is ever undertaken then it will be possible to show that no single medical approach can claim superiority over any other. All have some claims on 'medical truth' and all can be shown to have some clinical validity and some understanding of how the organism functions. However, seemingly such a time is not yet upon us and we ‘see but through a glass darkly’.

In fact, the same 'conceptual errors' that lead one to a belief in EBM, also lead to a belief in colo-rectal cancer as a real entity. I therefore ask Joseph Watine to provide the evidence and the observations for the existence of EBM and colo-rectal cancer. If he can provide this evidence then we can accept that these things are real. If he cannot, then it will be clear that they are merely beliefs or conceptual fabrications visible only when viewed through certain conceptual spectacles.

Avoiding any rash, simplistic or literal interpretation, he should carefully consider the fundamental conceptual error contained in the following statement: “we need to find proper cross sectional studies of patients clinically suspected of harbouring the relevant disorder.” [2]. This phrase reveals the source of 'conceptual errors'. Implicitly, I shall be assuming that this article by Sackett et al is a genuine, lucid and definitive attempt to summarise what EBM is.

Sources

[1] BMJ letter, Joseph Watine, 2 December 2000, Is EBM a belief?

[2] Sackett DL, Rosenberg WMC, Gray JAM, et al. Evidence based medicine: what it is and what it isn't. BMJ 1996; 312: 71-72 http://www.bmj.com/cgi/content/full/312/7023/71

Competing interests: None declared

Peter Morrell, Hon Research Associate, History of Medicine

Staffordshire University

Click to like:

Editor - Evidence-based medicine has emerged as a powerful problem- oriented approach to the practice of medicine that seeks to improve patient care by considering the quality of clinical evidence. Evidence based medicine is founded upon an ideal that decisions about the care of individual patients should involve the "conscientious, explicit and judicious use of current best evidence." (1)

From an ethical perspective, the strongest arguments in support of evidence based medicine are that it allows the best evaluated methods of health care, useless or harmful methods to be identified and enables patients and doctors to make best decisions. Medical decision making draws upon a broad spectrum of knowledge including scientific evidence, individual scientific medical education, personal experience, personal biases and values, economic and political considerations, and philosophical and social principles ; it may be not always clear how practitioners integrate these factors into a final decision, but it seems unlikely that medicine can ever be free of personal and intuitive judgments. For this reasons evidence based medicine recently has undergone to a cross fire of criticisms, where it is sometimes depicted as a method to improve more statystical analysis of pathologic cases, than clinical decision making to get the best treatment for a single patient.

While clinicians are exhorted to use up to date research evidence to give patients the best possible care, actually doing so in individual patients is difficult: at the heart of clinical medicine it is an unresolved conflict between the essentially case based nature of clinical practice and the mainly population based nature of the research evidence, and this is particularly truthful for phytotherapy. In phytotherapy besides usual problematics of medical research are added the specific issues of the vegetal extract: its qualitative and quantitative variability, different types of extraction, phenotipic and genotipic variabilities of vegetal drugs, different methods of growth, time of harvesting and preservation, gographical and climatic differences (2).

We believe that in the field of phytotherapy research, evidence based medicine is not only an unvaluable tool , but the sine qua non to verify efficacy , tollerability and safety of a herb extract. Metanalysis on Hypericum perforatum (St. John's Wort) (3) , a drug used for depression by millions of patients in all over the world, and for a long time considered just a placebo, recently thanks to important metanalysis it is now considered a valuable tool for the treatment of mild and medium depression. In the same way metanalysis on Allium sativum (Garlic) 4); Serenoa Repens (Saw palmetto) (5) and Menthae piperitae oleum (Peppermint oil) (6) confirmed the pharmacological activities of these herbal extracts.

Besides, evidence based phytotherapy is an unique tool to evaluate herbal drugs in a field where tradition and popular empirism is still very strong. The development of research on the basis of evidence medicine is an important international priority in response to the public's growing use of complementary therapeutic interventions and recent emergence of drug interactions; and it has to be conducted following parameters characteristic and proper of phytotherapy.

Bibliography

1. Sackett DL, Rosenberg WMC, Gray JAM, et al.: Evidence based medicine: what it is and what it isn't. BMJ 1996; 312: 71-72.

2. Firenzuoli F, Gori L.: Evidence-based phytotherapy .Recenti Prog Med, Nov 1999, 90(11) 628.

3. Stevinson C; Ernst E .:Hypericum for depression. An update of the clinical evidence. Eur Neuropsychopharmacol 1999 Dec;9(6):501-5 .

4. Neil HA; Silagy CA; Lancaster T ;et al.: Garlic powder in the treatment of moderate hyperlipidaemia: a controlled trial and meta- analysis.J R Coll Physicians Lond 1996 Jul-Aug;30(4):329-34.

5. Boyle P; Robertson C; Lowe F ,et al.:Meta-analysis of clinical trials of Permixon in the treatment of symptomatic benign prostatic hyperplasia. Urology 2000 Apr;55(4):533-5393.

6.Pittler MH; Ernst E .Peppermint oil for irritable bowel syndrome: a critical review and meta-analysis. Am J Gastroenterol 1998 Jul;93(7):1131- 5.

Firenzuoli F, Gori L.
Service of Phytotherapy - St. Joseph Hospital, Via Paladini 15 - 50053 Empoli (Florence) - ITALY

Competing interests: None declared

F Firenzuoli

Click to like:

THIS WEEK'S POLL