Evidence based medicine: a movement in crisis?
BMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g3725 (Published 13 June 2014) Cite this as: BMJ 2014;348:g3725
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
By writing this article, Trisha Greenhalgh and colleagues showed us that EBM has got limitations. I agree. I think EBM started as an unconventional way of practicing medicine that brought us improvements in healthcare we could never have foreseen. However, some people nowadays look at it from a completely different angle. They do not look at it disapprovingly; it is more likely to be a lack of satisfaction. We should face the fact that EBM is prone to a lot of criticism and that this will never change. This does not imply, however, that we do not have the power to reduce weaknesses, nor that we cannot implement a better version of EBM.
To start with, I think we should change the education of future healthcare providers: doctors, dentists and nurses. They will have to shape and use the new EBM. Will they be able to do this without even knowing what EBM actually means? A man cannot become a farmer without having seen a single vegetable in his life, right?
I see a lot of similarities between the way EBM replaced the old-fashioned way of practicing medicine and the birth of Western philosophy. The first generation of philosophers, the Ionic Natural Philosophers, questioned the mythological explanations of environmental and biological phenomena. Nowadays we try to undermine myths and to confirm correct theories present in the world of diagnostics and treatments.
To be able to study the world around them these philosophers divided their philosophy in sub-fields. These included the logic, ethic, and physic. I think it could be successful to use stratification, inspired by this philosophical subdivision, as a backbone of education. We could use this trinity for categorization of the fields in which students need to be educated, for them to understand EBM, and to discover the pitfalls and its solutions.
In the field of the logic, students need to understand how we reason and approach clinical problem solving (e.g. Occam’s razor versus Hickam’s dictum), how the patient and the professional interact around healthcare decisions, and of course the quality of the evidence itself. The ethic is the study of the moral. What is good or bad behavior of patients and healthcare professionals when handling clinical problems? And in the physic we could redefine EBM and the value of basic science for our profession. We could learn to assess the applicability of the evidence. This is very relevant, especially in case of the elderly (suffering from comorbidity).
In summary we could philosophize about how to reduce the weaknesses physically, but also how to adapt to these weaknesses as a person or as a healthcare provider.
The Radboud Honours Programme Medical Sciences enabled a few students to analyze EBM in daily practice and under professional guidance. I got the chance to get educated about this hot topic, which made me start to think about solutions. Perhaps my trinity will not cover EBM completely, but at least I got inspired by it. I think it would be extremely useful (for the medical profession) to provide every student with the opportunity to get educated about EBM. If we do so, the students will become great farmers… excuse me… great EBM practicing Healthcare providers. They are the next generation; they will have to make and practice our new EBM.
Competing interests: No competing interests
Very timely and valuable input. EBM has drifted away from practicing physicians and real life clinical situations. We should be well aware of many issues raised in the article and continue to discuss, debate and know further.
Competing interests: No competing interests
The article clearly explains the dilemma of practising evidence based medicine.The basic issue for a practising doctor is that most of the evidence based literature is by researchers and pharma companies.The so called evidence is ambiguous contradictory most of the time with no take home messages or real answers.The literature in EBM appears to give more importance to statistical validity and correctness and after the reading the articles leaves us more confused than ever.
I tried to learn statistics and invested considerable amount of time in attending workshops on Biostatistics but I still fail to get practical information which I can apply to the patient and explain to the patient in simple language. The research in evidence based medicine should include more of practising doctors from all over the globe and of patient experiences, as suggested in the article. Qualitative research done on a large scale will definitely be more representative than the so called objective evidence from selected groups with selective parameters. There needs to be a system where regular feedback is taken from practising doctors and patients to improve the evidence.
Competing interests: No competing interests
I agree, evidence based medicine is a movement in crisis, and indeed EBM needs to be more usable for clinicians and patients as well.
In order to serve this goal, a more comprehensive approach to EBM is needed.
Therefore I should like to point out, that in the area of occupational therapy, physiotherapy and speech and language therapy a new model for establishing the value of evidence from multiple research approaches in a systematic review was prosposed.
By separating the evidence-level criteria of internal and external validity, by incorporating explicitly the evidence provided by qualitative studies, and by retaining the critical notion of rigor, a pyramidal evidence model emerges. This model, the Research Pyramid, aligns itself with the basic modes of clinical reasoning.
The Research Pyramid should yield a superior portrayal of evidence by providing quantitative-abstract and qualitative-dense and complex information about both efficacy and effectiveness of clinical interventions.
More information about the model ist presented in the attached figure and in the following publications:
[1] Tomlin, G., & Borgetto, B. (2011). Research Pyramid: A new evidence-based practice model for occupational therapy.
American Journal of Occupational Therapy, 65, 189–196. doi: 10.5014/ajot.2011.000828
[2] Pfingsten A, Trickes C, Max S, Borgetto B. Die Forschungspyramide: Ein Modell zur Bewertung der Evidenz durch Zusammenführung verschiedener Forschungsansätze in einem systematischen Review. pt_Zeitschrift für Physiotherapeuten 2011; 63:16-8.
Competing interests: No competing interests
I thank Trisha Greenhalgh and colleagues for the article: Evidence based medicine: a movement in crisis?(1).
I essentially agree with many of the comments made in the text of the article and I think the debate should continue to improve both teaching and practice. I conduct an advanced course in EBM, on behalf of Reggio Emilia Healthcare Library and Documentation Centre of the Clinical Governance Healthcare Trusts of the Province of Reggio Emilia (http://biblioteca.asmn.re.it/), which this year is at its 9th edition.
In Italy the degree programs of the health professionals do not include the study of the EBM method, and then when they begin to treat patients they have few tools to read and evaluate publications, therefore they do not have the basic EBM knowledge to apply the method and by any chance tackle the problems posed by the authors.
In my teaching I always had in mind the statements made in 1992 by the Evidence Based Medicine Working Group, and in particular:
“Evidence-based Medicine also involves applying traditional skills of medical training.”(2)
“Another traditional skill required of the evidence-based physician is a sensitivity to patient’s emotional needs.”(2)
But above all “Evidence-based medicine deals directly with the uncertainties of clinical medicine and has the potential for transforming the education and practice of the next generation of physicians.”(2)
Teaching EBM helps become aware of this uncertainty, and I believe that this kind of teaching increases the effectiveness and reduces the risks.
It is obvious that the teaching and the consequent practice must update, and just to address some of the objections of the authors we have added the Health Literacy to the topics of teaching.
I hope (but I’m sure) it is “a developmental crisis”.
Your paper will be used in the first lesson in my new course.
1. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? Bmj 2014;348.
2. Evidence Based Medicine Working Group. Evidence based medicine. A new approach to teaching the practice of medicine. JAMA 1992;268:2420-5.
Competing interests: No competing interests
It is the prevailing and too simplistic intepretation of the EBM concept that resulted in its "crisis" (as described by the authors). Inherently, EBM was never "against" tacit knowledge, experience, intuition or talent (diagnostic, therapeutic...), background basic science and pathophysiological rationale..it only insisted that "hard clinical outcome data" are added on top.
Whoever cares to read carefully understands that:
a) it is not "just" statistics..it is the quality (determined primarily by the level of protection from bias and random error) of evidence combined with clear-cut "numerical data";
b) the "statistics part" is never (only) about p-values...but about the size and direction of effects, their practicl relevance and reproducibility;
c) the result, the evidence, is the best possible (given the circumstances) estimate of the "population"...and just a starting point when facing an individual patient. A milestone.
The decision, however, needs to account for all of the above. This is, I believe, what was always meant by EBM. If we have lost this kind of understanding over the years - then the exposed agenda appears just the right one to help us get it back. I commend the authors.
Competing interests: No competing interests
Dear Editor,
I have data to support the hypothesis described in the title of this letter.
Before rejecting the null hypothesis I would like to ask the following open question:
Could you support with data that hierarchical levels of evidence based medicine are correct? (1,2)
Additional explanation to this question:
- Only respond to this question attaching publicly available raw data.
- Be aware that more than a question this is a challenge: I have data (i.e., evidence) which is contrary to classic (i.e., McMaster) or current (i.e., Oxford) hierarchical levels of evidence based medicine. An important part of this data (but not all) is publicly available.
References
1. Ramirez, Jorge H (2014): The EBM challenge. figshare.
http://dx.doi.org/10.6084/m9.figshare.1135873
2. http://chaoticpharmacology.wordpress.com/2014/08/13/the-ebm-challenge-da...
Competing interests: I endorse the principles of open data in human biomedical research
We follow with interest the discussion generated by Greenhalgh and colleagues’ thought-provoking article about the controversy surrounding evidence-based medicine (EBM).1 It brings to mind the metaphor of Scylla and Charybdis, referring to the dangers that Odysseus had to negotiate on his voyage, that can be used to describe the challenges of navigating between uncritical application of EBM and excessive EBM bashing. Do we choose to crash up against the rocky shoal of quality checklists, clinical guidelines and algorithmic rules in the name of evidence or be sucked down into the whirlpool of abandoning an approach that has gone a long way in improving rational clinical practice despite its obvious shortcomings?
We believe that the historical perspective provides valuable insights into this conundrum. EBM started more than 20 years ago as a “new paradigm” in response to the then prevailing practice of “eminence-based medicine” where it was assumed that physicians “always thought the right thoughts and did the right things.” 2 The pioneers of the original EBM movement emphasized “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” and further developed this definition to explicitly include patient values by stating that “evidence based medicine is the integration of best research evidence with clinical expertise and patient values”.3-4 Along the way, however, concern was raised about the slide of EBM down the slippery slope of “evidence-biased medicine,” akin to the proverbial salesman who single-mindedly peddles his wares instead of offering solutions based on the customer’s needs.5 The over-riding emphasis on reductionist positivist approaches to ensure research quality has resulted in the production of “simple” average evidence that is neither representative of nor extrapolated to the “complex” sicker and older patients in the real world, such that “what is actual is actual only for one time and only for one place.” 6 Evidence also suggests that patient values and preferences often differ from the “best (average) evidence” that preoccupies healthcare managers and policy makers. For instance, the willingness of older persons to take medications for primary prevention of cardiovascular disease is often less related to potential benefits than to potential adverse effects.7
Along with Greenhalgh and colleagues, we believe that the time is now ripe to usher in a new era of “evidence-balanced medicine.” While not amounting to a paradigm shift in the strict Kuhnian sense,8 evidence-balanced medicine nonetheless upholds the original ethos of EBM by reinstating patients as the raison-d’etre of EBM such that evidence serves patient care and not vice versa. Congruent with developments elsewhere,9 we feel that the call to return to the original spirit of EBM, rather than being misguided, neatly navigates between Scylla and Charybdis in this ongoing discourse by looking backwards in order to move forwards.10 Many years ago, Evans cryptically observed that “the name (of EBM) is crass but the idea is worthy.”5 Evidence-balanced medicine provides a way forward between Scylla and Charybdis if EBM is to truly live up to its promise.
References
1. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725.
2. Eddy DM. Evidence-based medicine: a unified approach. Health Affairs 2005; 24(1):9-17.
3. Sackett DL, Rosenberg WMC, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312(7023):71-72.
4. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based Medicine: How to Practice and Teach EBM, 2nd ed 2000. Edinburgh & New York: Churchill Livingstone
5. Evans JG. Evidence-based and evidence-biased medicine. Age Ageing 1995;24:461–463.
6. Eliot TS. Ash-Wednesday. London : Faber, 1930.
7. Fried TR, Tinetti ME, Towle V, O’Leary JR, Iannone L. Effects of benefits and harms on older persons’ willingness to take medication for primary cardiovascular prevention. Arch Intern Med. 2011;171(10):923-928.
8. Bergman MM. On concepts and paradigms in mixed methods research. Journal of Mixed Methods Research 2010;4(3):171.
9. Lim WS. More About the Focus on Outcomes Research in Medical Education. Acad Med 2013;88(8):1052.
10. Tovey, David, Rachel Churchill, and Lisa Bero. "Evidence based medicine: looking forward and building on what we have learnt." BMJ 349 (2014): g4508.
Competing interests: No competing interests
In a response to the analysis by the Evidence Based Medicine Renaissance Group,(1) we describe how Belgium is currently paving the way to achieve real EBM. This takes place through a national electronic point-of-care information service, EBMPracticeNet (www.ebmpracticenet.be), funded by the National Health Insurance Institute (INAMI-RIZIV).(2) The concept consists of three different components: an up-to-date database of validated, trustworthy guidelines delivered by scientific organisations, continuously available on a national E-Health platform and connected to every electronic health record (EHR) software package as one of the criteria for homologation. In combination with our approaches for patient empowerment, clinical training and multidisciplinary research in EBM, this project provides an interesting model for nation-wide implementation of EBM. The project builds on a close collaboration with national EBM organizations and with Duodecim, the scientific medical society of Finnish physicians. This organisation successfully introduced healthcare information technology in Finland where guidelines are consulted very frequently.(3) At this moment the focus is on general practitioners. In the second phase, there will also be a multidisciplinary focus on allied health personnel and specialist physicians.
Making evidence usable - Belgian healthcare professionals get free access to an up-to-date database of validated guidelines (60 from Belgian producers and nearly 1,000 from Duodecim), incorporated in a portal that also provides information from sources other than guidelines, including computerised clinical decision support, integrated in the EHR. The Duodecim guidelines are translated into Dutch and French and adapted to the Belgian context. The adaptation is done in collaboration with Belgian EBM organizations. In addition more than 50 General Practice trainees of the different Belgian universities participate in the adaptation process as part of their Master’s thesis.
The guidelines link to more than 4000 evidence summaries with a short graded description of systematic reviews or original research. Integration with the CEBAM Digital Library for Health (www.cdlh.be) enables the users to obtain the full text of citations in the large collection of scientific journals subscribed to by the Digital Library. This allows users to move efficiently from guidelines to systematic reviews and primary studies, and provides them with the ability to critically appraise the recommendations by checking the underlying original research.
Two semantic resources enable a link between the clinical data in the EHR and the recommendations in the guidelines: The CEBAM Evidence Linker and the Evidence-based Medicine electronic Decision Support (EBMeDS) system. The Evidence Linker captures the patient diagnosis (coded in the International Classification of Primary Care and the International Classification of Diseases), uses a link table between codes and available guidelines, and then connects the physician dealing with a patient to the relevant guidelines on request. The EBMeDS system was developed by Duodecim, and the United Kingdom NHS accredited its content development process.(4) EBMeDS receives structured patient data from EHRs and returns therapeutic suggestions for a broad spectrum of clinical topics. The Evidence Linker is available in every software package for general practitioners. The EBMeDS system is currently implemented on an experimental basis in two Belgian EHR packages.(5;6)
Setting the bar for trustworthy information - Prior to publication in the database, Belgian guidelines need to be formally validated by CEBAM. For non-Belgian guidelines a formal accreditation of the development procedures is needed before guidelines can be included. The validation and accreditation procedures are formal evaluations based on the AGREE criteria.(7) This formal evaluation must prevent the consultation of invalid guidelines, resulting in replacement of normal clinician variation by consistently inappropriate practice.(8) Conflict of interest policies are established for all the member organisations and contributors to EBMPracticeNet.
Patient empowerment - Plain language versions of the guidelines have recently (December 2013) been developed for a website for the public at large, called “Health and Science” (www.gezondheidenwetenschap.be). These patient versions of the guidelines aim to increase adherence to counselling and to empower patients in self-care. This project is funded by the Flemish government. Currently 120 patient versions of the guidelines are available. Through an online voting system patients participate in the selection of topics and a patient panel revises the patient guideline versions before publication. This project follows the health news in the Belgian media on a daily basis. It picks items from the headlines, and comments for the general public the evidence base of the news articles. A close collaboration has been established with the NHS Choices editorial board of Behind the Headlines (www.nhs.uk/news/Pages/about-behind-the-headlines.aspx) .
The patient versions of the guidelines are linked to the original caregiver guidelines. A caregiver who consults a guideline can provide the patient with the patient guideline. In a later stage both the caregiver and the patient can be provided with prompts from EBMeDS.
Reorienting clinical training - Medical education should prepare medical students to take up multiple roles as a health professional. This requires the integrated acquisition of multiple competences such as clinical reasoning and decision making, communication skills and management skills. To promote complex learning, instructional design focuses on the use of authentic, real-life learning tasks that students perform in a real or simulated task environment. As in every other university medical department, students from the KU Leuven learn the skills for searching evidence and critical appraisal. The new curriculum, based on a model for real-life learning, uses an EHR with an integrated computerised decision support system (Vandewaetere M, et al. 4C/ID in Medical Education: How to design an educational program based on whole-task learning: AMEE Guide No 93 Accepted for publication in Medical Teacher). The rules are tailored to the pathology learned at that stage of the education. The complexity of the expected competency grows as students progress in their career, and evolves from using the EHR in simulated, well supported environments to using the EHR in its full potential in clinical non-supervised settings.
Multidisciplinary research on EBM - The implementation of EBM in every day practice is clearly a complex intervention with multiple interacting components. It requires expertise from multiple research areas including clinical science, decision science, medical informatics and implementation science. To guide the implementation and to verify the impact on clinical practice a research agenda is necessary. Priority research topics are the introduction and use of computerised decision support systems to improve diabetes care,(6) laboratory test ordering and medical imaging. In this research we will use the Medical Research Council guidance on developing and evaluating complex interventions which contains four phases: 1. modelling phase, 2. exploratory trial, 3. definitive trial and 4. long-term implementation.(9)
EBMPracticeNet is currently a “work in progress”, but there is potential value in the project. A computerised decision support system links all the EHRs from different vendors with a national database held on a single platform and controlled by independent EBM organisations in Belgium. Usage rates are currently at a baseline level with approximately 700 unique users a month, opening 8,200 information items. We expect this usage rate to rise substantially once the promotion and integrations are fully launched. Although the challenges are numerous, we believe that EBMPracticeNet introduces a new era for evidence based medicine in Belgium.
References
1 Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725.
2 Van de Velde S, Vander Stichele R, Fauquert B, Geens S, Heselmans A, Ramaekers D, et al. EBMPracticeNet: A Bilingual National Electronic Point-Of-Care Project for Retrieval of Evidence-Based Clinical Guideline Information and Decision Support. JMIR Res Protoc 2013;2(2):e23.
3 Kortteisto T, Minna K, Kunnamo I, Nyberg P, Anna-Mari A, Pekka R. Self-reported use and clinical usefulness of second-generation decision support - a survey at the pilot sites for Evidence-Based Medicine electronic Decision Support (EBMeDS). FinJeHeW 2009;1(3):161-9.
4 Varonen H, Kortteisto T, Kaila M. What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians. Fam Pract 2008 Jun;25(3):162-7.
5 Heselmans A, Aertgeerts B, Donceel P, Geens S, Van de Velde S, Ramaekers D. Family physicians' perceptions and use of electronic clinical decision support during the first year of implementation. J Med Syst 2012;36(6):3677-84.
6 Heselmans A, Van de Velde S, Ramaekers D, Vander Stichele R, Aertgeerts B. Feasibility and impact of an evidence-based electronic decision support system for diabetes care in family medicine: protocol for a cluster randomized controlled trial. Implement Sci 2013;8(1):83.
7 Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol 2010;63(12):1308-11.
8 Thomson R, McElroy H, Sudlow M. Guidelines on anticoagulant treatment in atrial fibrillation in Great Britain: variation in content and implications for treatment. BMJ 1998;316(7130):509-13.
9 Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337:a1655.
Competing interests: BA is the founding father of this proof of concept, head of the KU Leuven Department of General Practice and one of the founders and first director of CEBAM. Finally, he is responsible for the implementation of EBM in the medical curriculum at the KU Leuven. SVDV is the salaried Project Leader and Editor-in-Chief of EBMPracticeNet; BA and SG are founding members of EBMPracticeNet. BF is president of the EBMPracticeNet board and is a salaried editor of EBMPracticeNet. RVS and AH are salaried editors for EBMPracticeNet. PV is the director of CEBAM; RVS and DR are members of the CEBAM board.
Re: Evidence based medicine: a movement in crisis?
Dear Editor
This is a timely article. I wish to point out that EBM also ignores evidence outside of the current thinking paradigm, which means that the evidence for what's been termed whole food plant based nutrition and the resulting health benefits for many of the common chronic disease states we spend huge amounts of money on has been ignored by the mainstream for the last 20-30 years. Ornish published the lifestyle heart trial in the Lancet in 1990 (1) yet patients are still not routinely informed about this choice for treatment of their heart disease. As a profession we should be asking ourselves why this is so. We should look for similar instances and at least inform ourselves and our patients whether or not we "believe". Multiple lives could be saved at little cost if we did.
1. The Lifestyle Heart Trial. Ornish et al. Lancet 1990: 336: 129-33.
Competing interests: No competing interests