The illusion of evidence based medicine
BMJ 2022; 376 doi: https://doi.org/10.1136/bmj.o702 (Published 16 March 2022) Cite this as: BMJ 2022;376:o702
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor
Our article cited below made the same points about the reliance on commercialised research by health and education departments, broader institutions and government agencies. There is also a lack of consensus and transparency about what constitutes 'evidence' in the first instance. By brandishing the term itself - evidence based research - it is broadly assumed to be independent, of high quality and peer reviewed. There has been a rapid erosion of professional evaluation of the research 'product' and its potential applications. Arguably the term 'evidence based research' trades on the public confidence that the 'evidence' results from properly conducted research. This is not supported by adequate funding and direction for non-profit research endeavours.
This article: Lena Rodriguez, James R. George & Brent McDonald (2017) An inconvenient truth: why evidence-based policies on obesity are failing Māori, Pasifika and the Anglo working class, Kōtuitui: New Zealand Journal of Social Sciences Online, 12:2, 192-204, DOI: 10.1080/1177083X.2017.1363059 To link to this article: https://doi.org/10.1080/1177083X.2017.1363059
Competing interests: No competing interests
Dear Editor
While I applaud and agree with this opinion it seems to me short sighted. This problem of greed and corruption is greater than the medical aspect alone.
The growth of modern (8/9 decades) of non-contagious diseases seems to be dependent on the changes made to food intake and especially materials passed off as food, that have no natural occurrence, cannot be properly metabolised or cause hideous responses to our physiology. Added to this are engineering materials that while on the surface are seen as helpful and convenient to daily life, are now being recognised as a man-made ubiquitous blight on the planet (forever chemicals, heavy metals, plastics etc), which also undermine human life. This poisoning of the human race must stop.
Scientists of honour and conscientiousness are needed to monitor every aspect of man's technological progress in an honest, truthful and unbiased manner, paid sufficiently well yet moderate and funded by public funds administered by an honourable science based foundation. These funds should be heavy tax levies on the manufacturers of the products brought to market. This scientific administration should be open and unsullied by the corruption of financial interests. Therefore there should be no outside Governmental, pharmaceutical, institution, corporation or financial influence on how the system works or is funded other than to openly help regulate its effectiveness and only those that have proved to be responsible enough to undertake such positions should be recruited, which at the present time obviously excludes those previously mentioned.
Products based on evidential peer reviewed research that truly prove to be suitable for human consumption should be allowed to market based on honest findings. Foods and food handling materials that don't meet these criteria should not be allowed. In a stroke, human health would IMHO be considerably improved thereby reducing the medical load on our healthcare systems, the medication requirements and the real heath and safety of our peoples would automatically recover. The medical community can get back to finding solutions to real problems and not wasting time on the greedy man-made maladies. Doctors will have time for their patients again and the NHS will save a fortune and finally public trust may eventually return.
The subject is too large for a greater degree of consideration and I present it to you as a constructive opinion.
Vic Ryder
Competing interests: No competing interests
Dear Editor
Conflicts of interest drive distortions. The conflict here seems to lie between the pursuit of scientific truth and self or corporate interest. The former goal is admirable and, in the Karl Popper sense of the production of empirically grounded propositions should be encouraged.
However, the distortion of direction of development, deployment or description by those who wish to promote approaches that are self or corporate interested is corrosive to the former goal.
Use of resource power to pursue such distortions does indeed undermine the project. As the sea corrodes the legs of an oceanic oil rig unseen this process will in time destroy the entire edifice. This article is a timely warning that should be heeded, if it is not already too late.
Competing interests: Clinical work, clinical and other research. None direct.
Dear Editor
The limitations described in the BMJ Opinion “The illusion of evidence based medicine” (EBM) (1) are an allusion to chaos and complexity in EBM, suggesting the need for a chaos and complexity based medicine (2-4).
Jureidini and McHenry state that “the advent of evidence based medicine was a paradigm shift intended to provide a solid scientific foundation for medicine” (1). There is another paradigm shift since then, with an updated science relevant to the allusion, and the illusion of evidence based medicine (EBM).
Evidence based medicine is based on 17-20th century reductionist science, with increasing limitations (2-4). A new chaos and complexity based medicine is emerging using 21st century complexity science as its base (2-4), which Stephen Hawking says he thinks will be the science for the 21st century. Greenhalgh describes using this new paradigm for EBM, referring to the 20th century logic of EBM (4).
Put very simply – chaos based medicine is evidence based medicine in context – which we all do anyway, using a chaos and complexity thinking as a translation factor to the individual or population (2-4). The limitations alluded to in “The illusion of evidence based medicine”, can be overcome by using complexity thinking and a chaos and complexity lens to interpret the acquisition and use of evidence and EBM.
I learnt to seek new approaches at the founding home of EBM, graduating from McMaster University medical school in its early days, and was taught by Dr David Sackett, one of the Founders of EBM, to whom I had the pleasure to give a copy of my 2000 book “Chaos Based Medicine – the response to evidence”.
I had written in a Lancet Letter to the Editor about science and evidence lying in the eye of the beholder, and using evidence in context, and in a BMJ post about McMaster as a chaos and complexity medical school long before its time, as the real reason it should be credited for being one of the 3 major advances in medical education for the 20th Century, not just for Problem Based Learning.
Evidence and evidence based medicine are necessary but not sufficient for decision-making, as often said by Dr David Naylor, Former University of Toronto Dean of Medicine, in various talks and publications.
Evidence based medicine is necessary, and should be updated to the 21st Century using chaos and complexity science as its base, to overcome limitations and to better understand and address 21st century medicine, health and related complex global issues (2-4).
References
1. Jureidini J, McHenry L. The Illusion of Evidence Based Medicine. BMJ 2022; 376:o702 doi: https://doi.org/10.1136/bmj.o702 (accessed April 10, 2022).
2. Rambihar VS. Chaos Based Medicine 2000: the response to evidence. Vashna Publications 2000 Toronto.
3. Rambihar VS, Rambihar SP, Rambihar VS Jr. Chaos Complexity Complex Systems Covid-19: 30 years teaching health professionals chaos and complexity. 10th International Conference on Complex Systems, NECSI 2020. https://static1.squarespace.com/static/5b68a4e4a2772c2a206180a1/t/5f1f12... (accessed April 10, 2022)
4. Greenhalgh T. Will Covid-19 be evidence based medicine’s nemesis? Ed. PLOS Medicine https://doi.org/10.1371/journal.pmed.1003266 June 30, 2020 (accessed April 10, 2022).
Competing interests: No competing interests
Dear Editor
Jureidini and McHenry have published an important message.[1] As they mention, the industry suppresses negative trial results, fails to report adverse events, and does not share raw data with the academic research community. By these methods, the drug companies and their well-paid scientific supporters have for instance succeeded with starting the cholesterol campaign. It is a fact that the cholesterol hypothesis is unable to satisfy any of the Bradford Hill criteria for causality.[2] The most obvious contradictions are that people with low cholesterol become just as atherosclerotic as people with high cholesterol [2]; that numerous studies have shown that elderly people with high LDL-cholesterol live just as long or longer than people with normal or low cholesterol [3,4]; and that no trial using cholesterol-lowering drugs have shown exposure-response.[2] Furthermore, the reason why some of the people with inherited, high cholesterol die prematurely from a heart attack is not their high cholesterol, but elevated coagulation factors which only a few of them have inherited as well. In fact, people with familiar hypercholesterolemia live on average just as long as other people.[5]
As reported in a recent, systematic review and meta-analysis of 21 statin trials by Burne et al.,[6] it is necessary to treat 77 people for more than four years to prevent a single myocardial infarction. In spite of that, millions of healthy people all over the world are treated with a drug with many serious side effects.[7]
References
1. Jureidini J, McHenry LB. The illusion of evidence based medicine. BMJ 2022;376:o702 http://dx.doi.org/10.1136/bmj.o702
2. Ravnskov U, de Lorgeril M, Diamond DM, et al. LDL-C does not cause cardiovascular disease: a comprehensive review of the current literature. Exp Rev Clin Pharm 2018;11: 959-70. doi.org/10.1080/17512433.2018.1519391
3. Ravnskov U, Diamond DM, Hama R, et al. Lack of an association or an inverse association between low-density-lipoprotein cholesterol and mortality in the elderly: a systematic review. BMJ Open 2016;6: e010401 https://doi.org/10.1136/ bmjopen-2015-010401.cholesterol.
4. Ravnskov U, de Lorgeril M, Diamond DM, et al. The LDL paradox: higher LDL-cholesterol is associated with greater longevity. Ann Epidemiol Public Health 2000;3:1040-7. https://www.researchgate.net/publication/347488140_The_LDL_Paradox_Highe...
5. Ravnskov U, de Lorgeril M, Kendrick M, Diamond DM. Inborn coagulation factors are more important cardiovascular risk factors than high LDL-cholesterol in familial hypercholesterolemia. Med Hypotheses 2018:121: 60-3. PMID: 30396495 DOI: 10.1016/j.mehy.2018.09.019
6. Byrne P, Demasi M, Jones M, Smith SM, O’Brien KK, DuBroff R. Evaluating the association between low-density lipoprotein cholesterol reduction and relative and absolute effects of statin treatment. A systematic review and meta-analysis. JAMA Intern Med doi:10.1001/jamainternmed.2022.0134
7. Diamond DM, Ravnskov U. How statistical deception created the appearance that statins are safe and effective in primary and secondary prevention of cardiovascular disease. Expert Rev Clin Pharmacol. 2015;8:201–10. DOI: 10.1586/17512433.2015.1012494
Competing interests: I have published books with criticism of the diet-heart idea and the cholesterol campaign
Dear Editor,
Thank you Jon Jureidini for another necessary reminder of the corruption which has occurred in universities regarding evidence based medicine.
Your previous work , such as the critical analysis of Study 329 was groundbreaking in highlighting the corruption which occurs when big pharma is allowed to run and control clinical trials.
Nothing has changed since then, in fact, the process seems only to have accelerated, proving that financial gains will always trump good science unless we change our broken system.
Thanks again for your good work.
Competing interests: No competing interests
Dear Editor:
I agree with the authors that Evidence Based Medicine(EBM) has been corrupted by corporate interests, failed regulation, and commercialization of academia [1] However, their solutions are not practical. Instead, if you take EBM in totality you can find solution within the system itself in the form of Reverse Evidence Based Medicine.[2]
Evidence based medicine may have many shortcomings. But in the absence of any better system, it is the best option for good medical practice. Like two sides of a coin, all evidences have two sides - obverse and reverse. We tend to follow the obverse side and call it the "evidence" whereas the reverse is also evidence and true.
To examine the reverse evidence, the raw data of a clinical trial is analyzed and a commonsense appraisal of the number of patients in the placebo or existing treatment arm is done. If the majority in the comparator arm has favorable outcomes, this will constitute the reverse evidence. This is done without complicated statistical analyses. While the evidence would support the new treatment, the reverse evidence will examine if the placebo or existing treatment has reasonably favorable outcomes.
Take the case of the landmark PARADIGM-HF trial comparing angiotensin-neprilysin inhibition versus enalapril in heart failure [3] The sample size was 8442 patients of which 4187 were in the angiotensin receptor-neprilysin inhibitor LCZ696 group and 4212 were in the enalipril group. 914 patients experienced adverse events in the LCZ696 group and 1117 in the enalapril group. It means 3263 (77%) patients in the LCZ696 group and 3095 (73%) in the enalapril group had favourable outcomes. Besides, the LCZ696 group was not free of adverse events. Based on this reverse evidence we can conclude that enalapril is reasonably good and can be used instead of LCZ696. This is especially true if you consider the cost of therapy.
So let us not throw out the baby with the bath water. Instead of the Quixotic solutions provided by the authors, we can simply check the reverse evidence of any trial. What is required is that the guidelines writers must include the reverse evidence also in their recommendations.
References
1. Juredini and McHenry. The illusion of evidence based medicine BMJ 2022;376:o702
2. Thomas G. Reverse evidence based medicine. Pan Afr Med J. 2013;16:89. doi: 10.11604/pamj.2013.16.89.2782.
3. McMurray JJ, Packer M, Desai AS, Gong J, Lefkowitz MP, Rizkala AR, Rouleau JL, Shi VC, Solomon SD, Swedberg K, Zile MR; PARADIGM-HF Investigators and Committees. Angiotensin-neprilysin inhibition versus enalapril in heart failure. N Engl J Med. 2014;371(11):993-1004. doi: 10.1056/NEJMoa1409077.
Competing interests: No competing interests
Dear Editor,
Good Day. Vanakkam and Namaste.
I enjoyed reading the article and cannot agree more with the authors. Evidence based medicine as is often practiced today, is Evidence b(i)ased medicine. There cannot be 2 views about the absolute need for all medical practice to be evidence based. The question is only whose Evidence? People in favor of a procedure/test/drug often cherry pick their data and people opposing do much the same leaving the average practitioner confused.
Fear of litigation is another reason why some practices continue. Electronic Fetal monitoring is a case in point.
Insurance coverage or lack of it is another reason why some medical practices are followed or not followed.
If all this is not enough, patients often demand that something be done, even when clearly nothing could be done or nothing is effective. Medical and surgical management of Oligo/Astheno/Teratozoospermia, whatever that means is the best example of such a situation.
Professional societies and Governmental and Non Governmental organizations often issue guidelines which are open ended leaving the practitioner to Err on the safe side, whatever that means, which is to go with the current practice.
Besides all these, Medicine after all started as craft, became an art and is now slowly trying to become a science. Oftentimes, it is not always Black and white and the gray areas remain unsolved.
Therefore, though Evidence Based Medicine is the only way forward, until such time clear evidence is available, the current practices, I feel should continue. The onus of proof is on people introducing a new drug/test/procedure to prove beyond the shadow of doubt that the drug/test/procedure is effective and not rely just on p<0.05. ,
Professor Dr Pandiyan Natarajan,
Chief Consultant in Andrology and Reproductive Sciences,
Apollo 24/7, NOVA IVF FERTILITY,
Chettinad Super Speciality Hospital ( Retired )
Founder Editor,( Retired ) Chettinad Health City Medical Journal.
Competing interests: No competing interests
Dear Editor
As a former researcher and scientist in the medical field I advocate the following concept:
"Science is a Method of Investigation to Search for Evidence, not Beliefs, nor Wishes, Conveniences and Preferences" --- President: Infarct Combat Project
References:
1) E-book “Autonomic Dysfunction + Lactic Acidosis = Multiple Diseases”, by Carlos ETB Monteiro, July 2021 .
2) Article: “Looking for the Cause and Cure of Diseases -- Possible Mechanisms Underlying the Relationship of Stress to Disease” by Carlos ETB Monteiro, Published in Contentment Magazine: Spring 2021 at https://www.stress.org/looking-for-the-cause-and-cure-of-diseases-possib...
Competing interests: No competing interests
Re: The illusion of evidence based medicine - Beyond the target?
Dear Editor.
Throwing the baby out with the bathwater seems to me to be the wrong way to go in this case too.
First of all, Sackett et al. define evidence-based medicine as a way to provide the best available external evidence as a basis for concrete therapy decisions. How this evidence is formed and what formal, content-related and ethical preconditions it comes from is another topic. Sackett formulates nothing more than a seemingly self-evident counter-position to a practical medicine that lives on authority expertise and selective experience. The fact that this is still a significant paradigm today should not be devalued by pointing out inadequacies at meta-levels. Or should we be under the illusion that such authority-led medicine is less susceptible to negative influences than "evidence-based research" or even provides a "better medicine"?
No, it is not, because it lacks an important factor: the self-correction process of the scientific community. In this, too, as in all human-driven activities, there are shortcomings, weaknesses and also deliberate failures. For example, this includes the fading out of plausibility and consistence for a over-focus on medical statistics. But to let this affect the concept of Sackett et al. in such a way that the whole paradigm of EBM is to be regarded as a failure is completely wrong.
Winston Churchill's well-known bon mot that democracy is the best of all bad forms of government is therefore undoubtedly also applicable to evidence-based medicine, certainly to that defined by Sackett et al. We do not have a crisis of EBM per se, but a crisis of scientific research and its evaluation.
And this is often a crisis of framework conditions that, for example, put independent research under pressure to publish, hold evaluation work in low esteem and push more into hypes than promote careful research. This seems to me to have little or nothing to do with EBM as a concept.
Competing interests: No competing interests