What's the E for EBM?
BMJ 2004; 328 doi: https://doi.org/10.1136/bmj.328.7439.535 (Published 04 March 2004) Cite this as: BMJ 2004;328:535
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Systematic reviews of evidence summarise scientific evidence
addressing questions of treatment, causation, diagnosis and prognosis.
They help health professionals to keep abreast of their speciality of
interest as well as remain literate in the broader aspects of medicine.
Researchers use them to identify and refine their hypotheses leading to
better primary research. Health policy makers formulate guidelines and
legislation based on the conclusions of the reviews. They also provide
high quality information to patients as the evidence in them is not just
added but multiplied.
A landmark example is the review of trials that compared intravenous
streptokinase with a placebo or no therapy in patients with acute
myocardial infarction from 1959 -1988. Though the effect of treatment was
favourable on mortality in 25 of the 33 trials, it was statistically
significant only in six studies. The overall pooled effect favoured the
treatment significantly. When the same data were used to perform a new or
cumulative meta-analysis each time the results of a new trial were
reported, statistical significance was attained for a positive effect in
1973 for a two-sided P value of < 0.01.1,2 The evidence for the
beneficial effect of intravenous streptokinase on mortality was available
20 years before it was put into regular use.
Evidence based medicine has provided an easily accessible, vital tool
in the form of systematic review. It is a simple, transparent approach
that gives credibility to studies where due and assimilates good evidence
along the way. ‘Where is the wisdom we have lost in knowledge? Where is
the knowledge we have lost in information?’ asked T S Eliot. The Cochrane
Database of Systematic Reviews has created an oasis for knowledge in the
age of information. It is up to us to get the wisdom from it.
1 Lau J, Antman EM, Jiminez-Silva J, Kupelnick B, Mosteller F,
Chalmers TC. Cumulative meta-analysis of therapeutic trials for
myocardial infarction. N Engl J Med 1992;327:248-54.
2 Mulrow CD. Rationale for systematic reviews. BMJ 1994;309:597-99
Competing interests:
None declared
Competing interests: No competing interests
Effectiveness of agents (caustion, diagnostic or therapeutic)
Validated
Independently,
Decorously
Evaluated,
Neutral [without any bias or influence] and
Confirmed
Elegantly
Competing interests:
None declared
Competing interests: No competing interests
Matthew Kiln cites ghostwriting as one of the reasons why medical
evidence is inherently unreliable. This statement is itself not very
evidence-based.
Is there any evidence to show that ghostwriting makes a paper any
more unreliable than a paper written by its named authors? Not to my
knowledge. In fact, a systematic review published last year found that
remarkably little research has been done on this subject [1]. However, as
a professional medical writer, I resent the suggestion that the papers I
write are unreliable. I take great care to ensure that the papers I write
are scientifically valid, as I am sure do the overwhelming majority of
professional medical writers.
There is no reason why the named authors of ghostwritten papers
should not know how accurately their research has been written up, as Kiln
suggests. Discussions between the named author and the ghostwriter are
integral part of the ghostwriting process. As for seeing the raw data, it
is true that authors often do not see it, but what would they do with it
anyway? Raw data by itself is not usually very helpful: it is only after a
statistician has processed it into meaningful tables and graphs that it
becomes useful. I would always expect the named author of any paper that I
ghostwrite to have access to the statistical output.
Kiln is absolutely correct, however, to point out that we do not know
how many research papers are ghostwritten. Further research on this and on
the quality of ghostwritten papers is urgently needed. I hope that the
recently-established Ghostwriting Task Force of the European Medical
Writers Association will be able to make some progress in this area in due
course. Anyone with ideas about how such research could best be done is
welcome to contact me by email.
References:
1. Lagnado M. Professional writing assistance: effects on biomedical
publishing. Learned Publishing 2003;1:21–27
Competing interests:
I make a substantial portion of my living from ghostwriting. I am also vice-president of the European Medical Writers Association and head of its Ghostwriting Task Force.
Competing interests: No competing interests
I know this request was for submissions for a theme issue to be
published in October in the BMJ (1),but because the evidence used in EBM
is sometimes fundamentally flawed I felt I should write a letter as well.
The E in EBM stands for 'Equivocal', because that's the best value
that can ever be given to the majority of published, controlled, double
blind research studies (no matter what medical journal it is in).
The reasons for why the 'evidence' for EBM is inherently unreliable are as
follows;
a). The large volume of research funded by drug companies that is not
allowed by them, to be submitted for publication (2.3).
b). Totally unsuitable patients are sometimes into sensitive drug
trials,(personal observation from legal work that I do)one only needs a
few such patients to invalidate the conclusions of studies. There is no
way when doing peer reviewing, or reading the paper after publication that
this can be detected. I have only noticed this from reading the paper,
carefully going through the patients notes and talking to the patient at
length. I have seen this in diabetes research, but it could be commom in
areas of medical research, because no one usually has the time,the
position, and the information to check for such practices.
c). Ghost-writing,(34)we will never know how many research papers
this happens in but it is likely to be quite a few. As well as this the
choosen 'well known' professors or doctors that put their names on
research papers have never even seen the raw data, let alone know how
accurately it has been the written up !
d). Effect of Bias. Whether it is intentionally or unintentionally
introduced by researchers, doctors, stitisticians, ghost-writers the
effect is difficult to quantify. Declared and undeclared interest id sone
element (5). However, there are so many other ways that bias occurs, that
to publish a double blind research paper without bias seems virtually
impossible.
EBM has a small role in practcing medicine, and it is important to
remember how eqvivocal the E can be.
Yours faithfully
Dr Matthew Kiln.
Ref:1. Straus S. What's the E for EBM?. BMJ 2004;328:535-36.
2.Kiln M. Industry-Sponsored Research. Lancet 2001;357:1209-10
3.Moynihan R.Who pays for the pizza? Re defining the relationships
between doctors and drug companies. BMJ 2003;326:1189-92.
4. Reeves S. Who actually wrote the research paper?How to find
out.(rapid response to : Smith R.medical Journals and pharmaceutical
companies: uneasy bed fellows. BMJ 2003; 326:1202-25.
5. Barnes DF, Bero LA. Why review articles on health effects on
smoking reach different conclusions. JAMA 1998;279:1566-70.
Competing interests:
I am Co-Chairman of a patient support charity
Competing interests: No competing interests
Editor – Evidence-based medicine frequently encounters barriers to
its translation into evidence-based practice1. These stand both at an
individual level and at an organisational or national level. The failure
of individual clinicians to practice evidence-based medicine is well
described and may centre around the accessibility of the evidence,
relevance of the information retrieved, accuracy of available information
and difficulties with understanding or interpreting the evidence2. Adult
learning theory suggests that adults learn best experientially; that is in
context, with clear or explicit information delivered timeously in a
problem-oriented approach3. Information technology coupled with an
increasing body of available evidence can help to bridge some of these
gaps4. For many clinicians, pre-appraised evidence provides a useful and
accessible resource where time and appraisal skills may be lacking2,4.
The challenge for continuing medical education and the delivery of high
standards of medical care will be using modern media to reflect best
available evidence with accuracy, understandability and applicability5.
The Stroke Therapy Evaluation Programme (STEP) is developing an initiative
to develop and regulate a website of best current evidence for stroke
care. It is organised in a problem-oriented structure with accurate pre-
appraised evidence available to the end user. This process needs to be
transparent, unbiased and of high quality. Additionally we are aware that
users including patients may vary in their terminology and approach to
evidence, and an adequate ontology of end users will be a valuable part of
the development process. The website is expected to “go live” in October
2004 and feedback would be welcome. (www.effectivestrokecare.org)
PS. The Stroke Therapy Evaluation Programme has developed over
several years with supported from Chest Heart and Stroke Scotland, The
Healthcare Foundation, the Jeffrey Trust and the New Opportunities Fund
Graham Ellis,
Clinical Research Fellow
Lynn Legg,
Co-ordinator
Peter Langhorne
Professor of Stroke Care
Stroke Therapy Evaluation Programme,
Glasgow Royal Infirmary,
Castle Street,
Glasgow
G4 0SF
1 Straus SE What’s the E for EBM? BMJ 2004;328:535-6
2 Young JM, Ward JE Evidence-based medicine in general practice: beliefs
and barriers among Australian GPs
3 Knowles M. The Adult Learner: A Neglected Species (3rd Ed). Houston,
TX: Gulf Publishing
4 Guyatt GH, Meade MO, Jaeschke RZ, Cook DJ, Haynes RB. Practitioners of
evidence based care. BMJ 2000;320:954-5
5 Kunst H, Groot D, Latthe PM, Latthe M, Khan KS. Accuracy of information
on apparently credible websites: survey of five common health topics. BMJ
2002;324:581-.2
Competing interests:
Graham Ellis and Lynn Legg are employed by the Stroke Therapy Evaluation Project
Competing interests: No competing interests
Dear Editor,
Following a recent journal club, at which we discussed Smith and
Pell's article on the lack of evidence for the assumed beneficial effects
of parachutes (1), and Prof. Strauss's call for opinion (3), I should
like to offer an alternative overview of levels of evidence.
There seem to be three fundamental types of medicine practised:
evidence-based medicine, eminence-based medicine (3) and incidence-based
medicine. A discussion of the first is best left to the forum proposed by
Prof. Strauss and involves an attempt at empirical guidance of one's
practice. The second is often quoted with a preamble such as "John Smith
always used to 'because he thought that'". This depends, ideally, on
an eminent teacher having collated evidence of all types with his or her
(usually long) personal experience and understanding or intuition. Such
paradigms may either be passed down on tablets of stone or as seeds for
discussion or further study.
Incidence-based medicine is that which is predicated on infrequent
incidents, usually negative with n less than or equal to 2. Most doctors, particularly
surgeons, when asked why they do something in a particular way will launch
into the relevant case history accompanied by either nervous laughter or
resurrected horror.
Whilst it may seem logical that the evidence types should be in
reducing order of importance, as listed above, when it comes to guiding
one's clinical practice, it is apparent that the single disaster may have
the greatest effect. This may even be more influential than the teachings
of one's mentor and certainly the results of a prospective controlled
randomised clinical trial with several hundred patients in each arm of the
study. Of course, I have no evidence for this, but everyone I've asked
seems to agree.
Devinder Chauhan MD FRCOphth
(1) Smith GCS, Pell JP. BMJ 2003; 327: 1459-1461
(2) Strauss SE. BMJ 2004;328:535-536
(3) Stahl SM. Acta Psychiatr Scand. 2002 Nov;106(5):321-2.
Competing interests:
None declared
Competing interests: No competing interests
I am frustrated by this editorial. As an academic clinician, who
spends at least 5 days each week directly involved in the care of patients
with cancer, I may have acquired some insights into the issues raised by
Professor Straus. However, precisely because the care of patients is my
primary concern, it is unlikely that I will be able to contribute to the
debate. Where might I, and others like me, find the time within the next
six weeks to write an article and navigate it through the maze that is the
BMJ’s process for online submission? The imposition of this tight deadline
will bias the contributions in your special issue against those from
active clinicians and in favour of submissions from those with less direct
responsibility for the care of patients. I realise that this is a testable
(and perhaps unjustified) assumption. With this in mind, it will be
interesting to study the authorship of the papers that do eventually
appear in the special issue
Competing interests:
Editor, Cochrane Colorectal Cancer Group
Competing interests: No competing interests
It seems as it is in the air with some reflections on the EBM theme!
Some of us with clinical and research roots from the late half of last
century are still involved in lifelong learning and, more specific so ,
in various teaching processes!
As for myself it means courses for GPs – not that much, for obvious
reasons, in modern medicine but, for likewise obvious reasons, in the art
of medicine!
Concerning medical treatment it seems not to be much problem in
accepting the role of evidenced based medicine. Younger as well as older
GPs are well “on line” with recent development - in principle!
However- when it comes to medical practise the picture may be
somewhat different!
Not the least in to days general practice when some times patients with
subjective signs of illness, quite often lifestyle related and
psychosocial in type, are becoming more frequent in numbers than those
with objective signs of disease! Not that easy to be an GP there - in
spite of all evidenced based medicine which usually does not include that
much of evidence concerning “softer” modern lifestyle related problems!
In certain situations confidence in medicine may have to compensate
for the lack of evidence.
Such confidence that has to be developed between the patient and the GP,
as prerequisite for further treatment , is likely to be inspired by the
GPs earlier experience and likewise influenced by some ethical
considerations: his logical, evidence based professional ego would perhaps
tell and guide him no to offer much support as it would have no evidence
based effect but again his experiences from years of clinical praxis
would lead him to bypass the knowledge of evidence and offer help he feels
could help! In accordance to the more traditional ethical code most
doctors always have tried to adhere to !
One is here almost coming close to a somewhat “alternative attitude”-
indeed sometimes in a rather “alternative “ clinical situation as well -
from purely evidence point of view!
In that respect there are perhaps to more E to consider E as in Experience
and E as in Ethics!
Competing interests:
None declared
Competing interests: No competing interests
The commonest rebuttal that I hear from my colleagues in practice,
especially those less connected with academic practice, is that they
resent the way that EBM places implied strictures on practice, limiting
their "art of medicine", and is rarely relevant to their actual patients.
"I have no intention of practising cookbook medicine" seems to be the oft
heard cry.
And yet, when we look at the frightening numbers seen when
considering medical error rates and the huge amount of death and
destruction attributable to medical practice, it is hard to reconcile this
view. Our patients worry about safety in the flying industry but are so
far blithely unaware of the very real dangers involved in medical
transactions. This will change, and if so, will we be forced to adopt the
very rigid safety structures of the flying industry, with all its
checklists and redundancy?
Perhaps this is taking an unnecessarily bleak standpoint, but if we
don't get our own house in order, we will all be the worse for having this
done for us and to us. I look forward to this October issue and the
debates that may ensue.
Competing interests:
None declared
Competing interests: No competing interests
GPs say evidence-based information is changing practice
As the publishers of the BMJ’s Clinical Evidence we have more than a
passing interest in the extent to which the provision of evidenced-based
information changes clinical practice. In this context, the results of an
evaluation of Clinical Evidence assumed some importance, and were awaited
with some nervousness.
5960 GPs in England were contacted by Stingray Research, an
independent market research company, and asked to provide some broad
perceptions about the role of evidence in their day to day practice and to
pass judgement on Clinical Evidence. The response rate was 838/5960
(14.1%).
Some findings were not surprising. 75% of GP respondents reported
that their patients were likely to demonstrate interest in the latest
research findings. 97% of the GPs had used an information resource to find
the latest evidence and 45% expected to do so at least once every
fortnight. Subject matter ranged across the broad spectrum of disease.
Clinical Evidence was mainly used around clinical consultations – before,
during and after. However, other uses, such as education and teaching, and
assisting the development of practice guidelines, were also reported by
77% and 52% of respondents respectively.
The most crucial findings related to the proportion of doctors, 75%,
who reported that they had changed their practice as a result of using
Clinical Evidence. Two thirds of these had done so in the previous 6
months. Change of practice in response to Clinical Evidence was most
likely in younger doctors, but was also reported by 56% of respondents
aged over 55.
Quotes from respondents underlined the message; that provision of
evidence from an independent, trusted source supported them in developing
their practice and improved the quality of their consultations.
There are limitations to this evaluation. We don’t know what the non-
respondents thought for example. However, the evidence does seem to imply
that there is a substantial body of clinicians of all ages – in this case
English GPs, but we know that similar results have been found amongst
Italian (1) and US doctors – who are motivated to use evidence based
sources to improve their care for patients.
1. Clinical Evidence: a useful tool for promoting evidence-based
practice? Giulio Formoso, Lorenzo Moja, Francesco Nonino, Pietro Dri,
Antonio Addis, Nello Martini, Alessandro Liberati
Competing interests:
Both authors are Clinical Evidence authors
Competing interests: No competing interests