Alternative (complementary) medicine: a cuckoo in the nest of empiricist reed warblersCommentary: A warning to complementary medicine practitioners: get empirical or else
BMJ 1999; 319 doi: https://doi.org/10.1136/bmj.319.7225.1629 (Published 18 December 1999) Cite this as: BMJ 1999;319:1629
All rapid responses
Upon reading this article I was not sure whether to laugh or be
outraged. As a nontraditional student, I work hard to maintain my
composure during classes where outdated material is still being taught
concerning health. The time is long overdue for the empiricist like
yourself to step down and admit that your chemical and toxic approach to
curing the body of its ailments has failed. As long as you continue to
rely on Pharmaceutical and Chemical companies to do your research for you
and provide you with your education and knowledge of the drugs you
administer, you will never find the magic cure you are looking for. The
cures are right before your eyes and you not only don't see them, you are
too arrogant to open your eyes and look for them!
Competing interests: No competing interests
>1) Does he think all Chinese medical research showing positive
clinical outcomes should be disregarded?
No. However, I would be suspicious of any claim based purely on
Chinese data. Moreover, unqualified acceptance of Chinese research and
facile descriptions of Chinese research as far superior to Western
research (cf. Heptonstall's "pimple" comments) are to be avoided.
> 2) Does he consider there could have been any bias, or poor
quality research, in the acupuncture for smoking research that he quoted
unquestioningly in his letter dated 21 December?
Of course. I was referring to a systematic review published on the
Cochrane collaboration. This review (which was conducted by a practising
acupuncturist) looked at all data, good and bad, and concluded that there
really is any good reason to think that acupuncture is particularly
effective in smoking. Do you really expect acupuncture to be effective for
everything? How come research which shows acupuncture to be effective for
something (e.g. headache, nausea) just goes to show how good acupuncture
is, whereas research which finds acupuncture ineffective must be bad
research?
I am choosing the end this correspondence at this time. I will leave
two thoughts, however.
1) I will re-echo Haynes comment to acupuncturists (and others) to
"embrace empiricism". Many of the eBMJ letters by complementary
practitioners have a rather "antiestablishment" tone. Despite the fact
that I have worked in the complementary medicine field for 10 years,
published positive research on acupuncture and taught research methodology
at an acupuncture school for 4 years, I seem to be tarnished with being an
"enemy" simply because I am critical about some aspects of acupuncture.
Incidentally, "embracing empiricism" involves understanding empirical
methods. Some of the letters published belie poor understanding of
technical statistical or methodological issues. Do Heptonstall or
Churchill have any formal training in research methods? Has Heptonstall
ever conducted a funnel plot? Or do either have practical experience, for
example, by publishing original research in a peer reviewed journal?
2) Complementary practitioners need to accept that:
a) like every other health professional who has ever lived, not everything
they believe will turn out to be true;
b) it is possible to be critical of parts of something while being
supportive of the whole;
c) it may be the case that acupuncture is of benefit; it is probably not
the case that acupuncture is of benefit with the degree and scope claimed
by practitioners;
d) if evidence shows that something acupuncturists believed is actually
mistaken, that's okay;
e) uncertainty is a good thing.
Competing interests: No competing interests
Andrew Vickers (21 December and 30 March) seems to be condemning the
entirety of Chinese medical research:
- ‘there is good evidence that Chinese papers are subject to either
or both of methodological bias and publication bias’;
- ‘Chinese researchers never seem to publish negative results.’
I have known and studied with Chinese doctors who conducted research
on Chinese medicine. They were sincere, ethical, highly professional,
erudite individuals, committed to determining and elucidating best
treatment with the interests of patients at heart. If Dr.Vickers looked at
Chinese research, for example the abstracts published in the excellent
(western) Journal of Chinese Medicine, he might see that the research is
highly utilitarian, of great value to professional practitioners of TCM.
Yet from his letters on this page, one gains the clear impression that he
would have us regard all this as worthless.
Would Dr. Vickers have us believe that 100% of Chinese medical
research, on traditional Chinese medicine (TCM) or otherwise, should be
disregarded? On the basis of the two studies he cites, should we
make our working hypothesis that you just can’t trust a single positive
Chinese TCM study, only (possibly) the negative ones? Should we believe
that the Chinese can’t be trusted to evaluate their own medicine, and that
it is up to biomedical doctors in the West to do this for them?
It is interesting to note that Dr. Vickers makes no comment about
John Heptonstall’s detailed rebuttal (23 December, above) of Dr. Vickers’
statement that acupuncture is a useless treatment for smoking. Dr.
Heptonstall identified the possibility of poor quality research and
interpretation of research for smoking.
Two direct questions therefore to Dr. Vickers:
1) Does he think all Chinese medical research showing positive
clinical outcomes should be disregarded?
2) Does he consider there could have been any bias, or poor quality
research, in the acupuncture for smoking research that he quoted
unquestioningly in his letter dated 21 December?
Kind regards,
Wainwright Churchill
Competing interests: No competing interests
Editor
Vicker's tardy response to my letter deserves a prompt reply.
Yes, the wealth of Chinese research (and corresponding data
publication) into Traditional Chinese Medicine (TCM) is to be applauded as
compared to Medline, Cochrane etc. The latter are to be applauded for
their special values as a fount of excellence in many fields but, to me,
not TCM at the moment; this is obviously being addressed as I note that
more and more studies borne of true Chinese TCM-based research are
appearing amongst Western Medicine (WM) studies.
Isn't "the lack of negative findings in Chinese literature" you cite
really a myth? As I already said, much research is aimed at 'proving' to
WM science facts already known in TCM.
If I had been prescribing paracetamol for 2,000 years on my patients I may
have sufficient understanding and evidence that it works; how, when and if
to prescribe it for each individual - if I was then asked to prove this
through RCT I would apply the same methodology that had worked for me
after 2,000 years refinement - and would expect to be right most of the
time, wouldn't you?
I do not believe Prof. TANG et al say that ALL Chinese literature is
biased; the study says "MOST trials claimed that the tested treatments
were effective....". "Most" could be from just over 50% to 100%.
As to Vicker's 3 comments, the answers are:-
1. No
2. Isn't "Universally positive findings" a delusion?
3. TANG's 'funnel plot' was of 49 acupuncture studies for treatment of
stroke. The plot compares sample size with effects, and therefore ideally
requires treatments to be similar (eg all TCM or all non-TCM) and for
there to be a good spread of sample size. This was not the case. TANG says
only a few studies had sample sizes of 300 or more.
TCM-based studies ought to show a consistently better effect as each
diagnosis and treatment is patient specific; WM based studies are more
medication specific so effects would be more varied. I think that the plot
spread would be affected by the percentage of each type of trial in the
49; in this case a 'chaotic' pattern may have emerged which one could read
as proof of bias as one is not comparing like-with-like. What was the mix?
I note from the plot that 'acupuncture' was generally favoured over
control - that's a pretty expected result as far as we TCM practitioners
are concerned, even with WM-based acupuncture (when carried out by Chinese
WM/TCM medics as is usually the case).
According to 'Ernst and Resch' ALL non-RCT trials are potentially
biased. How many of the 49 were RCTs (TANGs report suggests many were non-
RCT) and does this factor affect funnel plot, which serves only to assess
likelihood of bias?
Whatever the state of bias or not in this funnel plot of 49 trials,
how does Vickers extrapolate that result to all of the other almost 3000
studies which formed part of Tang et al's project for which no funnel plot
was done and for which he intimates are 'universally flawed'?
Last time I communicated with prof. TANG he confirmed he did not have
original study papers; indeed his paper states that Journals were used for
data collection, and that many studies were published as only 'short
reports' thus having little detail. Most were of herbal medicine, and of
Western Medicine-based methodology. This reduces the impact of the
findings but clearly suggests more research is necessary. I would add that
better protocols, more use of original study papers, funnel plots based on
like methodology, and more attention to TCM-based studies (if one really
wishes to assess Traditional Chinese Medicine modalities) are advisable as
these are assessing best practice; most of these points I have already
taken up with Prof. Tang.
Answer to Q1; yes
Answer to Q2; I didn't.
Regards
John H.
Competing interests: No competing interests
Heptonstall originally claimed that a "wealth of Chinese data" was
something to be applauded (at least in contrast to "Medline papers"). I
pointed out that there is good evidence that Chinese papers are subject to
either or both of methodological bias and publication bias. The main
evidence for this is the lack of negative findings in the Chinese
literature. In response Heptonstall states: "Why are you so surprised when
experiments succeed in showing you what the Chinese already knew?"
I have three comments:
i) Does Heptonstall find it inconceivable that the Chinese could be
wrong about something?
ii) How come the universally positive findings from Chinese research
include both traditional medicine and modern techniques such as drugs and
surgery?
iii)Tang's funnel plot shows that the estimate of effect of the largest
Chinese trials are systematically different from those of smaller trials.
This is evidence of bias even if the trials are of a therapy known to be
of benefit.
Points ii) and iii) suggest that Heptonstall has not read the papers
I quoted in my letter.
Two direct questions therefore to Heptonstall:
a) Did you actually read the cited papers before commenting?
b) If so, how did you miss points ii and iii? If not, why do you find it
appropriate to comment on scientific literature that you have not read?
Competing interests: No competing interests
Dear Sir
I am writing with regard to your recent article in the BMJ: “Alternative
(complementary) medicine: a cuckoo in the nest of empiricist reed
warblers.”
I felt that in this article you made some very valid and important points.
Certainly no one would debate the need to protect the patient. However, I
feel that in the way you have presented your case with an opinion biased
from the start against complementary therapies, you have undermined the
case
of empiricism.
You explain many of your arguments against alternative medicine by
comparing alternative medicine to magic, and then asking do we believe in
magic? No, probably we don’t. However magic is a very unlikely word to
associate with
alternative medicine in the first place. So if we take that negative
connotation away, do we believe in alternative medicine? Well, we may not
understand it, nor may we be able to explain it within the terms we use to
explain conventional medicine. Does this prove alternative medicine is not
useful - no. Many forms of alternative medicine have evolved from
centuries of experience and observational knowledge. Do we believe in
these principles
- yes. Over the centuries, conventional medicine has also evolved,
incorporating principles that for a time may not have been understood, or
even believed my the majority of people (for example in the days of the
black death, no one believed in such alien concept as germs carried in the
air). Furthermore, many of these concepts could not be proven with methods
available at the time. This was just an indication that our knowledge was
incomplete at that time. Well maybe our knowledge is incomplete at this
time, and when we develop means of investigating the things we do not yet
understand, we will see that alternative medicines have the weight of
rational science behind them. Until then, we do ourselves no favours by
denying the worth of alternative therapies.
You suggest that there are some things that cannot even be included on the
fringes of scientific method: “...there are hypothesis that cannot be
accommodated... that a substance that causes complaints similar to the
ones observed in a patient will, if diluted to an infinitesimal
concentration,
cure them.” I appreciate that to the eyes of conventional medicine, this
concept is puzzling, and yet, puzzling or not, this concept (of
homoeopathy) has been shown to work in practice in many cases. Surely this
alone gives it
the right to be accommodated, or at the least tested empirically. If we
deny it out of hand, without even investigating it, we lose something that
has stood the test of time and practice, something that may well be
effective
and of benefit.
I mentioned above, our lack of means of investigating these therapies, and
in your article you referred to the problems associated with the use of
randomised controlled trials for acupuncture. Such trials, whilst
excellent for conventional medicine, are based on the principles of that
medicine.
Namely that for a particular illness, there will be a particular cure.
However many alternative therapies work on more specific principles that
this. For one particular illness, in one particular person, and for the
particular symptoms that they have, there will be a particular cure. Thus
for a group of patients with, for example, migraine, there will be many
homoeopathic remedies, based on the patient, and the particular symptoms,
rather than just on the fact that they all have what conventional medicine
terms “migraine”. I have noticed also that at times conventional medicine
appears more of an art form than a science. For example: varying diagnosis
from separate doctors of the same complaint; various different
prescriptions for the same complaint; different responses from the same
doctor to the same
complaint, on different days. However, some alternative medicines provide
much more stringent guidelines for the diagnosing of illness and
prescribing of treatment, based on the whole patient, rather than just the
one presenting illness. Thus when looking to test such alternative
therapies, we should design our trials to take into account the different
principles underlying their use, rather than trying to make them comply
with conventional medicine’s principles. To attempt such compliance, is
not an
empirical trial. This situation may well be an example of where we do not
yet have the methods to test out hypothesis (in this case, the hypothesis
being alternative medicine.)
In your article, you create generalisations based on putting the whole
spectrum on alternative therapies in one category. Unsurprisingly, such
generalisations bear no relationship to the original alternative
therapies.
You say that “they are saying that herbs are beneficial and can do no
harm.”
I have never heard this said, although I have heard it said that some
herbs, in some situations, can be beneficial. No one could deny this (for
example the plant product digitalis). By generalising we deny the worth of
the thing
generalised, and thus lose the potential for advancement of knowledge.
Surely the empirical approach would be to study that which we do not fully
understand, in order to enrich our knowledge from it? No one could suppose
that a therapy of no use or value, could not only evolve, but also stand
the
test of time. Common sense tells us that there must be some truth in these
therapies, even if we do not yet understand it.
In conclusion, I agree with you that alternative medicines should be
investigated. Maybe then conventional medicine can learn from them. More
importantly, though, they must be investigated with a mind to their
underlying principles, so that any investigations are capable of giving a
true result that will be believed. In order to do this, we must not scorn
complementary medicine, but instead treat it as an objective hypothesis
worthy of our investigation. Only then will be able to gain from the
investigations, and only then will we be empirical.
Charlotte Jago
Competing interests: No competing interests
Professor Leibovici makes the excellent suggestion that, in a real
world situation with constraints on time, money and the sustained
concentration of researchers, "a deep model of the physical world is
essential for choosing hypotheses to be tested and from learning from
failures".
While his context is alternative/ complementary medicine, this point
could be applied to any area of research.
There is, however, an underlying assumption.
This is: that one's "deep model" - be it of life/ the universe/
whatever - is correct, full, and accurate.
This in turn implies a certainty which we cannot not possess. It also
allows only for what we already know, and not for what we have not yet
discovered or learned.
History is littered with examples of "deep models" which have been
later shown to be inaccurate. The most famous is probably Galileo: the
senior Roman Catholic clergy of the day held a deep model which stated the
earth was the centre of the universe, around which all other "heavenly"
bodies revolved. Later in England, a clockmaker made an extremely accurate
and reliable timepiece, which enabled sailors for the first time to
accurately measure longtitude at sea. However the panel of experts to whom
he presented this were astronomers, who had a deep model which stated that
it was impossible for any mechanical clock - no matter how well presented
- to be sufficiently accurate for this purpose. And modern nuclear physics
has probably turned much of the previously held scientific deep model on
its head.
The discovery of penicillin may have also owed much to Fleming
holding a slightly different deep model to his peers - one which resulted
in him being open to the possibility that mould was not simply a domestic
and laboratory nuisance.
Also, on a more individual basis, modern day psychotherapy - from
cognitive therapy to age regression, and virtually all other
psychotherapeutic approaches - aims to help an individual identify and
correct errors in their own "deep model" of life.
It is more valid to consider that our deep model of the physical
world is probably - perhaps very probably - true. We cannot state that it
is certainly true, for this does not allow for what we do not yet know and
may yet discover.
Such an approach - based upon a "probably true" deep model of the
physical world - will allow for a degree of caution in deciding where and
how to use our resources in medical (or indeed any other) research, and in
evaluating the results. It will also allow (hopefully)for the open-
mindedness needed when an unexpected and possibly major discovery is -
literally - in front of our very eyes.
Competing interests: No competing interests
I enjoyed your article.
I did a psychology degree before I trained as a homoeopath, then I
joined a homoeopathic research group because I did understand the need for
empirical research, having been trained to do it.
However, I was the only person in the group that had any training in
research, I found it very difficult to operate professionally in a group
with no other expertise in research, no helpful connections to anyone who
did, no finance, no resources, no ground plan and no possibility of
getting anywhere!!! We did produce an attempt at an audit of homoeopathic
practice, presenting symptoms and some patient feedback, but the group
lurched about so much, you could see that it would (and has) taken years
before such a group will produce meaningful data. Alternative therapists
do not have research expertise, though it is improving now, they do not
have any resources, nor any funding, nor any hope of funding.
Eventually, I did a small audit of my own practice with the doctors I
work for (my day job in a local NHS GP surgery). I was trying to highlight
the very high level of mental trauma seen at the first presentation of
clients - check out the results on my web page www.sue@falcon1.demon.co.uk
I had no funding, nor any extra time set aside to do this work, so most of
it was done late at night, after my other two jobs were over and instead
of sleep. All in all, it took me and my son about eight weeks to finish
this small piece of preliminary work, to enable the local doctors to see
why I was being consulted. I could only analyse first consultation data,
as including data from follow up appointments was completely beyond my
resources then and now.
I recal a chat I had on lyghtforce.com - an alternative mailing list.
Some bright spark suggested that we should do some empirical research -
some evidence based medicine! Someone replied that they had a spare 50p,
another person donated a pound, someone else promised a fiver. All in all
we collected about £15 on the discussion line - this is not a joke - it is
the reality of alternative empirical research.
If you don't mind me suggesting this, you lot have all of the
resources and the time and the expertise to do research. Most of us
alternatives are at least twenty years behind it terms of resources and
expertise. A simple experiment such as the use of arnica after facial
plastic surgery, which was statistically significant I believe, in favour
of homoeopathic treatment, is a simple thing for you to design, organise
and carry out. How are we supposed to do this sort of research without
access to proper facilities?
I have asked many times to be allowed to visit an A&E department
for example, to work with doctors suggesting simple alternative treatments
such as the use of aconite and arnica in A&E settings, but of course
I'm shut out. We have real problems because we are on the fringe, but we
are only allowed to be on the fringe. Vested interests and patriarchal
top of the food chain behaviour will only change if patients demand it.
We can only provide evidence based medicine if we are allowed do the
research, and if we know how to do the research. The cookoo clamour you
hear is from the patients - your patients, now our patients, because they
are demanding humane medical techniques and they are desperate to be
listened to.
All of the alternative therapists I know would agree with every word
of your article, but we are below a glass floor here, with you lot dining
above us with all the resources in your hands. Thus I challenge you to do
the research, and to learn to include us (your subjects too), and to truly
hear what we are all saying. The charlatans on both ends of both our
professions will then drop off.
Regards
Sue Young RSHom
Competing interests: No competing interests
After reading this article, I am confused as to the definition of
"empirical," "empricism." The authors seem to suggest that mainstream
medicine is empirical, suggesting that it is medicine which is scientific
and evidence based. They suggest that complementary medicine should
develop an empirical ideology.
However if one looks at the definition of empirical; based on
experience and observation without regard to science or knowledge of
principles(Butterworths Medical Dictionary), instantly the arguement falls
apart. I am confused, perhaps I read the article incorrectly, or perhaps
the authors have not understood the definition of empirical and
empiricism. Surely not!!
Competing interests: No competing interests
Re: The evolving nature of empiricism
To Tim McClune,
I've had the same problem in coming to terms with empiricism.
Observation and experiment (of the clinical type) are what has driven
natural therapies practices forward for centuries. It seems to me that
empiricism has evolved and in its contemporary version the nature of the
experiments and the observation have both changed.
Experiments must now be
based on statistical data of a certain kind, and observation currently
leaves out most of the senses. Touch, smell, hearing and taste, those
ancient clinical skills, are abandoned in favour of sight, ie looking at
the data (not the patient). Curiously, the mind seems to be left out of
the definition yet is central to experimental design, the interpretation
of data, and the assigning of meaning, all of which influence the
direction and outcome of research. The postmodern philosophers have been
eager to point out the inescapable influence of prior beliefs due to
culture, and the culture of science is no exception. By reducing so
dramatically the nature of acceptable evidence and operating within a
mechanistic view of life, the world of medical science has produced in
many ways a distorted and biased approach to human health.
Empiricism,
like all 'isms', is a philosophical position. As a deep model its meaning
needs to be explored by CAM, its history deconstructed, and a way found to
continue its evolution in a way that embraces the full range of
observation and experiment necessary to understanding human health. The
perennial philosophical question, 'how should one live' is not purely a
moral question. Health and peace of mind are inseparable partners, and I
believe philosophy generally, not just empiricism, needs to be brought
back into the arena of human health. Philosophy has the potential to make
a significant contribution to helping people make the daily choices that
support health and prevent illness. We need to realise that philosophy has
also been cast out by science, cursed with the epithet 'subjective', and
left to a journey divorced from the natural world for the last century or
so. Complementary health care researchers can help reclaim philosophy,
restoring it as a living discipline that is fundamental to life and health
as well as research.
Competing interests: No competing interests