Alternative (complementary) medicine: a cuckoo in the nest of empiricist reed warblersCommentary: A warning to complementary medicine practitioners: get empirical or else

>1) Does he think all Chinese medical research showing positive
clinical outcomes should be disregarded?

No. However, I would be suspicious of any claim based purely on
Chinese data. Moreover, unqualified acceptance of Chinese research and
facile descriptions of Chinese research as far superior to Western
research (cf. Heptonstall's "pimple" comments) are to be avoided.

> 2) Does he consider there could have been any bias, or poor
quality research, in the acupuncture for smoking research that he quoted
unquestioningly in his letter dated 21 December?

Of course. I was referring to a systematic review published on the
Cochrane collaboration. This review (which was conducted by a practising
acupuncturist) looked at all data, good and bad, and concluded that there
really is any good reason to think that acupuncture is particularly
effective in smoking. Do you really expect acupuncture to be effective for
everything? How come research which shows acupuncture to be effective for
something (e.g. headache, nausea) just goes to show how good acupuncture
is, whereas research which finds acupuncture ineffective must be bad
research?

I am choosing the end this correspondence at this time. I will leave
two thoughts, however.

1) I will re-echo Haynes comment to acupuncturists (and others) to
"embrace empiricism". Many of the eBMJ letters by complementary
practitioners have a rather "antiestablishment" tone. Despite the fact
that I have worked in the complementary medicine field for 10 years,
published positive research on acupuncture and taught research methodology
at an acupuncture school for 4 years, I seem to be tarnished with being an
"enemy" simply because I am critical about some aspects of acupuncture.
Incidentally, "embracing empiricism" involves understanding empirical
methods. Some of the letters published belie poor understanding of
technical statistical or methodological issues. Do Heptonstall or
Churchill have any formal training in research methods? Has Heptonstall
ever conducted a funnel plot? Or do either have practical experience, for
example, by publishing original research in a peer reviewed journal?

2) Complementary practitioners need to accept that:

a) like every other health professional who has ever lived, not everything
they believe will turn out to be true;

b) it is possible to be critical of parts of something while being
supportive of the whole;

c) it may be the case that acupuncture is of benefit; it is probably not
the case that acupuncture is of benefit with the degree and scope claimed
by practitioners;

d) if evidence shows that something acupuncturists believed is actually
mistaken, that's okay;

e) uncertainty is a good thing.

Editor

Vicker's tardy response to my letter deserves a prompt reply.

Yes, the wealth of Chinese research (and corresponding data
publication) into Traditional Chinese Medicine (TCM) is to be applauded as
compared to Medline, Cochrane etc. The latter are to be applauded for
their special values as a fount of excellence in many fields but, to me,
not TCM at the moment; this is obviously being addressed as I note that
more and more studies borne of true Chinese TCM-based research are
appearing amongst Western Medicine (WM) studies.

Isn't "the lack of negative findings in Chinese literature" you cite
really a myth? As I already said, much research is aimed at 'proving' to
WM science facts already known in TCM.
If I had been prescribing paracetamol for 2,000 years on my patients I may
have sufficient understanding and evidence that it works; how, when and if
to prescribe it for each individual - if I was then asked to prove this
through RCT I would apply the same methodology that had worked for me
after 2,000 years refinement - and would expect to be right most of the
time, wouldn't you?

I do not believe Prof. TANG et al say that ALL Chinese literature is
biased; the study says "MOST trials claimed that the tested treatments
were effective....". "Most" could be from just over 50% to 100%.

As to Vicker's 3 comments, the answers are:-

1. No
2. Isn't "Universally positive findings" a delusion?
3. TANG's 'funnel plot' was of 49 acupuncture studies for treatment of
stroke. The plot compares sample size with effects, and therefore ideally
requires treatments to be similar (eg all TCM or all non-TCM) and for
there to be a good spread of sample size. This was not the case. TANG says
only a few studies had sample sizes of 300 or more.
TCM-based studies ought to show a consistently better effect as each
diagnosis and treatment is patient specific; WM based studies are more
medication specific so effects would be more varied. I think that the plot
spread would be affected by the percentage of each type of trial in the
49; in this case a 'chaotic' pattern may have emerged which one could read
as proof of bias as one is not comparing like-with-like. What was the mix?

I note from the plot that 'acupuncture' was generally favoured over
control - that's a pretty expected result as far as we TCM practitioners
are concerned, even with WM-based acupuncture (when carried out by Chinese
WM/TCM medics as is usually the case).

According to 'Ernst and Resch' ALL non-RCT trials are potentially
biased. How many of the 49 were RCTs (TANGs report suggests many were non-
RCT) and does this factor affect funnel plot, which serves only to assess
likelihood of bias?

Whatever the state of bias or not in this funnel plot of 49 trials,
how does Vickers extrapolate that result to all of the other almost 3000
studies which formed part of Tang et al's project for which no funnel plot
was done and for which he intimates are 'universally flawed'?

Last time I communicated with prof. TANG he confirmed he did not have
original study papers; indeed his paper states that Journals were used for
data collection, and that many studies were published as only 'short
reports' thus having little detail. Most were of herbal medicine, and of
Western Medicine-based methodology. This reduces the impact of the
findings but clearly suggests more research is necessary. I would add that
better protocols, more use of original study papers, funnel plots based on
like methodology, and more attention to TCM-based studies (if one really
wishes to assess Traditional Chinese Medicine modalities) are advisable as
these are assessing best practice; most of these points I have already
taken up with Prof. Tang.

Answer to Q1; yes
Answer to Q2; I didn't.

Regards

John H.

Dear Sir

I am writing with regard to your recent article in the BMJ: “Alternative
(complementary) medicine: a cuckoo in the nest of empiricist reed
warblers.”
I felt that in this article you made some very valid and important points.

Certainly no one would debate the need to protect the patient. However, I
feel that in the way you have presented your case with an opinion biased
from the start against complementary therapies, you have undermined the
case
of empiricism.

You explain many of your arguments against alternative medicine by
comparing alternative medicine to magic, and then asking do we believe in
magic? No, probably we don’t. However magic is a very unlikely word to
associate with
alternative medicine in the first place. So if we take that negative
connotation away, do we believe in alternative medicine? Well, we may not
understand it, nor may we be able to explain it within the terms we use to
explain conventional medicine. Does this prove alternative medicine is not
useful - no. Many forms of alternative medicine have evolved from
centuries of experience and observational knowledge. Do we believe in
these principles
- yes. Over the centuries, conventional medicine has also evolved,
incorporating principles that for a time may not have been understood, or
even believed my the majority of people (for example in the days of the
black death, no one believed in such alien concept as germs carried in the
air). Furthermore, many of these concepts could not be proven with methods
available at the time. This was just an indication that our knowledge was
incomplete at that time. Well maybe our knowledge is incomplete at this
time, and when we develop means of investigating the things we do not yet
understand, we will see that alternative medicines have the weight of
rational science behind them. Until then, we do ourselves no favours by
denying the worth of alternative therapies.

You suggest that there are some things that cannot even be included on the
fringes of scientific method: “...there are hypothesis that cannot be
accommodated... that a substance that causes complaints similar to the
ones observed in a patient will, if diluted to an infinitesimal
concentration,
cure them.” I appreciate that to the eyes of conventional medicine, this
concept is puzzling, and yet, puzzling or not, this concept (of
homoeopathy) has been shown to work in practice in many cases. Surely this
alone gives it
the right to be accommodated, or at the least tested empirically. If we
deny it out of hand, without even investigating it, we lose something that
has stood the test of time and practice, something that may well be
effective
and of benefit.

I mentioned above, our lack of means of investigating these therapies, and
in your article you referred to the problems associated with the use of
randomised controlled trials for acupuncture. Such trials, whilst
excellent for conventional medicine, are based on the principles of that
medicine.
Namely that for a particular illness, there will be a particular cure.
However many alternative therapies work on more specific principles that
this. For one particular illness, in one particular person, and for the
particular symptoms that they have, there will be a particular cure. Thus
for a group of patients with, for example, migraine, there will be many
homoeopathic remedies, based on the patient, and the particular symptoms,
rather than just on the fact that they all have what conventional medicine
terms “migraine”. I have noticed also that at times conventional medicine
appears more of an art form than a science. For example: varying diagnosis
from separate doctors of the same complaint; various different
prescriptions for the same complaint; different responses from the same
doctor to the same
complaint, on different days. However, some alternative medicines provide
much more stringent guidelines for the diagnosing of illness and
prescribing of treatment, based on the whole patient, rather than just the
one presenting illness. Thus when looking to test such alternative
therapies, we should design our trials to take into account the different
principles underlying their use, rather than trying to make them comply
with conventional medicine’s principles. To attempt such compliance, is
not an
empirical trial. This situation may well be an example of where we do not
yet have the methods to test out hypothesis (in this case, the hypothesis
being alternative medicine.)

In your article, you create generalisations based on putting the whole
spectrum on alternative therapies in one category. Unsurprisingly, such
generalisations bear no relationship to the original alternative
therapies.

You say that “they are saying that herbs are beneficial and can do no
harm.”
I have never heard this said, although I have heard it said that some
herbs, in some situations, can be beneficial. No one could deny this (for
example the plant product digitalis). By generalising we deny the worth of
the thing
generalised, and thus lose the potential for advancement of knowledge.

Surely the empirical approach would be to study that which we do not fully
understand, in order to enrich our knowledge from it? No one could suppose
that a therapy of no use or value, could not only evolve, but also stand
the
test of time. Common sense tells us that there must be some truth in these
therapies, even if we do not yet understand it.
In conclusion, I agree with you that alternative medicines should be
investigated. Maybe then conventional medicine can learn from them. More
importantly, though, they must be investigated with a mind to their
underlying principles, so that any investigations are capable of giving a
true result that will be believed. In order to do this, we must not scorn
complementary medicine, but instead treat it as an objective hypothesis
worthy of our investigation. Only then will be able to gain from the
investigations, and only then will we be empirical.

Charlotte Jago

I enjoyed your article.

I did a psychology degree before I trained as a homoeopath, then I
joined a homoeopathic research group because I did understand the need for
empirical research, having been trained to do it.

However, I was the only person in the group that had any training in
research, I found it very difficult to operate professionally in a group
with no other expertise in research, no helpful connections to anyone who
did, no finance, no resources, no ground plan and no possibility of
getting anywhere!!! We did produce an attempt at an audit of homoeopathic
practice, presenting symptoms and some patient feedback, but the group
lurched about so much, you could see that it would (and has) taken years
before such a group will produce meaningful data. Alternative therapists
do not have research expertise, though it is improving now, they do not
have any resources, nor any funding, nor any hope of funding.

Eventually, I did a small audit of my own practice with the doctors I
work for (my day job in a local NHS GP surgery). I was trying to highlight
the very high level of mental trauma seen at the first presentation of
clients - check out the results on my web page www.sue@falcon1.demon.co.uk
I had no funding, nor any extra time set aside to do this work, so most of
it was done late at night, after my other two jobs were over and instead
of sleep. All in all, it took me and my son about eight weeks to finish
this small piece of preliminary work, to enable the local doctors to see
why I was being consulted. I could only analyse first consultation data,
as including data from follow up appointments was completely beyond my
resources then and now.

I recal a chat I had on lyghtforce.com - an alternative mailing list.
Some bright spark suggested that we should do some empirical research -
some evidence based medicine! Someone replied that they had a spare 50p,
another person donated a pound, someone else promised a fiver. All in all
we collected about £15 on the discussion line - this is not a joke - it is
the reality of alternative empirical research.

If you don't mind me suggesting this, you lot have all of the
resources and the time and the expertise to do research. Most of us
alternatives are at least twenty years behind it terms of resources and
expertise. A simple experiment such as the use of arnica after facial
plastic surgery, which was statistically significant I believe, in favour
of homoeopathic treatment, is a simple thing for you to design, organise
and carry out. How are we supposed to do this sort of research without
access to proper facilities?

I have asked many times to be allowed to visit an A&E department
for example, to work with doctors suggesting simple alternative treatments
such as the use of aconite and arnica in A&E settings, but of course
I'm shut out. We have real problems because we are on the fringe, but we
are only allowed to be on the fringe. Vested interests and patriarchal
top of the food chain behaviour will only change if patients demand it.
We can only provide evidence based medicine if we are allowed do the
research, and if we know how to do the research. The cookoo clamour you
hear is from the patients - your patients, now our patients, because they
are demanding humane medical techniques and they are desperate to be
listened to.

All of the alternative therapists I know would agree with every word
of your article, but we are below a glass floor here, with you lot dining
above us with all the resources in your hands. Thus I challenge you to do
the research, and to learn to include us (your subjects too), and to truly
hear what we are all saying. The charlatans on both ends of both our
professions will then drop off.

Regards

Sue Young RSHom