Rapid Responses to:
|
|
can we finally see the light?
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Another conclusion
- Milos Zarkovic (2 July 2001)
-
Not all placebos are equal
- Peter Ewing (2 July 2001)
-
Acupuncture and neck pain
- Arthur Rifkin (5 July 2001)
-
A critique of recent acupuncture trial methodology
- George Lewith, Peter White (6 July 2001)
-
Slight trouble with figures...
- Peter Morrell (11 July 2001)
-
Using Citation Figures Carefully
- Reinhard Wentz (15 July 2001)
-
Diagnosis before treatment
- C Chan Gunn (17 July 2001)
-
confusing analysis
- David Lewis (17 July 2001)
-
Preferring to shoot the messenger?
- Peter Morrell (19 July 2001)
-
Plain Mister
- Reinhard Wentz (20 July 2001)
-
Authors`reply
- Dominik Irnich (3 August 2001)
-
Randomised trial of acupuncture compared with conventional massage and "sham" laser acupuncture
- George Lewith, Peter White (14 August 2001)
-
Results of reanalysis of Irinich trial using statistical methods of greater efficiency.
- Andrew Vickers (12 September 2001)
|
|
|||
|
Milos Zarkovic, Consultant in endocrinology Institute of Endocrinology, KCS, Belgrade, Yugoslavia
Send response to journal:
|
To the Editor, If sham laser acupuncture is only a placebo, and its effect is equal to the acupuncture, which is better then massage, then alternative conclusion is possible: massage is harmful. This point to the necessity of revaluating accepted diagnostic and therapeutic procedures, especially the ones that have been long in use and probably never adequately evaluated. It also points to the problem of “positive thinking” in medical literature. Negative results are not welcome. |
|||
|
|
|||
|
Peter Ewing, GP Principal Crieff Health Centre
Send response to journal:
|
It is very difficult to devise a sham treatment to act as a placebo when attempting to assess the effects of physical therapies such as acupuncture. I congratulate the authors on a valiant and ingenious attempt to do so, and going to considerable trouble to assess patients confidence in the sham therapy. However, I do not believe the results justify their conclusion that acupuncture is an effective short term treatment for patients with chronic neck pain, if by 'effective' we mean 'better than placebo'. Not all placebos are equal; comparative studies of different coloured placebo tablets show that they have markedly differing effects (1). We could conclude from these studies that the food dye in the tablets has a specific pharmocological effect, but this would have no more validity than the authors conclusion that acupuncture is effective. A more plausible explanation is that certain colours of tablets are more potent placebos than others. It may well be that this trial is merely comparing three placebos of differing potency. To my mind, there appears to be a hierarchy in the impressiveness of suggestion for each therapy. Massage is simple and 'low- technology'; the sham treatment appears more complex and 'high- technology'. Acupuncture, however, has the dual advantages of being invasive and, in many peoples eyes, 'beyond technology', having centuries of chinese mysticism behind it (burning moxa herbs on the needle, as is commonly practiced no doubt enhances this suggestion still further). The power of suggestion behind these treatments is complex and difficult to measure on a four item questionnaire, especially if it was administered before the patients had actually undergone the treatment. One other way to try to tease out whether or not benefit is due to placebo effect is to look at the time scale of the improvement. This has been done comparing antidepressants with placebo for depression (2) (3). As expected, the placebo produces a rapid onset but short-term improvement, whereas antidepressants take a few weeks to give benefit but the improvement is more sustained. Given the very short term benefits of acupuncture, and the greater suggestion effect of an invasive, mystical technique, I believe that the main message from this paper is that acupuncture is a placebo, but a very good one. (1)de Craen AJM, Roos PJ, de Vries AL, Kleijnen J. Effect of colour of drugs: systematic review of perceived effect of drugs and their effectiveness. BMJ 1996;313:1624-6. (2)Rothschild R, Quitkin FM Psychother Psychosom 1992; 58(3-4) 170-1 (3)Quitkin FM et al. Am J Psychiatry 1993 Apr; 150(4) 562-5 (3) |
|||
|
|
|||
|
Arthur Rifkin Hillside Hospital, New York
Send response to journal:
|
The authors conclude that acupuncture is effective treatment for chronic neck pain. I don't understand how they arrive at this conclusion, since acupuncture did not differ from sham laser acupuncture. Their only justification for this conclusion is their statement that sham laser may have had a therapeutic effect from palpating acupuncture points. This seems far too speculative to support their conclusion. They designed the experiment with the sham laser as a placebo. They should not turn around after they collect their data and say sham laster is not placebo. In their statistical analyses they report pairwise comparisons of the quantitative data, but don't report the analysis of variance for all three treatments as a whole, which should show statistical significance before proceeding to pairwise comparisons. Further, they don't tell us how they adjusted for multiple comparisons. |
|||
|
|
|||
|
George Lewith, Senior Research Fellow, Research Physiotherapist University of Southampton, Peter White
Send response to journal:
|
Sir We have been involved in the development of clinical trial methodology within the field of acupuncture for over 20 years1. We feel that Irnich et al's study as reported in the BMJ has methodological faults which we hope to address in the context of our randomised controlled trial that evaluates the effects of acupuncture versus a control treatment in chronic mechanical neck pain and which will be completed later this year. We note that there is a much fuller report of this trial on the BMJ website. This is effectively a large but negative study for acupuncture according to the predefined primary outcome measure. However, the treatment groups are inadequately balanced for primary diagnosis with respect to myofascial pain; there were substantially fewer patients with this diagnosis in the acupuncture group and acupuncture may be particularly effective in this syndrome. We also have too little information about the acupuncture and massage treatments to be able to reproduce this study accurately. For instance we are unaware of which type of traditional Chinese medicine was employed and we do not know if the massage involved any stimulation over acupuncture points. The choice of placebo/controls in acupuncture studies is a vexed and probably unresolvable issue. Cummings suggests that sham laser is "a good choice"2, but this is not substantiated by any evidence provided in other studies although it appears to be credible in the context of this study. Cummings' support for the "placebo needles" which we are currently investigating in our research group in Southampton, is based on two small studies from one research group3;4. We believe it is questionable, particularly without detailed knowledge of the type of acupuncture provided, whether the "placebo needles" described could be used effectively and credibly in this study. Irnich et al have also suggested that they measured ‘patients beliefs’ about treatment using a scale that they attributed to Vincent5. We have used this credibility scale ourselves in a number of studies6. The scale they used was originally designed and validated by Borkovec and Nau7 and was intended to measure credibility of treatment, not patients beliefs. The treatment provided by Irnich et al was limited to 5 acupuncture sessions, possibly too little for effective treatment8 . Outcome was not assessed continuously but every week, this is unusual in a study involving acupuncture and chronic pain9;10. The primary outcome measure does not appear to have been piloted or validated prior to its use in the study. It involved change in pain related to motion which was tested by one blinded assessor. Unfortunately, no details are given as to how this was effected. Were patients asked to repeat a movement several times? Was a standardised set of instructions given? How hard were they instructed to push? No information on intratester reliability was offered and therefore the value of the primary outcome must be questioned. Unfortunately we believe this is a poorly constructed study with unstratified treatment groups and poor primary outcome. It uses a reasonable but under-evaluated placebo/control with limited and unclearly described treatments. As a consequence it does not take us significantly further forward in attempting to evaluate the specific effects of acupuncture. George Lewith MD FRCP Reference List 1. Lewith GT,.Machin D. On the evaluation of the clinical effects of acupuncture. Pain 1983;16:111-27. 2. Cummings M. Commentary: Controls for acupuncture - can we finally see the light? British Medical Journal 2001;322:1578. 3. Streitberger K,.Kleinhenz J. Introducing a placebo needle into acupuncture research. Lancet 1998; 352:364-5. 4. Kleinhenz J, Streitberger K, Windeler J, Gussbacher A, Mavridis G, Martin E. Randomised clinical trial comparing the effects of acupuncture and a newly designed placebo needle in rotator cuff tendinitis. Pain 1999;83:235-41. 5. Vincent C. Credibility Assessment in Trials of Acupuncture. Comp.Med.Res. 1990;4:8-11. 6. Vincent C,.Lewith G. Placebo controls for acupuncture studies. Journal of the Royal Society of Medicine 1995;88:199-202. 7. Borkovec T,.Nau S. Credibility of Analogue Therapy Rationales. J.Behav.Ther.and Exp.Psychiat 1972;3:257-60. 8. Ezzo J, Berman BM, Hadhazy V, Jadad A, Lao L, Singh BB. Is acupuncture effective for the treatment of chronic pain? A systematic review. Pain 2000;86:217-25. 9. Lewith G, Field J, Machin D. Acupuncture compared with placebo in post-herpatic pain. Pain 1983;17:361-8. 10. Dowson D, Lewith G, Machin D. The effects of acupuncture versus placebo in the treatment of headache. Pain 1985;21:35-42. |
|||
|
|
|||
|
Peter Morrell, Hon Research Associate, History of Medicine Staffordshire University, UK
Send response to journal:
|
Sir, "We have been involved in the development of clinical trial methodology within the field of acupuncture for over 20 years...Lewith, G T., Machin D. On the evaluation of the clinical effects of acupuncture. Pain 1983; 16: 111-27." [1] Well, it is a great pity George Lewith cannot count so well [1] - 1983 to 2001 is NOT "over 20 years" but in fact, just 18. Furthermore, his letter, not unusually [2], gives a rather high citation to his own previous publications. In this case, it reaches the dizzy heights of 40%. This self-citation rate, it must be said, is a tad on the high side; 1-5% might seem more respectable. I'm sure even Dr Lewith will agree that reading any book or article with 40% references to the author's own publications, must make one justifiably suspicious that such is a narrow field, constructed by only a few and presided over by a small team of workers. It is depressing to see that this practice has become quite widespread within this so-called research field of complementary medicine, where all the main figures seem to keep citing their own or each other's work [3]. I fail to see how such a practice can attract any merit to this topic or these researchers. It brings more like shame than merit. Nature thrives on diversity. Self-referencing invalidates the pluralistic basis of academic discourse and in effect arrogantly declares that the opinions of a few are superior to those of many; or, that one is blissfully ignorant of, or despises, the work of others in the same field. Either way, such is surely a contemptible attitude. It should be universally condemned as a form of intellectual deceit, and casts a shadow of doubt over every word they publish. I wish they would seriously contemplate the bad impression imparted by such behaviour. Sources [1] BMJ letter, A critique of recent acupuncture trial methodology, George Lewith, Peter White, Senior Research Fellow, Research Physiotherapist University of Southampton (6 July 2001) http://www.bmj.com/cgi/eletters/322/7302/1574#EL6 [2] Lewith in Australia http://www.mja.com.au/public/issues/172_03_070200/lewith/lewith.html 4 out of 18 refs [22% self-citation] Other Lewith BMJ articles: http://www.bmj.com/cgi/content/full/322/7279/154 4 out of 23 refs [17.4% self-citation] http://www.bmj.com/cgi/content/full/322/7279/131 1 out of 12 [8.3% self-citation] http://www.bmj.com/cgi/content/full/309/6947/103 1 out of 11 [9.1% self-citation] average here is 10 out of 64 = 15.6% [still way too high] [3] Ernst article: http://www.bmj.com/cgi/content/full/321/7269/1133 one of the very worst examples of high self-citation in this field; 16 to his own work out of 24 refs [66.7% self-citation] probably 10 times 'the norm', by no means untypical of his work and pretty outrageous. |
|||
|
|
|||
|
Reinhard Wentz, Medical Librarian Imperial College, Chelsea & Westminster Hospital
Send response to journal:
|
Sir, Dr Lewith's team published a paper in 1981 on research methods in acupuncture (Lewith GT, Machin D. A method of assessing the clinical effects of acupuncture. Acupunct Electrother Res. 1981;6:265-76), which suggests that his team's research interests in this subject go back to at least 1980, well 'over 20 years' ago. Dr Lewith is one of the authors of at least 70 papers published between 1981-2001. He is the first author of some 50 papers, twelve of which cite more than ten papers (range 11-115). In these papers he cites himself on average 2.25 times (range 0-5) which suggests a mean self- citation rate of 4.8%. This is well within the range of what Peter Morrell considers ethical in these matters. I invite Peter Morrell to apologise to Dr Lewith. Reinhard Wentz |
|||
|
|
|||
|
C Chan Gunn, President and Clinical Professor Institute for the Study and Treatment of Pain
Send response to journal:
|
We commend Irnich et al for their effort, and for trying to account for the placebo effect. Whatever the merits of their study, there is a more important question: Why examine traditional Chinese acupuncture when its application varies so widely from one practitioner to another? Effective treatment of pain, like any symptomatic complaint, requires a medical diagnosis. Traditional acupuncture does not include a medical examination, and so cannot provide a correct diagnosis. The study notes that acupuncture is more effective against the allodynia in myofascial pain syndrome (1). This agrees with our observations at the Institute for the Study and Treatment of Pain, where virtually all patients consistently show physical signs of peripheral neuropathy (2). These non-nociceptive signs do not appear on routine laboratory tests, but require a trained examiner. Needle therapy seems to work through the peripheral nervous system because these clinical signs disappear following effective treatment (3). We have found needling most effective when applied to tender, shortened muscles in affected segments (4). We call our technique Intramuscular Stimulation (IMS) (5). Because IMS dry needling resolves clinical signs that are objective, how relevant are subjective reports, such as symptoms and placebos? Meaningful research on chronic pain requires a distinction between ongoing nociception and increased sensitivity in allodynia. The problem in this debate stems from the absence of proper examination and diagnosis. Any treatment is "sham" without proper examination and diagnosis. C. Chan Gunn, MD, PhD (hon.), DSc (hon.) References 1. Gunn CC. Neuropathic Myofascial Pain Syndromes, Chapter 28 of Loeser JD et al, Bonica's Management of Pain, 3rd ed. Lippincott Willams & Wilkins, 2000 2. Gunn CC, Milbrandt WE. Early and Subtle Signs Low-Back Sprain. Spine Vol. 3 No. 3, 1978
3. Gunn CC. Acupuncture and the peripheral nervous system -- Chapter 9 of Flishie J, White A. "Medical Acupuncture: A Western Scientific Approach" Churchill Livingstone, 1998 4. Gunn CC. Radiculopathic Pain: Diagnosis and Treatment of Segmental Irriation or Sensitization. Journal of Musculoskeletal Pain, Vol. 5(4) 1997 5. Gunn CC. The Gunn Approach to the Treatment of Chronic Pain 2nd ed. Churchill Livingstone, 1996 |
|||
|
|
|||
|
David Lewis, General Practitioner Vauxhall Primary Health Care, Limekiln Lane, Liverpool, L5 8XR
Send response to journal:
|
I am confused by the analysis in Irnich et al's randomised trial of acupuncture.[1] They say that analyses were based on intention to treat analysis. The figure showing progress through the trial shows that 56 patients were randomised to receive acupuncture, 60 massage, and 61 sham laser. Yet for all the outcomes they analyse 51 patients, 57 and 57 respectively. Even if they include only those patients who completed the trial (which surely is not the same as intention to treat?), the numbers according to their figure should be 49 for acupuncture, 59 for massage, and 57 for sham laser. Is the figure wrong or did some patients get left out? yours faithfully David Lewis 1. Randomised trial of acupuncture compared with conventional massage and "sham" laser acupuncture for treatment of chronic neck pain. Dominik Irnich, Nicolas Behrens, Holger Molzen, Achim König, Jochen Gleditsch, Martin Krauss, Malte Natalis, Edward Senn, Antje Beyer, Peter Schöps, and Mike Cummings BMJ 2001; 322: 1574 |
|||
|
|
|||
|
Peter Morrell, Hon Research Associate, History of Medicine Staffordshire University, UK
Send response to journal:
|
Sir, It is a great pity that Dr Wentz [1] seeks to personalise an issue, which is mostly of general importance. I can assure him that I have no personal stake in whether the point of view I expressed was right or wrong. My life does not depend upon it and I am not especially bothered either way; I merely pointed to what seems like a genuine pattern [2]. I also know I am not alone in forming this impression of this field. I am sure those who wish to make such a trend disappear, will have their say, whether they give their reasons or not, buried under a welter of obfuscating figures, or otherwise. "Dr Lewith is one of the authors of at least 70 papers published between 1981-2001. He is the first author of some 50 papers, twelve of which cite more than ten papers (range 11-115). In these papers he cites himself on average 2.25 times (range 0-5) which suggests a mean self- citation rate of 4.8%." [1] If Dr Wentz can provide the data upon which he bases this reduced global average of self-citation for Dr Lewith, which I believe has not been calculated correctly, then I will reconsider the points I have made. Clearly, in order to reduce a global average from 15.6% to 4.8%, a large number of very small figures must have gone into the mix, especially if, as in this case, the range extends upwards to 40%. As I do not believe the sample I used was very untypical, being randomly and un-prejudicially selected, I find it hard to believe Wentz’s low figures. He should list as ‘one’ - as I did - every time the name Lewith appears as an author in an article cited, regardless of how many co-authors there are. He should indicate the total number of references cited in ALL the 70 articles he mentions [not just a selection] and the number of those that contain the name Lewith as a co-author. Then convert that into an average percentage. I think if he does that, the average will be higher than 4.8%. At no point did I claim that Dr Lewith had done what he does deliberately. It must be very easy to quote yourself all the time, without even realising it. But it is a sloppy practice. To demand apologies is therefore decidedly premature and would of course require that a deliberate error of fact or interpretation had been made, from my side, compounded no doubt with malicious intent. As no such requirements apply in my case, I do not therefore see much reason to offer an apology. Indeed, if anyone should be condemned to stumble forwards armed with apologies then we might look instead to those who use this loathsome self- citation habit so liberally in their publications. Of course self-citation rates of 40-70% deserve to be denounced as unjustifiable and disreputable, and I am astonished that Dr Wentz is so reluctant to condemn the practice himself. If we also include regular citation of a small coterie of co-workers, then he can rest assured, these figures would be even higher. I did not actually use Dr Wentz’s word ‘ethical’; in fact, the word I used was 'respectable' and I see no need to apologise for questioning how a 40% self-citation rate can attract much respect. Indeed, as I previously indicated, 15.6% is still on the high side. If I now introduce the word 'decent', I would say that as a rule-of -thumb 'below 10%' is a 'decent' average self-citation rate, that is 1 in 10 listed references. 5% is better, but I would accept 1 in 10 as quite 'decent'. Certainly, high self-citation rates are flagrantly unethical and it is worth readers briefly contemplating why that is. The basis for anyone using references in their work is to use the pooled resources of the academic community to buttress their views, research methods, and to interpret their findings, measured, as they so often are, against what their predecessors have found. This is the established basis for article writers quoting the works of others. It shows that one has read all the most relevant literature preceding one’s work and digested its merits. To which I would also add the following very wise sentiment: "the urge to ensure the survival of a point of view with which an academic has become identified is often a stronger driving force leading to bias than is any possible financial gain or loss." [3]. Self-referencing looks uncannily like a subtle means to ensure "the survival of a point of view with which an academic has become identified." I would therefore say that a more scrupulously balanced use of citations is an important way to avoid accusations of bias in publications. Therefore, those who mostly reference themselves, and a few carefully selected cronies, and who habitually exclude a range of other workers publishing in the same field, certainly open themselves up to some accusation of bias – not by me, Dr Wentz, but by anyone. Such people are certainly deviating from the unwritten academic norm and such behaviour is anomalous precisely to the degree that it seems to treat the academic community with contempt. I am sure that anyone viewing this matter in a neutral and detached manner would have to agree with these perfectly valid points, which are indeed the points that inspired my first email on this matter. I do not know Dr Lewith, am not in any way associated with him, and nor do I have much deep interest in his work. I merely pointed to what seemed to be an unfortunate pattern. I have no money on any runner in this race and idly watch it in a detached manner; if I can be proven wrong in what I have said, then I will gladly apologise. Instead of petulantly demanding apologies, perhaps Dr Wentz will now supply all the requested data, and as an academic librarian, explain why he approves and defends such anomalous referencing behaviour. Sources [1] BMJ letter, Reinhard Wentz, Using Citation figures Carefully, 15 July 2001 http://www.bmj.com/cgi/eletters/322/7302/1574#EL8 [2] BMJ letter, Peter Morrell, Slight trouble with figures..., 11 July 2001 http://www.bmj.com/cgi/eletters/322/7302/1574#EL7 [3] BMJ 1995; 311:688 (9 September), Letters, Ethical imperative to publish extends to academics too, David Horrobin http://www.bmj.com/cgi/content/full/311/7006/688/a |
|||
|
|
|||
|
Reinhard Wentz, Medical Librarian, Imperial College Chelsea & Westminster Hospital
Send response to journal:
|
Sir, I do not intend to continue the debate with Paul Morrell about proper use of citation frequency calculations, but should like to point out that I am just plain 'Mr.'. With kind regards Reinhard Wentz |
|||
|
|
|||
|
Dominik Irnich, Anaesthesist University of Munich
Send response to journal:
|
Sir We appreciate very much the opportunity to discuss by electronic letters on the BMJ web site. When we analysed the results of our trial [1] we were aware that they would be subject to a controversial discussion. This was foreshadowed in the comment by Mike Cummings [2]. We agree that the results of our trial may be interpreted in different ways, but we are surprised that the statistical approach and the study methodology have been criticised. Analysing the comments we believe that some points of criticisms are due to superficial reading of the paper, while others are due to details omitted from the paper because of space restrictions. In the latter case missing details should not be interpreted in a negative way. It would be preferable to contact the authors before commenting on unknown details. Our reply in detail: To Dr. Cummings:
To Dr. Zarkovic:
This is not to be rejected a priori and we discussed this conclusion
extensively in the study committee. We rejected this conclusion for
different reasons:
We agree with Dr. Zarkovic that "positive thinking" in the medical literature is a problem. We discussed clearly the limitations of our study, and do not feel concerned by this. To Dr. Ewing:
We need to remember, that a real placebo in clinical trials has to be physiologically inert and needs to be applicable double-blind. Thus, there is no real placebo which can be used in acupuncture research, and also in many, many other therapies (physiotherapy, physical therapies, psychotherapy, patient education, surgery and so on). Hence, in all these therapies we will never be able to prove efficacy by clinical trials, and it is always easy to argue that differences to any sham procedure are due to an enhanced placebo effect. Dr. Ewing stated that differences between therapies might be due to a hierarchy of impressiveness of suggestions for each therapy, but suggestibility seems not to be a major factor in modifying the power of a placebo [7]. However, patients` expectations of treatment effects clearly influence their responses [8]. We do not believe that patients expectations influenced our results, because the credibility assessment of treatments did not show differences between the three groups before and during treatment. The only exception was a small difference after treatment between therapies which went in the same direction as outcome measures, indicating an influence of achieved effects for each group. But even after treatment there was a relatively high credibility for sham laser and massage. (Data have been presented to the reviewers and editors). The effects of placebos in pain research are complex and they are influenced by many known and unknown factors. Dr. Ewing cites some of the most interesting studies. It is a fact that we do not understand in detail how placebos work, we do not even know if it exists [9]. To explain our results by stating that acupuncture in our trial is a very good placebo, which works better than an established treatment or a sham procedure with similar credibility, does not challenge our conclusion. To conclude that acupuncture is not effective regarding the clinically relevant effects we observed in patient with a long history is not justifiable. To Dr. Rifkin:
We think you would agree that it is difficult to explain how placebo works in detail and why sometimes there are strong effects and sometimes only small? Regarding the statistical analysis, we have not included in the tables the p-value for the overall test in the analysis of variance including all three treatments. This test should show statistical significance before proceeding to pairwise comparisons, if the overall significance level should be less than or equal to 0.05 for one hypothesis. For the main outcome measure, the overall test resulted in a p-value of 0.0104 for all patients, p=0.0027 for the subgroup with myofascial pain syndrome and p=0.0265 for the subgroup with pain > 5 years. The last point of criticism from Dr. Rifkin is unnecessary, since Dunnetts´s test includes the adjustment for multiple comparison. To Dr. Lewith and Mr. White:
Our study has undergone an extensive review process by the BMJ. It has been reviewed by an internationally renowned specialist in clinical trial methodology and by a statistician. The study design has also been examined by international reviewers as part of a tender for research sponsorship from the German Ministry for Education and Research (BMBF, formerly BMFT) and was found to be worthy of sponsorship. In contrast to previous studies, our bi-center trial is characterised by a large sample size, adequate outcome measures evaluated by blinded observers, blinded patients, sham control and alternative treatment control, individual acupuncture treatment by more than one licensed acupuncturist, data analyses performed by an independent institution, follow-up assessments and documentation of drop-outs and adverse events. Taken together it will score highly on the well-known scales assessing the methodological quality of clinical trials (e.g. Jadad Scale, Oxford Pain Validity Scale). It is said that our study is negative for acupuncture. We do not accept this interpretation (see reply to Dr. Zarkovic, Dr. Ewing and Dr. Rifkin). Furthermore, the arguments of Lewith and White are contradictory: They say that the main outcome measure is not valid, but they accept it to say that the result of the trial is negative. They should be logical: If an outcome measure is not valid than the result is not valid ! In detail:
We are looking forward to the results which Dr. Lewith plans to present later this year when his neck pain study is completed, and we do not wish him a similar superficial comment which does not respect the fair -play necessary for a scientific discussion. To Dr. Morell and Mr. Wentz:
To Dr. Gunn:
To Dr. Lewis:
With kind regards Dr. D. Irnich
M. Krauss, Dipl. Stat.
Dr. N. Behrens
Dr. A. König
Dr. H. Molzen
Reference List 1. Irnich D, Behrens N, Molzen H, et al. Randomised trial of acupuncture compared with conventional massage and "sham" laser acupuncture for treatment of chronic neck pain. BMJ. 2001;322:1574-1577. 2. Cummings M. Commentary: Controls for acupuncture-can we finally see light? BMJ 2001;322:1578. 3. Streitberger K, Kleinhenz J. Introducing a placebo needle into acupuncture research. Lancet 1998;352/9125:-365 4. Park J, White A, Lee H, Ernst E. Development of a new sham needle. Acupunct Med 1999;17/2:-112 5. Aker PD, Gross AR, Goldsmith CH, Peloso P. Conservative management of mechanical neck pain: Systematic overview and meta-analysis. BMJ. 1996;313/7068:-1296 6. Braverman DL, Schulman RA. Massage techniques in rehabilitation medicine. Phys Med Rehabil Clin N Am. 1999;10:631-49 7. Evans FJ. Expectancy, therapeutic instructions and the placebo response. In: White L, Tursky B, Schwartz G. Placebo-Theory and Research. New York, Guilford Press, 1985:215-228. 8. Turner JA, Deyo RA, Loeser JD, Von Korff M, Fordyce WE. The importance of placebo effects in pain treatment and research. JAMA 1994;271/20:1609- 1614. 9. Hrobjartsson A, Gotzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N Engl J Med 2001;344:1594-1602. 10. Willer JC, Roby A, Le BD. Psychophysical and electrophysiological approaches to the pain-relieving effects of heterotopic nociceptive stimuli. Brain 1984;107:1095-1112. 11. Sandkühler J. The organization and function of endogenous antinociceptive systems. Prog Neurobiol 1996;50/1:-81 12. Irnich D. Demands, possibilities and limits of evidence-based evaluation in acupuncture. Dtsch Z Akupunkt 2000;43:117-125. 13. Vincent C. Credibility assessment in trials of acupuncture. Complement Med Res 1990;4/1:8-11. 14. Dowson DI, Lewith GT, Machin D. The effects of acupuncture versus placebo in the treatment of headache. Pain 1985;21:35-42. 15. Lewith GT, Field J, Machin D. Acupuncture compared with placebo in post-herpetic pain. Pain 1983;17:361-368. 16. Ezzo J, Berman B, Hadhazy VA, Jadad AR, Lao L, Singh BB. Is acupuncture effective for the treatment of chronic pain? A systematic review. Pain 2000;86/3:217-225. |
|||
|
|
|||
|
George Lewith, Senior Research Fellow and Research Physiotherapist University of Southampton, Peter White
Send response to journal:
|
Sir, We have read with interest the discussion of the recent paper by Irnich et al 1 and their thoughtful reply. Dr Irnich suggests that some of the criticisms are due to detail omitted from the paper because of space restrictions, and we agree that this is an important issue. We are not sure, however, if this argument can also be used for the ‘electronic’ version of the BMJ, as clearly much more detail is given here. Perhaps the BMJ editor or the paper's authors could clarify this for us? Even though Irnich et al's paper is far from perfect, we believe it is an important study. We have no doubt that it will score highly on the Jadad scale, but this is simply a measure of bias and not a reflection of the quality of a trial per se. Scientific progress is generally a slow process which is partially dependent on peer review, and it is vital that we are able to learn from each other's constructive criticisms. We must assure Dr Irnich that our comments were made in an attempt to further the process of constructive scientific discussion after thoughtful consideration. With regard to Dr Irnich's comments on placebo, in particular the needle developed by Streitberger 2 which Irnich felt was not a real placebo because it could not be applied double blind. The definition of a placebo is that it is inactive, harmless and given "to please the patient" 3 i.e. that it has no specific therapeutic effect 4. The fact that the practitioner cannot be blinded during its use is an issue relating to bias, not whether the Streitberger needle is a placebo. A single blind, placebo controlled trial would conventionally require that the patient is unable to detect the difference between verum and placebo. We feel, therefore, that Dr Irnich may be a little premature in his castigation of this needle, particularly in the light of Streitberger’s early results. Whether the Streitberger needle is truly physiologically inert requires further investigation. We feel, however, that the use of this needle is not very far removed from the reality of acupuncture practice, and probably represents one of the better options that we have at this present time, although its use may be limited by the fact that it cannot be applied to all acupuncture points. In our trials thus far, we have found the technique can be performed easily and convincingly. With regard to the outcomes used, we have not suggested that visual analogue scales (VAS) are not valid in their own right. Our concern is centred around the issue of how this was conducted. Combining VAS with range of movement in this way, creates a new, previously unvalidated, process of outcome measurement. We feel that it is scientifically legitimate to question the validity of the outcome process for the reasons stated in our original letter. Whilst Dr Irnich has answered some of the questions we have raised in this regard, there was not a sufficient description of this in the published methodology or the subsequent correspondence. Questioning the validity of the primary outcome does not in any way preclude us from commenting on the overall results of the study as presented by the authors. The results clearly suggest that acupuncture treatment was not superior to the ‘placebo’ control, and we therefore feel justified in concluding that this trial was negative for acupuncture, given those results. We feel that details of the intervention should be clear in the study methodology and that it is unacceptable to bury this within the reference section. Reproducibility must be a prerequisite for any controlled trial. The quality of scientific reporting within the field of acupuncture has improved significantly over the last 20 years, both in our own and other studies. Our quoted research relates to the process of outcome measurement, not our inadequate reporting of treatment protocols over 15 years ago. We also fail to see why it might be ethically acceptable to use a placebo treatment for five sessions but not for six or eight. Surely an intervention is either ethically acceptable or it is not. Lastly, with regard to the sham laser, we do accept Dr Irnich’s statement that “there is always a first time”, although it is perhaps a shame that such a large and important study has utilised a previously unevaluated control. We would have expected some appropriate and published pilot work prior to such a large scale study. In conclusion we feel that this study would be difficult, if not impossible, to reproduce from the information that we currently have available to us. We are also unsure of the robustness of the outcome measures and the placebo employed, and are therefore unclear how much value we should place on the study's conclusions. Reference List 1. Irnich D, Behrens N, Molzen H, Konig A, Gleditsch K, Krauss M et al. Randomised trial of acupuncture compared with conventional massage and sham laser acupuncture for treatment of chronic neck pain. British Medical Journal 2001;322:1574-7. 2. Streitberger K,.Kleinhenz J. Introducing a placebo needle into acupuncture research. Lancet 1998; 352:364-5. 3. Webster's Dictionary. The new international Websters dictionary of the English language. Florida: Trident Press International, 1995. 4. Lynoe N. Is the Effect of Alternative Medical Treatment Only a Placebo Effect. Scand J.Soc.Med. 1999;18:149-53. |
|||
|
|
|||
|
Andrew Vickers, Assistant Attending Research Methodologist Memorial Sloan-Kettering Cancer Center
Send response to journal:
|
Editor Irnich et al reported acupuncture superior to massage though not to sham acupuncture for neck pain [1]. This suggests that acupuncture is effective but that this is due to a placebo effect. The authors compared improvements in pain between groups using pairwise t-tests. This statistical method is of questionable efficiency. Firstly, It has been amply demonstrated that regression analysis including baseline score as a covariate has greater statistical power than comparison of change [2, 3]. Secondly, each pairwise comparison in a three group trial ignores one- third of the patients; such comparisons are thus underpowered compared to regression modelling of all data. Moreover, analysis of change scores, such as that reported by Irnich, favors the group with worse baseline pain scores (in this case, sham acupuncture) due to regression to the mean [4]; conversely, analysis of follow-up scores alone favors the group with lower baseline pain. Regression analysis gives similar results regardless of the direction of baseline imbalance. Dr Irnich kindly provided me with raw data for reanalysis. To compare the effects of treatment on pain score one week after treatment (the prespecified primary outcome measure) I undertook a linear regression analysis. The covariates used were baseline score, treatment group and the following diagnostic variables: somatization, depression, history of trauma, pain localization, pain site (neck / other), pain type (relived by heat: yes or no), concomitant symptoms, neurological findings, diagnosis (myofascial v other). Treatment was coded as two dummy variables: use of any acupuncture technique and use of true acupuncture. Acupuncture, sham laser and massage were thus coded 1, 1; 1, 0 and 0, 0 respectively. This analysis estimates the effects of acupuncture needling and placebo effects of acupuncture independently. Backwards stepwise regression was used where a p value of 0.05 was the criterion for keeping a variable in the model. Analyses were conduced on Stata 6 (College Station, Texas). Depression, baseline score and use of true acupuncture remained in the final regression model. The interpretation is that acupuncture needling is of benefit in neck pain and that this is not attributable to a placebo effect. Patients receiving true acupuncture had improvements in pain, adjusted for baseline score and presence of depression, of 11.5 points (95%CI 3.5, 19.5; p=0.005) more than those in the massage and sham groups. Restricting the analysis to patients who received either sham laser or true acupuncture, acupuncture led to a reduction in pain score, adjusted for baseline pain, of 9.4 points greater than sham (95%CI 0.9, 18.0; p=0.031). These results differ substantively from those reported in the original paper. Andrew Vickers
References 1 Irnich D, Behrens N, Molzen H, Konig A, Gleditsch J, Krauss M, Natalis M, Senn E, Beyer A, Schops P. Randomised trial of acupuncture compared with conventional massage and "sham" laser acupuncture for treatment of chronic neck pain. BMJ 2001;322(7302):1574-8 2 Frison L, Pocock SJ. Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat Med 1992; 11:1685-1704 3 S Senn. Statistical Issues in Drug Development. Chichester: John Wiley 1997. 4 Bland JM, Altman DG. Regression towards the mean. BMJ 1994;308(6942):1499 |
|||
