Rapid Responses to:

EDITORIALS: Jeffrey K Aronson Anecdotes as evidence BMJ 2003; 326: 1346 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

The Plural of Anecdote

Michael GOODYEAR, CANADA B3H2Y9   (21 June 2003)

Anecdotes as Evidence---not only for side effects

YL Yip   (23 June 2003)

Guidelines for reporting anecdotes

Sergio Abanades, Magí Farré   (27 June 2003)

Geoffrey Venning's classic BMJ paper

Iain Chalmers   (29 June 2003)

Anecdotes as Evidence---not only for side effects and not only for drug reactions

Gordon Morton, Michael Lim   (11 July 2003)

A call for more rigorous evaluations of the incidence and severity of drug interactions

Jeffrey J. Glasheen, Randolph V. Fugit, Allan V. Prochazka   (15 July 2003)

An anaesthetist writes:

Oliver R Dearlove   (30 July 2003)

Re: An anaesthetist writes:

Peter Morrell   (31 July 2003)

Herbal Medicines Anecdotes in Adverse Reaction Reporting

Debbie Shaw, Glyn Volans   (12 September 2003)

The Plural of Anecdote 21 June 2003
Michael GOODYEAR, Assistant Professor, Dept Medicine Dalhousie University, Nova Scotia, CANADA B3H2Y9 Send response to journal: Re: The Plural of Anecdote	Now who was it that said that the Plural of Anecdote is Not Data? Competing interests:   None declared
Anecdotes as Evidence---not only for side effects 23 June 2003
YL Yip, Family Medicine Hong Kong Send response to journal: Re: Anecdotes as Evidence---not only for side effects	I shared the same feeling as the editor in the importance of the anecdotes. However, my opinion is that such importance should be extended to other aspects of clinical medicine. As the emphasis towards big, multicentre, perennial, double blind, randomised studies increases disproportionately, we are seeing meta- analysis of 'umbrella'analysis of multiple big trials as used frequently in evidence based medicine. In a sense, we are seeing more and more forests and begin to forget our trees.Under such analyses, if something is useful, then it is really very useful, such as an analgesic. On the other hand, if it is not useful in the 'forest' sense, it does not mean that it is not useful ,or even very useful for a single tree. In documenting side effects of drugs,even a single patient is significant. However,in considering other cases, individual patient responses, seems to be totally disregarded in the final conclusion. The bigger the trial or meta-analysis it is , the less the significance of the individual. This may be okay for an armchair professor, or statistician, or a politician holding the budget. However, for the clinician, every single patient counts. Imagine this very individual patient is your mother, your wife or your son. Then it is not difficult to appreciate why our patients go more and more towards complementary medicine, even when 'statistically', many such therapies are not 'documented' to be useful. I would suggest to our editor, guidelines should be given to anecdotes, all anecdotes,not just side effects of drugs......... Competing interests:   None declared
Guidelines for reporting anecdotes 27 June 2003
Sergio Abanades, Clinical Pharmacology Resident Pharmacology Unit, Institut Municipal Investgació Mèdica, Hospital del Mar, UAB, 08003 Barcelona, Magí Farré Send response to journal: Re: Guidelines for reporting anecdotes	We completely agree with JK Aronson (21 june 2003 issue) about the need of uniformly presentation of anecdotes/case reports using a common standardised guidelines. The proposed guidelines is very complete and systematic, however in our opinion some items could be added in order to improve this format. In the drug therapy section, the drug therapy contents should include prescription drugs, over-the-counter medicines, and preparations of complementary and alternative medicines, including herbal therapies, homeopathic and ethnic preparations. Use of these therapies has increased over the last years, mainly in patients with chronic diseases taking multiple drugs, in which drug adverse reactions are more likely to appear. In the other relevant history section, it is important to include history of use of psychostimulant substances as methylxanthines, and natural or synthetic drugs of abuse (e.g. cannabis, amphetamines, hallucinogens or opiates), in addition to drinking and smoking habits. Other aspects to consider is the compulsory communication of the adverse reaction to the national pharmacovigilance centre or the local WHO drug monitoring programme prior to submit the paper for publication. Sergio Abanades and Magí Farré Competing interests:   None declared
Geoffrey Venning's classic BMJ paper 29 June 2003
Iain Chalmers, Editor, James Lind Library James Lind Initiative, Summertown Pavilion, Middle Way, Oxford OX2 7LG Send response to journal: Re: Geoffrey Venning's classic BMJ paper	I am pleased that Jeff Aronson has drawn attention to the potential importance of anecdotes as evidence (1) and that he referred to Milos Jenicek's important book on clinical case reporting (2). Another relevant publication, which I regard as a classic, is Geoffrey Venning's 1982 cohort study of published case reports of postulated adverse drug reactions (3). Venning showed that over half of these are confirmed in subsequent research as likely to be real - a very strong tribute to clinical acumen. I have been trying since the 1980s (4) to draw Venning's study to the attention of others, and to persuade them to consider replicating it, but with only limited success (5). More than two decades have passed since Venning reported his study in the BMJ. Perhaps those of your readers who are unhappy about acquiescing in the fashionable view that anecdotes are a very lowly form of evidence will help to clarify just how good or bad they are by replicating Venning's study. Iain Chalmers 1. Aronson JK. Anecdotes as evidence. BMJ 2003;326:1346. 2. Jenicek M. Clinical case reporting in evidence-based medicine. 2nd ed. London: Arnold, 1999 3. Venning GR. The validity of anecdotal reports of suspected adverse drug reactions. BMJ 1982;284:249-252. 4. Chalmers I. Evaluating the effects of care during pregnancy and childbirth. In: Chalmers I, Enkin M, Keirse MJNC, eds. Effective care in pregnancy and childbirth. Oxford: Oxford University Press, 1989: 3-38. 5. Vandenbroucke JP. In defense of case reports and case series. Ann Intern Med 2001;134:330-334. Competing interests:   None declared
Anecdotes as Evidence---not only for side effects and not only for drug reactions 11 July 2003
Gordon Morton, Specialist registrar in anaesthetics London Chest Hospital, Bethnal Green, London, Michael Lim Send response to journal: Re: Anecdotes as Evidence---not only for side effects and not only for drug reactions	Aronson’s1 excellent editorial is a timely reminder of the importance of anecdotes. Whilst we welcome his proposed guidelines, we would like to see their scope expanded. We agree that anecdotes “should explode the whole truth” and therefore we feel that such reports should not be confined to adverse events but also include incidental events which do not pose any obvious harm to the patient. An example is nitrous oxide. Nitrous oxide was used for ‘frolics’ at public demonstrations in the US during the 1840s. At one of these demonstrations, one of the participants injured himself but was oblivious to the pain. This was noticed by a dentist in the audience, Dr Wells, who had one of his own teeth painlessly extracted under nitrous oxide the next day.2 Additionally, many perceive the term “adverse event” to imply some degree of fault with the practitioners involved. This is likely to act as a disincentive to reporting. Furthermore, we agree with Yip3 that a reporting system should be expanded to other areas of medical practice. In our speciality (anaesthesia), the incidence of adverse events is low. (In a study of 18 500 intubations, only 0.3% eventually resulted in a cricothyroidotomy4). Levitan5 notes that “alternative devices (for laryngoscopy and intubation) are infrequently used…because direct laryngoscopy with pharmacological adjuncts is very successful, very fast, and has low complication rates”. Therefore “assessment of alternative techniques in a randomised manner would be technically impractical and would require such a large number of patients that it is not feasible.” In this present culture, we are always looking at our faults rather than what we do well. We feel that as much can be learnt from the good practice of others as from their errors. 1. Aronson JK. Anecdotes as evidence. BMJ 2003; 326: 1346. 2. Vandam L. in Miller RD. Anaesthesia, 5th edn. Philadelphia: Churchill & Livingstone, 2000. 3. Yip YL. Anecdotes as Evidence---not only for side effects. BMJ Rapid Responses, posted 23 June 2003. 4. Rose DK and Cohen MM. The airway: problems and predictions in 18,500 patients. Can J. Anaesth. 1994; 41: 372-383. 5. Levitan RM, Kush S, Hollander JE. Devices for difficult airway management in academic emergency departments: results of a national survey. Ann Emerg Med 1999 Jun; 33(6): 694-8 Competing interests:   None declared
A call for more rigorous evaluations of the incidence and severity of drug interactions 15 July 2003
Jeffrey J. Glasheen, Assistant Professor of Medicine University of Colorado Health Sciences Center, 4200 East Ninth Ave; Box B178, Denver, CO, USA 80262, Randolph V. Fugit, Allan V. Prochazka Send response to journal: Re: A call for more rigorous evaluations of the incidence and severity of drug interactions	JK Aronson’s call for higher standards in the reporting of adverse drug events (1) is timely. We wanted to further stress that in certain situations even more rigorous investigation of potential interactions is needed. The clinician often retrospectively asks if a sign or symptom is a masquerading drug interaction and in that situation case reports are useful and should be reported by standardized, rigorous methods. In this case, anecdotal evidence is crucial to raising awareness of potential interactions and often is the only indicator that a potential interaction exists. Still, with the ever-increasing number of medications being prescribed, the same clinician is even more frequently challenged with whether the addition of a new drug is likely to lead to an interaction. No matter how rigorously the cases are reported, the question of the incidence and severity of the interaction will always be suboptimally defined unless these clues are followed up with more rigorous evaluations. For example, in recent years we have cared for more than a dozen patients with probable interactions between warfarin and trimethoprim/sulfamethoxazole (TMP/SMX). The published literature includes numerous case reports and even a couple of prospective studies that detail this interaction. While these reports alert the clinician that the interaction may occur, they say very little about the likelihood of its occurrence in the acutely infected chronic warfarin user requiring an antibiotic. The absence of a denominator in case reports means that even the most rigorous reports ultimately offer minimal assistance in assessing the probability of an interaction. Therefore, the above clinician may ask whether another antibiotic, for example, levofloxacin, would have a lower likelihood of interacting. In reviewing the literature, he or she would find 8 case reports, which document this interaction (2,3,4). These reports alone do not provide an answer to the question of whether to choose levofloxacin or TMP/SMX to treat that particular patient (assuming equivalent efficacy). Consequently the decision is based on even more anecdotal evidence than that which Aronson refers to in his report— that being the clinician’s own subjective opinion of the likelihood of the interaction based on his or her own prior experience. We concur with Aronson’s viewpoint and would encourage more rigor in the methods used to report cases of potential drug interactions. However, we are further hopeful that his proposal will stimulate studies with more clinically relevant data regarding the incidence and severity of these suspected interactions. 1) Aronson JK. Anecdotes as evidence. BMJ 2003;326:1346. 2) Ravnan SL, Locke C. Levofloxacin and warfarin interaction. Pharmacotherapy 2001;21(7):884-5. 3) Giuseppe G, Cinetto L. Levofloxacin—warfarin interaction. Eur J Clin Pharmacol 2001;57:427 4) Jones DB, Fugate SE. Levofloxacin and warfarin interaction. Ann Pharmacother 2002;36:1554-7. Competing interests:   None declared
An anaesthetist writes: 30 July 2003
Oliver R Dearlove, Consultant Anaesthetist Royal Manchester Childrens Hospital M27 4HA Send response to journal: Re: An anaesthetist writes:	One thing that Aronson does not mention about case reports is that they allow you to cite your own personal favorites(1). But we all know that actions speak louder than words: he does cite himself four times. Although case reports still exist and thrive in an environment of adverse drug reactions, Aronson does not explain why this is so. In fact the taxonomy(2) and survival of case reports has been discussed before(3). Mason asserts that around half of case reports are about adverse effects. Case reporting should have withered before the combined onslaught of randomised control trials and the hierarchy of information, but this has not happened. We know the hierarchy of information is important because there have been claims for credit for its first use. Olkin may have a claim, however Poincare may have used a similar phrase – hierarchy of proofs – in a different context around 1915. There is however one philosophical difficulty which has not been addressed by any of our intellectual leaders. A randomised controlled trial is never ethical if harm is an outcome measure. No one would consent to trial where one outcome is known to be worse and the trial just wishes to quantify how bad it is. This means that there is a pressure against reports of harm being randomised trials. Therefore reports of harm or adverse effects are likely to be case reports or case series, and so below the line and ignored in the hierarchy of information. I also think that if the adverse effect is dire then one should have a lower threshold for reporting. Everyone is aware of the post hoc propter hoc fallacy, which is if B follows A, then B must be caused by A, and this necessarily limits discussions of causation in a case report. One can also see that reports of adverse effect are likely not be found on evidence based databases as they do not fulfil the declared criteria for entry. If one is going to use evidence based practice in the field of adverse effects, one can see that a lot of the information on harm will not be available, because they are too low in the hierarchy to be counted. So in the field of adverse effects, evidence based practice is likely to perform worse and not better than those who will attach weight to the evidence of case reports and case series. If McBride had reported his cases following thalidomide(4) today, would the referees complain, in the time-honoured fashion of petulant and dismissive BMJ assessors, that the case series suffered from not being a randomised controlled trial? A lot of this might have been avoided if the emphasis on randomised trials had been given an adequate philosophical basis then they were first introduced rather than being accorded wild adulation of the BMJ in 1994(5). A randomised controlled trial was never carried out to see if evidence based practice performed better or worse than the practices and techniques that preceded it. As the reader can see from the analysis above, when it comes to adverse effects and their reports, evidence based practice may well come out worse as the new method excludes most of the relevant evidence. Oliver R Dearlove 1. Dearlove J Dearlove O, Cortical Reflex myoclonus after propofol Anaesthesia 2002 57 383-4 2.R A Mason The case report, an endangered species? Anaesthesia 2001 56 99-102 3. Dearlove O Vashisht R The value of Case Reports Anaesthesia 2001 56 2 4. McBride WG, Thalidomide and Congenital Abnormalities. Lancet 1961 ii 1358 5. Dearlove O R Huntley A, Authors’ redefinition better but not perfect Br Med Journal 1996 313 170 Competing interests:   The author is an elected Council Member of the Royal College of Anaesthestists. The views are his own
Re: An anaesthetist writes: 31 July 2003
Peter Morrell, researcher, history of medicine, UK Send response to journal: Re: Re: An anaesthetist writes:	Sir, When Oliver Dearlove says "But we all know that actions speak louder than words: he does cite himself four times," [Dearlove’s letter] then this sounds a little like ‘the pot calling the kettle black-arse.’ A cool look at the facts shows that Aronson cites himself 4 times out of a total of 15 refs = 26.7% self-citation; but Dearlove cites himself 3 times in only four refs [=75% self-citation]; thus Dearlove cites himself 3 times more than Aronson himself. On a very similar theme, Benjamin Disraeli once said: “an author who speaks about his own books is almost as bad as a mother who talks about her own children.” [Speech at a banquet in Glasgow, 19 Nov 1873] Likewise, Auden thinking in a similar vein, said: “most people enjoy the sight of their own handwriting as they enjoy the smell of their own farts.” [Writing, in The Dyer’s Hand, 1962] Patently, therefore, Dr Dearlove conforms to the narcissistic sentiments of Auden’s remark about three times greater than Aronson, and thus there would seem to be no basis for his feeble allegation of excessive narcissism in the latter. One hopes that these points will settle this little matter in a most genial manner. Competing interests:   None declared
Herbal Medicines Anecdotes in Adverse Reaction Reporting 12 September 2003
Debbie Shaw, Research Scientist Medical Toxicology Unit, Guy's & St. Thomas' Hospital Trust, Avonley Road, London SE14 5ER, Glyn Volans Send response to journal: Re: Herbal Medicines Anecdotes in Adverse Reaction Reporting	We agree with JK Aronson [1] about the importance of single case reports and the need to improve reporting standards for adverse drug reactions. However, we fear that the label “anecdotal” could undermine Dr Aronson’s objectives by implying that the account is “unreliable or hearsay” (2) When the safety of Herbal and Alternative medicines is considered single case reports assume an even greater importance since, unlike orthodox drugs, few have undergone extensive controlled clinical trials, and information on adverse effects is largely dependent on spontaneous or anecdotal reports. Unfortunately most reports in the medical literature lack the information needed for validation. For adverse reaction reports on orthodox drugs, accurate identification of the drug is rarely an issue. However, because regulation and labelling requirements for herbal medicines vary between countries, verification of the identity of the herbs used and implicated is critical in making a valid adverse reaction report. We therefore suggest that, in addition to the guidelines listed by Aronson, herbal reports should require the following information: • Latin binomial of each ingredient and plant part [2] with type of extract (local names can be used to provide additional support) • Method used to verify identity – by botanical expert and/or laboratory analysis • Is it an herbal product (ie tablets, capsules, extracts as bought over the counter) or crude herbal formulation (ie plant material, leaves, roots etc)? • Was it prescribed by a herbal practitioner or bought over the counter? Medical professionals may not realise that different plant parts and types of extract will have different pharmacology and that these details are important in evaluating any report. In the 10 years since the Medical Toxicology Unit first started investigating suspected adverse reactions from herbal medicines, the quantity of information available has increased considerably, but there is still scope for improvement in the quality of information. Inaccurate reporting can cause much unnecessary concern and confusion. The problem of under-reporting of adverse effects has often been raised. We therefore support the recommendation by Abanades and Farre [3] of compulsory reporting of the adverse reaction to the national pharmacovigilance centre prior to publication. Reports to these centres (including herbals) are brought together by the Uppsala Monitoring Centre who have clearly stated the need for improved reporting standards [2]. Debbie Shaw. Glyn Volans Medical Toxicology Unit, Guy’s & St Thomas’ Hospital Trust London 1 Aronson JK Anecdotes as evidence. BMJ 2003; 326; 1346 2. New Oxford Dictionary of English (1988) Oxford University Press, Oxford, UK 3 Farah MH, Edwards R, Lindquist M, Leon C, Shaw D. International monitoring of adverse health effects associated with herbal remedies. Pharmacoepidemiology and Drug Safety 2000; 9:105-112 4 Abanades S, Farre M. Guidelines for anecdotes might include more information. BMJ 2003; 327:290 Competing interests:   None declared