Case reports of suspected adverse drug reactions—systematic literature survey of follow-up
BMJ 2006; 332 doi: https://doi.org/10.1136/bmj.38701.399942.63 (Published 09 February 2006) Cite this as: BMJ 2006;332:335
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Loke et al in their survey of case reports concluded that they are of
limited value. They made seven references to my earlier work on this
subject (1) with the implication that my findings lent support to the
predictive accuracy of case reports as cited by Vandenbroucke(2). In
fact, I had emphasized the problem of false alarms as indicated in the
title of the paper, confirmed also by the accompanying editorial “Crying
wolf on drug safety”.
It is important to distinguish between Type A (augmented) and Type B
(bizarre) reactions. (Rawlins classification, from Textbook of Adverse
Drug Reactions (3)). Case reports of Type A reactions (related to the
pharmacological action of a drug) may have convincing evidence of validity
with no need for randomized controlled trials for verification. In my
study, 11 of 12 case reports which remained unvalidated during 20 years of
follow up were Type B reactions (not to be expected from the known
pharmacology). In my later series of five BMJ articles on adverse
reactions I dealt in detail, in part IV, with methods of verification (4).
While there is a clear need for better post marketing surveillance, case
control studies may be more efficient for verifying rare adverse reactions
than randomized controlled trials which would need to be very large.
Geoff Venning
References
1. Venning, G R. Validity of anecdotal reports of suspected adverse
drug reactions: the problem of false alarms. BMJ 1982; 284; 249-52
2. Vandenbroucke, J P. In defence of case reports and case series.
Ann Intern Med 2001; 134; 330-4
3. Rawlins, M D and Thompson, J W in Text book of Adverse Drug
Reactions. Ed D M Davies; 1977; Oxford University Press
4. Venning, G R. Identification of adverse reactions to new drugs, IV
Verification of suspected adverse reactions. BMJ; 1983; 286; 544-7
Competing interests:
None declared
Competing interests: No competing interests
Loke and colleagues underestimate the importance place by drug
regulators on case reports of suspected adverse drug reactions. The
pharmaceutical industry in Europe is mandated by current legislation to
collect information about safety of products (including older active
ingredients) from all sources. Once collected, data are evaluated and
incorporated into periodic safety update reports which are submitted
regularly to competent authorities. These reports document the ongoing
risk/benefit profiles. It is only by collecting data from all sources that
action can be taken to minimize risk.
Incorporation into commonly used drug sources is becoming less of an
issue as better and joined-up information systems mean that a wider
audience can access more comprehensive data, but the interpretation may
have to reside with the reader. Ultimately the quality of the information
depends on the objective reporting of suspected adverse events by
prescribers, and the recognition that risk minimization is the ultimate
goal.
Competing interests:
The author is the qualifed person for pharmacovigilance for Forest Laboratories UK Ltd
Competing interests: No competing interests
Dear Editor
I would disagree with Professor Bell that case reports are "effective
at early identification of ADRs" [1]
The paper by Loke et el was accepted by the BMJ on 4th October 2005,
published online on 18th January 2006 and didn't make it into the print
edition until 11th February 2006.[2] Allowing for the peer review and
corrections it might have been 4 months or more since it was first
submitted. The BMJ prides itself on rapid publication!
The most effective way of identifying ADRs early is by filling in a
Yellow card (in the UK) - regulators could then be working on signal
detection within a week of a drug being licensed, or a problem arising.
1. http://bmj.bmjjournals.com/cgi/eletters/332/7537/335#128092.
Accessed 13/2/06
2. http://bmj.bmjjournals.com/cgi/reprint/332/7537/335. Accessed
13/2/06
Competing interests:
I work at a MHRA regional monitoring centre and hence collect Yellow Cards
Competing interests: No competing interests
The conclusions of Loke and colleagues (1) are based on the erroneous
assumption that the only way to determine causality of a case report of an
adverse drug reaction (ADR) is by performing a controlled follow-up study.
They ignore pharmaceutical and regulatory authority databases and decision
-making which play a significant role in further evaluation.
Case reports act to direct attention to a possible ‘new’ ADR (or a
known ADR in a specific risk group). This serves to stimulate
practitioners to be alert for this in their patients and to report
suspected cases to regulatory authorities and pharmaceutical companies.
The information from these reports, together with possible reanalysis of
clinical trial data held by the companies and the regulators, will often
form the basis of the causality assessment, and potential change of
product labelling with or without new risk-benefit analysis and possible
withdrawal of the drug from the market. Such a system is very effective at
early identification of ADRs.(2,3)
It is therefore both expected and comforting that Loke et al. found
more changes to product labelling than there were validation studies for
case reports. Investigation of the basis for safety-related product
labelling changes by regulatory authorities would probably arrive at
sharply different conclusions about the value of case reports in
pharmacovigilance than the present study.
References
1. Yoon Kong Loke, Deirdre Price, Sheena Derry, and Jeffrey K
Aronson. Case reports of suspected adverse drug reactions—systematic
literature survey of follow-up. BMJ 2006; 332: 335-339
2. Stricker BH, Psaty BM. Detection, verification, and quantification of
adverse drug reactions. BMJ 2004;329: 44-7.
3. Vandenbroucke JP. In defense of case reports and case series. Ann
Intern Med 2001;134: 330-4.
Competing interests:
None declared
Competing interests: No competing interests
We endorse the views by Loke et al [1] suggesting that the warning
signals from published case reports are not systematically incorporated
into commonly used drug information sources. For example, a ‘Lesson of the
week’ BMJ publication had reported on a series of cases where adverse
reactions were encountered by using higher doses of colchicine in acute
gout [2], as per the recommendations of the British National Formulary
(BNF) [3]. The advice on dosing of colchicine by the BNF in 2002 was to
give - ‘1mg initially followed by 500µcg every 2-3 hours until relief of
pain is obtained or vomiting or diarrhoea occurs or until a total dose of
6 mg has been reached; the course should not be repeated within three
days’ [3]. Morris et al suggested an alternative low dose regime which
was effective yet less toxic [2], and lesser doses have also been
previously advocated in the ABC of Rheumatology more than a decade ago
[4].
Three years later (2005), the BNF has only slightly altered the
language regarding the treatment of gout and the total recommended dose
remains the same - ‘initially 1 mg, then 500 micrograms no more frequently
than every 4 hours until pain relieved or vomiting or diarrhoea occur,
max. 6 mg per course; course not to be repeated within 3 days’[5]. As a
result, the non-specialist will continue to prescribe the higher dosage
(which we have encountered repeatedly more recently in clinical practice),
as per the current BNF recommendations.
References:
1. Loke YK, Price D, Derry S, Aronson JK. Case reports of suspected
adverse drug reactions--systematic literature survey of follow-up. BMJ
2006; 332:335-339. (11 February)
2. Morris I, Varughese G, Mattingly P. Colchicine in acute gout. BMJ
2003; 327(7426):1275-6.
3. British Medical Association, Royal Pharmaceutical Society of Great
Britain. British National Formulary. London: BMA, RPS, 2002: 500. (No. 44)
4. Snaith ML. ABC of rheumatology. Gout, hyperuricaemia, and crystal
arthritis. BMJ 1995; 310(6978):521-4.
5. British Medical Association, Royal Pharmaceutical Society of Great
Britain. British National Formulary. London: BMA, RPS, 2005: 519. (No. 50)
Competing interests:
GIV was an author on the article cited in Reference 2, and both GIV & AAT have more recently been involved with patients prescribed higher doses of colchicine during acute non-selective general medical takes.
Competing interests: No competing interests
Loke et al. judge the value of case reports of suspected adverse drug
reactions (ADR) by the outcome measures of further formal validation and
the
incorporation into the product information and conclude that such reports
are of limited value.
The principle sources of information or tools that are available for
the
detection and evaluation of ADR are spontaneous reports, formal
epidemiological studies and controlled clinical trials, plus meta-
analyses.
Misunderstandings about the value of drug safety activities regularly
occur
when inappropriate tasks or unrealistic expectations are assigned to these
tools: attempts are made to derive causal “proof” or quantitative risk
estimates from spontaneous reports, spontaneous reporting systems are
blamed because they don’t detect associations when adverse events occur
with a high background incidence, epidemiological drug safety studies are
criticized for being retrospective and non-randomized, the number of
subjects in clinical trials is too low to detect rare ADR, and meta-
analyses are
subject to publication bias. However, used and interpreted appropriately,
all
of these methods have made major contributions to drug safety (1-4).
According to Loke’s report 83% of reports had not been subjected to
further
evaluation. However, the first question is whether the majority of
clinically
relevant ADR are initially communicated by case reports, and whether this
process is efficient. In other words, sensitivity combined with low cost
is the
primary concern with spontaneous reports.
In contrast to many reports of limited quality and relevance that end
up
being buried in spontaneous reports databases, published case reports of
ADR undergo peer-review for quality and relevance and are accessible to
anyone caring about drug safety. They increase specific as well as general
awareness of adverse drug effects and therefore their diagnosis in
clinical
practice, they contribute to mechanistic hypotheses, and they provide
important information for the design and conduct of formal safety studies;
and they miraculously do all this at no cost and are therefore highly
efficient.
Yes, in rare instances case reports do more harm than good, but
further
evaluations are able to identify such false signals or otherwise confirm
and
quantify safety problems (5). In their discussion, Loke et al. therefore
make
the important point that efforts should be intensified to further follow
safety
signals by formal safety studies, and that much of the considerable
resources
spent on spontaneous reporting systems may be better shifted towards such
activities. However, published case reports remain a valid and efficient
source
for signal generation and are of great value for drug safety. From that
point
of view it would be unfortunate if Loke's report
discouraged the publication of ADR.
1. Arnaiz JA, Carne X, Riba N, Codina C, Ribas J, Trilla A. The use
of evidence
in pharmacovigilance. Case reports as the reference source for drug
withdrawals. Eur J Clin Pharmacol 2001;57(1):89-91.
2. Bresalier RS, Sandler RS, Quan H, Bolognese JA, Oxenius B, Horgan
K, et al.
Cardiovascular events associated with rofecoxib in a colorectal adenoma
chemoprevention trial. N Engl J Med 2005;352(11):1092-102.
3. Jick H, Kaye JA, Jick SS. Antidepressants and the risk of suicidal
behaviors.
JAMA 2004;292(3):338-43.
4. Juni P, Nartey L, Reichenbach S, Sterchi R, Dieppe PA, Egger M.
Risk of
cardiovascular events and rofecoxib: cumulative meta-analysis. Lancet
2004;364(9450):2021-9.
5. Jick H, Kaye JA. Epidemiology and possible causes of autism.
Pharmacotherapy 2003;23(12):1524-30.
Competing interests:
None declared
Competing interests: No competing interests
The main difference between Loke et al.’s replication of Venning’s
1980s studies and the original is the evidence that is accepted as further
follow-up. Loke et al. mention that 56 of 63 case reports on adverse
reactions were cited at least once, but accept only 9 instances as valid
follow-up: attempts at proof of mechanisms or controlled studies. However,
unexpected adverse effects are by definition due to an unknown mechanism
and many established adverse effects still escape elucidation after years.
Moreover, further accumulation of anecdotal reports can be sufficient
evidence. A 10-year tally of drugs that were withdrawn from the market
showed that of 22 withdrawals, 18 were prompted by case reports and only 4
by a controlled study; for at least 13 the case reports proved sufficient
reason for withdrawal in themselves (1).
When a highly unusual happening is repeatedly reported in a drug that
is recently marketed, in the absence of publicity, this points to an
exceedingly high relative risk. Exceedingly high relative risks are good
arguments for cause and effect (2). Moreover, they are the one’s that will
be picked up spontaneously (3). By way of example: a first report of
sudden taste loss due to terbinafine was followed by a report of three
similar cases reported to the yellow card system (4), and the adverse
effect became accepted.
Apart from the 9 instances of follow-up that were accepted by Loke et
al., there were 15 datasheets amended and 7 BNP drug monographs changed.
So, even with some overlap, close to half of the case reports were somehow
sufficiently noteworthy to have had consequences. Given the small sample
size, this is not far from Venning's original. It is a pity that Loke et
al. do not make all follow-up more explicit, for all 56 instances that
followed the 63 reports.
Their overall conclusion that lack of follow-up means that case
reports are of limited value is a non-sequitur. It shows that a more
consistent scheme of hypothesis-testing studies by industry, government,
or third parties is needed instead of the current haphazard approach to
drug safety. And even if there is little follow-up, these publications may
alert and help clinicians in recognising potential adverse events in their
patients. Often there is no other way of discovering something unusual.
The record of systematic, i.e. controlled, postmarketing surveillance in
the detection of unexpected adverse effects remains surprisingly poor.
1. Arnaiz JA, Carne X, Riba N, Codina C, Ribas J, Trilla A. The use
of evidence in pharmacovigilance. Case reports as the reference source for
drug withdrawals. Eur J Clin Pharmacol 2001;57:89-91.
2. Cornfield J, Haenszel W, Hammond EC, Lilienfeld AM, Shimkin MB,
Wynder EL. Smoking and lung cancer: recent evidence and a discussion of
some questions.
J Natl Cancer Inst 1959;22:173-203.
3. Vandenbroucke JP. In defense of case reports and case series. Ann
Intern Med 2001;134:330-4.
4. Ottervanger JP, Stricker BH. Loss of taste and terbinafine. Lancet
1992;340:728.
Competing interests:
None declared
Competing interests: No competing interests
Lack of follow-up negates value of case reports
We are grateful to all those who have responded to our original
paper. We want to stress that anecdotal reports of suspected adverse drug
reactions (ADRs) can, in the right circumstances and with proper
interpretation, be very useful, and we certainly do not want to discourage
anecdotal reporting, either in journal articles or to large databases.
Indeed, our view is that comprehensiveness in anecdotal reporting is
important [1,2], particularly since about 30% of all published information
on ADRs is anecdotal [3]. However, if an ADR that is suggested by a single
anecdote in a peer-reviewed journal is not somehow confirmed, the use of
that anecdote is limited, as previously detailed [1]. What we have shown
is that only 11% of anecdotes were properly confirmed by modern evidential
standards. That does not mean that the rest were not true associations,
merely that they were not confirmed as such. However, citation does not
imply confirmation or follow-up, as Vandenbroucke seems to suggest.
Similarly, Venning's criteria for deciding that an anecdote had been
confirmed were not strict enough by today's standards. This lack of follow
-up devalues the efforts of reporters who may have gone to great lengths
to collect data and write up their suspicions, only to find that their
alert remains uninvestigated.
Vandenbroucke and Ball try to draw some comfort from the instances in
which the product information sheet was amended after the case report had
been published. But we found that 33 Summaries of Product Characteristics
(SPCs) in the Medicines Compendium and 41 monographs in the British
National Formulary (BNF) were unchanged. We do not know why the product
information was amended in some cases, while in others it was not. There
are currently no clear criteria for including ADRs in secondary sources of
information, such as SPCs and the BNF, or for excluding them. At the
moment, anecdotal ADRs may or may not be mentioned in such sources, and
the quality of the evidence on which a reported association is based is
not made clear. This leaves patients and prescribers less well informed
than they should be.
Moreover, the respondents appear to believe that action by regulatory
authorities is sufficient to confer scientific validity on suspected
adverse effects. However, we believe that regulatory action should be
underpinned by obtaining good evidence, rather than science being guided
by regulation. While we have no doubt that regulatory authorities are
continuously vigilant in risk minimization, we are concerned that the data
on which they base their decisions may be of poor quality, through lack of
confirmation. We are not unaware of the several techniques that are
currently available for analysing the large numbers of anecdotal reports
that are submitted to regulatory authorities, such as proportional
reporting rates (PRRs) and relative odds ratios (RORs) [4]) and Bayesian
techniques (the Bayesian confidence propagation neural network, BCPNN, and
the Multi-item Gamma Poisson Shrinker, MGPS) [5,6]. However, our work
concerned the individual anecdotes that appear in peer-reviewed journals,
not those large databases.
In contrast to Vandenbroucke’s positive reaction, we were
disappointed with the finding that 18 of 22 drug withdrawals appeared to
be based purely on case reports. We would have no qualms about this
process, were it not for the fact that patients may have been deprived of
potentially beneficial treatments for their illnesses. The withdrawal of
alosetron following case reports of ischaemic colitis is an important
example: patients with irritable bowel syndrome who had benefited from
alosetron would have been deprived of it had they not protested against
the FDA’s decision to withdraw it and had it reinstated [7]. In contrast,
and perhaps reassuringly, MMR was not withdrawn on the basis of
Wakefield's anecdotal evidence in 12 patients—instead it was subjected to
proper scrutiny. Such problems in regulatory decision-making undoubtedly
stems in part from the obvious difficulties in accurately determining the
benefit:harm balance when efficacy has been established through high-
quality randomized trials, but the safety data come from unconfirmed
anecdotes.
Case reports of suspected adverse reactions do have their uses [1],
but until we have adequate follow-up and verification of isolated reports,
we cannot be certain whether they are genuine alerts that should be taken
seriously, or false alarms that lead us to make wrong treatment or
regulatory decisions.
References
1. Aronson JK. Anecdotes as evidence. BMJ 2003; 326: 1346.
2. Aronson JK. Unity from diversity: the evidential use of anecdotal
reports of adverse drug reactions and interactions. J Eval Clin Pract
2005; 11: 195-208.
3. Aronson JK, Loke Y, Derry S. Adverse drug reactions: keeping up to
date. Fundam Clin Pharmacol 2002; 16: 49-56.
4. Waller P, van Puijenbroek E, Egberts A, Evans S. The reporting odds
ratio versus the proportional reporting ratio: 'deuce'. Pharmacoepidemiol
Drug Saf 2004; 13: 525-6.
5. Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, De
Freitas RM. A Bayesian neural network method for adverse drug reaction
signal generation. Eur J Clin Pharmacol 1998; 54: 315-21.
6. Szarfman A, Tonning JM, Doraiswamy PM. Pharmacovigilance in the 21st
century: new systematic tools for an old problem. Pharmacotherapy 2004;
24: 1099-104.
7. Traynor K. Alosetron use drops dramatically with risk management. Am J
Health Syst Pharm 2004; 61: 1210-12.
Competing interests:
None declared
Competing interests: No competing interests