Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Randomised controlled trials are complementary to observational studies for the postmarketing assessment of drug safety
- Sérgio de A. Nishioka (25 March 2008)
-
Too high a pedestal
- Jeffrey K Aronson (26 March 2008)
-
The argument is just plain wrong
- Simon Hatcher (8 April 2008)
-
Interpreting observational study data; a different view?
- David Royston (9 April 2008)
|
|
|||
|
Sérgio de A. Nishioka, Assistant professor Faculdade de Medicina, Universidade Federal de Uberlândia, Uberlândia, MG, Brazil
Send response to journal:
|
Prof Freemantle and Dr Irs [1] criticize recent American and European initiatives to strengthen pharmacovigilance that promote the conduct of observational studies, arguing that these studies provide biased evidence of drug safety and therefore are no substitute for randomised controlled trials (RCT). Although confounding by indication can be in fact an important validity issue in observational studies in pharmacoepidemiology, and statistical techniques have limited value to correct this type of bias [2], postmarketing RCT are not always the only or the more efficient solution to obtain valid estimates of drug safety. Drug regulatory agencies can ask for additional trials to address safety issues as a postmarketing commitment for new drugs from their manufacturers. This is not always done, in part because there might be no perception for their need. There should exist a good reason for a DRA to ask for postmarketing safety studies, as it can be argued that enough evidence for safety should be a prerequisite for the marketing authorisation. In a case when a clinical trial is a regulatory postmarketing requirement for a certain purpose, this trial will not necessarily be powered to rare adverse drug reactions (ADR) that may have been undetected during the clinical investigation of a new drug, and that are therefore unknown at the time of the (postmarketing) study design. Depending on the type of ADR, and how it is detected or measured, only after its recognition by spontaneous reports from smart clinicians it could be properly determined as an outcome in a clinical trial. If a RCT is then designed, it will take time to give answers. These answers will not necessarily be applicable to different age groups, co-morbidity profiles, and for individuals from different races or geographical regions. Observational studies can assess relatively rapidly an association between drug exposure (sometimes different doses and durations) and health outcomes, and the measure of association will not always be biased. They may provide information to support regulatory measures before results from clinical trials, if ever conducted, become available. When the findings are controversial, and bias is likely to be an issue, then RCT may become a must, but this is not always the case. From a regulatory standpoint, RCT and observational studies are not mutually exclusive alternatives, they are complementary. References 1. Freemantlr N, Irs A. Observational evidence for determining drug safety: is no substitute for evidence from randomised controlled trials. BMJ 2008:336:627-8. 2. McMahon AD. Approaches to combat with confounding by indication in observational studies of intended drug effects. Pharmacoepidemiol Drug Saf. 2003;12:551-8. Competing interests: The author worked formerly for the Brazilian drug regulatory agency, ANVISA |
|||
|
|
|||
|
Jeffrey K Aronson, Reader in Clinical Pharmacology Dept of Primary Health Care, Rosemary Rue Building, Old Road Campus, Oxford OX3 7LF
Send response to journal:
|
Freemantle and Irs are right to call for more randomized controlled trials to detect adverse drug effects. They could have added a call for more systematic reviews, of which there are too few.1,2 However, they are wrong to suggest that “only properly randomised trials can provide truly reliable evidence on adverse events, just as these are the only source of convincing data on drug efficacy.” Harms due to drugs differ from benefits in several ways. Harms are multifarious, benefits usually single. Harms affect fewer individuals, some of whom may have particular susceptibilities. And harms often cannot be identified in advance. In some cases these features militate against the practicable use of randomized trials. If observational studies show no evidence of harms, randomized trials are certainly necessary. They are always desirable, and some adverse effects can only be elicited reliably in this way. The increased risk of cardiovascular disease with rofecoxib is an example. However, definitive evidence of some types of adverse effects can emerge from anecdotes or formal observational studies, notwithstanding the problems that Freemantle and Irs highlight, in which case a randomized trial would not be necessary. Bayesian data mining techniques can provide powerful evidence of associations that would not have been detected otherwise.3 In some cases a randomized trial may even be less powerful than other types of study. There are examples of anecdotal reports that provide definitive evidence of both harms and benefits.4,5 Randomized trials are unnecessary in such cases. There are also examples of adverse effects that have only emerged from observational studies, having failed to be elicited by randomized studies. The adverse interaction of chronic paracetamol administration and warfarin emerged from a case-control study.6 Despite various preceding reports, mostly anecdotal, that hinted at such an association, the size of this effect was unexpected, with little apparent biological plausibility, as is not infrequently the case with adverse effects. The study also showed that the effect was related to dose, an aspect of drug interventions that is often not studied in randomized trials, and to duration of therapy. It has since been confirmed,7 and the design of confirmatory studies has been informed by the findings of the case-control study. If an observational study suggests a serious adverse effect, it would be hard to justify studying it in a randomized trial. In the case quoted by Freemantle and Irs, after aprotinin had been associated with a large increase in mortality in an observational study, a randomized trial was not justifiable. Fully informed patients would reasonably refuse to take part in such a trial, even though thereby denying themselves possible benefit from the intervention. If the benefit to harm balance is unknown, and may be unfavourable, erring on the side of caution is justified. A Cochrane review showed no increased risk of lactic acidosis from 48,000 patient-years of metformin therapy.8 This is not surprising, since the incidence is estimated at about 3 per 100,000 patient-years. There have been numerous observations of this association, and other biguanides have been similarly implicated. Should we ignore this evidence of an adverse effect, rare but with a high mortality, in preference to the evidence from trials, which even in combination were not powerful enough to detect it? This is highly controversial, but some consider it wise to use the drug carefully in those who are thought to be most at risk.9 This may be crying wolf, but sometimes the wolf does come, and we may not want to ignore the call and risk the consequences. We should take note of what two eloquent proponents of randomized controlled trials have written10: “Our main wish, from which all others stem, is that RCTs be taken off their pedestal, their exalted position at the top of an artificial evidence hierarchy; that all forms of evidence be appreciated for what they can offer.” References 1. Ernst E, Pittler MH. Assessment of therapeutic safety in systematic reviews: literature review. BMJ 2001; 323(7312): 546. 2. Aronson JK, Derry S, Loke YK. Adverse drug reactions: keeping up to date. Fundam Clin Pharmacol 2002; 16(1): 49-56. 3. Hauben M, Horn S, Reich L. Potential use of data-mining algorithms for the detection of 'surprise' adverse drug reactions. Drug Saf 2007; 30(2): 143-55. 4. Aronson JK, Hauben M. Anecdotes that provide definitive evidence. BMJ 2006; 333(7581): 1267-9. 5. Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ 2007; 334(7589): 349-51. 6. Hylek EM, Heiman H, Skates SJ, Sheehan MA, Singer DE. Acetaminophen and other risk factors for excessive warfarin anticoagulation. JAMA 1998; 279(9): 657-62. 7. Mahé I, Bertrand N, Drouet L, Bal Dit Sollier C, Simoneau G, Mazoyer E, Caulin C, Bergmann JF. Interaction between paracetamol and warfarin in patients: a double-blind, placebo-controlled, randomized study. Haematologica 2006; 91(12): 1621-7. 8. Salpeter S, Greyber E, Pasternak G, Salpeter E. Risk of fatal and nonfatal lactic acidosis with metformin use in type 2 diabetes mellitus. Cochrane Database Syst Rev 2006; (1): CD002967. 9. Nisbet JC, Sturtevant JM, Prins JB. Metformin and serious adverse effects. Med J Aust 2004; 180(2): 53-4. 10. Jadad AR, Enkin MW. Randomized controlled trials: questions, answers, and musings. 2nd edition. Oxford: Blackwell Publishing/BMJ Books, 2007: 128. Competing interests: JKA is Editor of Meyler's Side Effects of Drugs: The International Encyclopedia of Adverse Drug Reactions and Interactions, and of its annual updates, the Side Effects of Drugs Annuals. |
|||
|
|
|||
|
Simon Hatcher, Senior Lecturer in Psychiatry University of Auckland
Send response to journal:
|
The argument that randomised controlled trials are the only way to determine drug safety rapidly runs into the buffers when rare but catastrophic outcomes are considered. Take for instance suicide. This occurs at a rate between 10-30/100,000 in most developed countries. To design a randomised controlled trial that would have adequate power to determine if a drug caused suicide would require tens of thousands of patients. This is impractical. In these circumstances we are left with considering observational studies and indeed the BMJ has published several when trying to answer the question do antidepressants cause suicide. The reason that antidepressant suicide is controversial is, in part, due to the fact that randomised controlled trials cannot provide a clear answer. To argue that randomised controlled trials are the only way to determine drug safety is wrong in practice and ignores the problem of rare but catastrophic events such as suicide. Competing interests: None declared |
|||
|
|
|||
|
David Royston, Consultant Anaesthetist Harefield Hospital, UB9 6JH
Send response to journal:
|
The authors of this editorial raise a number of points regarding analysis and interpretation of observational studies that are especially pertinent to the aprotinin story. The main problems in the interpretion of these studies are the inclusion criteria used for confounding variables and the mathematical modelling used for the analysis; in particular the vogue for the use of propensity scores. The propensity score used in the article of January 2006 in the New England Journal of Medicine (NEJM) (1) was based on 45 variables for post- operative bleeding and not the patient risk for adverse outcomes (2). At the advisory committee of the Food and Drug Administration (FDA) held on September 12th 2007. Mark Levenson, PhD, of the FDA made three key points concerning the analysis of the so-called McSPI Epi II database analysis(3): First, if the data are analyzed using the method employed for the NEJM article the same results are achieved. So far, so good. Second, the FDA analysis found that the propensity scores derived by linear regression and used in the NEJM article did not overlap. Not so good. I am informed that lack of overlap means that risk adjustment is unreliable—and perhaps totally inappropriate. This questions the dependability of any of the conclusions reached in the NEJM article. The FDA produced a risk adjustment based on stratification (specific factors included were not described but did not adjust for geographic differences in the dataset; the third point from the FDA analysis of the McSPI dataset). This FDA analysis of the data showed no increase in the relative risk (RR) for death (RR 0.91 (95% confidence interval [CI] 0.54-1.53), heart failure (RR 1.05 (95% CI 0.75-1.47), myocardial infarction (RR 1.10 (95% CI 0.88-1.39) or renal dysfunction (RR 1.26 (95% CI 0.76-2.11) when data from 1,222 aprotinin treated patients were compared to 1,307 patients who did not receive the drug(3). So is aprotinin the culprit risk factor or did practitioners give this drug to patients with more co-morbidites whose risks were not adequately adjusted for in the published analysis? This question is highlighted further when considering a more recent offering from the database of the Duke University Medical Center (4). A propensity score based on linear regression analysis was also used to allow for large differences in the baseline risk profiles of the patients. The aprotinin treated patients had an increased proportion of patients having other than isolated coronary artery bypass surgery (19.8 % v 4.0%), reoperation (41.5% v 15.9%), operated as non-elective cases (72.1% v 34.2%) and with heart failure (based on use of angiotensin converting enzyme inhibitors and diuretics) compared with those given EACA. Most clinicians will recognize that there must be a difference in outcome between a patient with isolated myocardial ischaemia having primary, elective revascularization ( who should not require any drug therapy to prevent transfusions) when compared with one having a non- elective, reoperation for heart failure associated with valve pathology ( who would certainly receive aprotinin in about 70-80% of cases worldwide). Despite this the authors concluded that aprotinin use was the factor associated with mortality when comparing data from 1343 aprotinin treated patients with 6776 given epsilon aminocaproic acid (EACA) and 2029 given neither therapy. Two aspects may lead the interested reader to question this conclusion. First, the propensity analysis did not include red cell transfusion numbers as a factor (transfusion was either yes or no). This is despite the authors quoting the North West consortium analysis showing that any aprotinin effect on adverse outcome is lost when red cell transfusion numbers are included as a variable(5). The original NEJM article (1) also treated transfusion as a yes/no variable. This is all the more interesting when considering that one of the multiple publications from the same dataset concluded that ‘at the same hemoglobin level, the risk of suffering a postoperative complication increased significantly with transfusion of RBCs’ and ‘there was a direct relationship between the number of units of RBCs transfused intraoperatively and the incidence of adverse outcomes’(6). Second and more worrying and perplexing is that the supplementary data provided by Shaw and colleagues (7) reported a matched pairs analysis including 1992 patients with comparable risks using the factors in their propensity score derivation data. This matched pair analysis showed aprotinin had no effect on 30 day (Relative Risk 1.25; 95% Confidence Interval 0.74-1.72) or 1 year mortality (RR1.27; 95% CI 1.1-1.46) compared to EACA. So if propensity scoring is achieved by linear regression and confounding variables known to be associated with adverse outcomes are excluded then observational studies show aprotinin is a dangerous drug. Aprotinin is not dangerous when the analysis is performed using matching or stratification of risk. Based on the confusions raised by the methodology of the analyses of these observational studies who would sit in judgment of aprotinin on the FDA or EMEA? 1. Mangano DT, Tudor IC, Dietzel C. The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006;354(4):353-65. 2. Mangano.ppt. www.fda.gov/ohrms/dockets/ac/cder06/.html#CardiovascularRenal. 3. Levenson.ppt. www.fda.gov/ohrms/dockets/ac/07/transcripts2007- 4316t1-index.pdf. 4. Shaw A, Stafford-Smith M, White W, Phillips-Bute B, Swaminathan M, Carmelo M, Welsby I, Aronson S, Mathew J, Peterson E, Newman M. The effect of Aprotinin on Outcome after Coronary-Artery Bypass Grafting. N Engl J Med 2008;358(8):784-93. 5. Furnary A, YX W, Hiratzka L, Grunkemeier G, Page U. Aprotinin does not increase the risk of renal failure in cardiac surgery patients. Circulation 2007;116((Suppl I)):I-127 I-133. 6. Kulier A, Levin J, Moser R, Rumpold-Seitlinger G, Tudor I, Snyder-Ramos S, Moehnle P, Mangano DT. Impact of Preoperative Anemia on Outcome in Patients Undergoing Coronary Artery Bypass Graft Surgery. Circulation 2007;116:471-479. 7. Supplement to: Shaw AD Stafford-Smith M, White WD, et al. The effect of aprotinin on outcome after coronary artery bypass grafting. N Engl J Med 2008;358:784-93. Dr David Royston FRCA
Competing interests: Conflict of Interest DR acted as consultant to Bayer Pharmaceuticals (now Bayer Shering Pharma) during the FDA advisory committee and received honoraria for that. DR is on the speaker panel of a number of pharmaceutical companies including Bayer Pharmaceuticals |
|||