Risks of harms using antifibrinolytics in cardiac surgery: systematic review and network meta-analysis of randomised and observational studies
Cite this as: BMJ 2012;345:e5798
Rapid responses are electronic letters to the editor. They enable our users to debate issues raised in articles published on bmj.com. Although a selection of rapid responses will be included as edited readers' letters in the weekly print issue of the BMJ, their first appearance online means that they are published articles. If you need the url (web address) of an individual response, perhaps for citation purposes, simply click on the response headline and copy the url from the browser window.
Displaying 1-3 out of 3 published
We appreciate the chance to respond to recent letters regarding our review[1,2]. Dr. Freemantle and colleagues raise two issues: qualitative differences between our review and theirs, and concerns with our inclusion of observational studies using propensity scoring. We have difficulties with their commentary. As for qualitative differences, careful inspection shows the authors appear to have inappropriately compared our mortality estimate for aprotinin versus tranexamic acid with their estimate for re-operation, an outcome we did not review. This is uninformative and inappropriate. The authors also question the narrower credible interval of our mortality estimate comparing aprotinin with tranexamic acid and suggest qualitative differences in findings.
We disagree there is a qualitative difference between estimates; the direction of increased mortality risk with aprotinin compared to tranexamic acid is consistent, while our point estimate has narrower credible intervals. More importantly, we explored reasons for the difference in this estimate and were disappointed to find that, despite explicitly stating that their review was for cardiac surgery patients, one can see from their Table 1 that Freemantle et al appear to have inappropriately included trials in liver transplantation[5-7], hip replacement, cancer, and infrarenal abdominal aneurysm in their analysis. They also included studies of off-pump cardiac procedures, which we excluded a priori[11-14]. As an exercise, we repeated our mortality analysis after incorporating these unrelated RCTs and achieved estimates similar to theirs, however these studies do not belong in a meta-analysis of cardiac surgery. We agree propensity score matching techniques cannot guarantee complete removal of selection bias from observational studies. We noted this previously, and it was the rationale for sequentially pooling our data. It has long been suggested to consider observational data for analyses of harms due to limitations encountered by studying RCTs only[15-22], and our paper used one possible approach for doing so. We stand by findings from our work. Regarding the BART study, the erroneous and misleading criticisms made by the European Medicines Agency and the Expert Advisory Panel on Trasylol are being addressed in a manuscript currently in peer review. Finally, it is disappointing to see researchers in a scientific discussion resort to ad hominems such as ‘scaremongering’ to dismiss conflicting evidence.
The comments from Dr. Weng relate to diversity of patients and treatments and are relevant for meta-analysis. Regarding the testing of homogeneity/similarity, we believe these assumptions are best considered by careful review of study characteristics by methodologists and clinical experts. Our team judged the studies appropriate for meta-analysis based on their clinical and methodologic features, and this is most important as statistical heterogeneity may be present even in the presence of homogeneous studies. Regarding our inclusion criteria, it is fair to describe the patient populations as broad. Our approach is however in line with past reviews of antifibrinolytic agents, and the debate of ‘lumping’ versus ‘splitting’ populations may be argued reasonably both ways. We believe these patients to be appropriate for meta-analyses. Although the desire to look separately at low risk and high risk groups is understandable, doing so in this field is difficult as there are multiple factors to determine risk (surgical history, procedure type, procedure urgency), and many studies include multiple types. The observational studies in this area include many types of patients, making their inclusion in pure subgroup analyses infeasible, and one is then left with small trials for such explorations.
The author’s final suggestion related to splitting nodes in our treatment network to reflect dose. We agree this may be interesting, however we did not do this given our interest in studying associations with any exposure, given the multiple and disparate dosing regimens for these agents, and also given the rarity of the outcomes. The notion is further complicated by the limitation that adequate dosing information is not available for some observational research. Despite this, given the limitations of safety meta-analyses which are restricted to RCTs, we believe efforts to include observational data remains important.
In conclusion, we are not aiming to raise misplaced fears about aprotinin, but rather we hope to highlight evidence clinicians and patients must consider regarding its potential risks.
(1) Freemantle N, Howell N, Pagano D, Senanayake E. Re: Risks of harms using antifibrinolytics in cardiac surgery: systematic review and network meta-analysis of randomised and observational studies. BMJ Rapid Response, October 25 2012.
(2) Meng W. The methodological assumptions for network meta-analysis. BMJ Rapid Response Sept 19, 2012.
(3) Hutton B, Joseph L, Fergusson D, Mazer CD, Shapiro S, TInmouth A. Risks of harms using antifibrinolytics in cardiac surgery: systematic review and network meta-analysis of randomised and observational studies. Sept 2012.Published online first. 2012.
(4) Howell N, Senanayake E, Freemantle N, Pagano D. Putting the record straight on aprotinin as safe and effective: Results from a mixed treatment meta-analysis of trials of aprotinin. J Thorac Cardiovasc Surg Aug 10. Published online first. 2012.
(5) Dalmau A, Sabate A, Acosta F, et al. Tranexamic acid reduces red cell transfusion better than epsilon-aminocaproic acid or placebo in liver transplantation. Anesth Analg 91, 29-34. 2000.
(6) Ickx B, Van Der Linden P, Melot C et al. Comparison of the effects of aprotinin and tranexamic acid on blood loss and red blood cell transfusion requirements during the late stages of liver transplantation. Transfusion 46, 595-605. 2006.
(7) Dalmau A, Sabate A, Koo M, Bartolome C, et al. The prophylactic use of tranexamic acid and aprotinin in orthotopic liver transplantation: a comparative study. Liver Transplantation 10, 279-284. 2004.
(8) Colwell C, Chelly J, Murkin J et al. Randomized study of Aprotinin effect in transfusions and blood loss in primary THA. Clinical Orthopaedics and Relation Research 465, 189-195. 2007.
(9) Norman P, Thall P, Puruggannan R, et al. A possible association between aprotinin and improved survival after radical surgery for mesothelioma. Cancer 115, 833-841. 2009.
(10) Leijdekkers V, Vahl A, Mackaay A, Huijgens P, Rauwerda J. Aprotinin does not diminish blood loss in elective operations for infrarenal abdominal aneurysms: a randomized double-blind controlled trial. Ann Vasc Surg 20, 322-329. 2006.
(11) Mehr Aein A, Sadeghi M, Madani-civi M. Does tranexamic acid reduce blood loss in off-pump coronary artery bypass? Asian Cardiovascular and Thoracic Annals 15, 285-289. 2007.
(12) Murphy G, Mango E, Lucchetti V, et al. A randomized trial of tranexamic acid in combination with cell salvage plus a meta-analysis of randomized trials evaluating tranexamic acid in off-pump coronary artery bypass grafting. J Thorac Cardiovasc Surg 132, 475-480. 2006.
(13) Desai P, Kurian D, Thirumavalavan N et al. A randomized clinical trial investigating the relationship between aprotinin and hypercoagulability in off-pump coronary surgery. Anesth Analg 109, 1387-1394. 2009.
(14) Jares M, Vanek T, Straka Z, Brucek P. Tranexamic acid reduces bleeding after off-pump coronary artery bypass grafting. Journal of Cardiovascular Surgery 44, 205-208. 2003.
(15) Bradburn M, Deeks J, Berlin J, Russell Localio A. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Statistics in Medicine 26, 53-77. 2007.
(16) Chou R, Helfand M. Challenges in reviews that assess treatment harms. Annals of Internal Medicine 142, 1090-1099. 2005.
(17) Chou R, Aronson N, Atkins D, et al. Assessing harms when comparing medical interventions. Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet].Rockville (MD): Agency for Healthcare Research and Quality . 2008.
(18) Ho P, Peterson P, Masoudi F. Evaluating the evidence: is there a rigid hierarchy? Circulation 118, 1675-1684. 2008.
(19) Ioannidis J, Lau J. Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA 285, 437-443. 2001.
(20) Ioannidis J, Lau J. Improving safety reporting from randomised trials. Drug Safety 25, 77-84. 2002.
(21) Sutton A, Cooper N, Lambert P, Jones D, Abrams K, Sweeting M. Meta-analysis of rare and adverse event data. Expert Rev Pharmacoecon Outcomes Res 2, 367-379. 2002.
(22) Sweeting M, Sutton A, Lambert P. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Statistics in Medicine 23, 1351-1375. 2004.
Competing interests: None declared
Ottawa Hospital Research Institute, The Ottawa Hospital, 501 Smyth Road, Ottawa, Canada, K1H 8L6
Click to like:
The continued controversy concerning aprotinin is surprising given the quite modest potential use of this agent, a protective agent in cardiac surgery, and yet it serves to highlight the limitations of the methods that we use. Hutton and colleagues  have been close to this debate (Dr Fergusson was for example the co-Chair of the BART trial steering committee ). The JTCVS recently published a useful summary  which for example highlighted the Health Canada strong criticisms of discrepancies in the BART trial, criticisms which it appears have not yet been answered in the public domain.
A couple of points are helpful to note with regards the current review. First, the faith of the authors in the ability of the propensity score approach to deal with confounding is touching although misplaced. Where treatments are given according to the judgement of a clinician, in the knowledge of risk information, any additional value to that clinical judgement above the crude conditioning available from observed risk factors is a latency which cannot be estimated or accounted for (being completely confounded with treatment). This latency, commonly called ‘confounding by severity’ or ‘confounding by indication’ is a most troublesome bias for this design , and can in our experience lead to very large biases in studies which are apparently well conducted.
Second, the results of the authors’ meta analyses of trials  are slightly surprising, given that the results are qualitatively different from a meta analysis that we conducted using the same randomised trials , and to (for the external observer) apparently equally high standards! In our analysis  the odds ratio comparing aprotinin and tranexamic acid for the trials alone was 0.68 (95%CI 0.48 to 1.06) (reported in the other direction and thus as the reciprocal in our paper). While in the analysis by Hutton et al it is 0.64 (95% CI 0.41 to 0.99). Of note are the relatively narrow confidence intervals from the Hutton analyses  which must be due to the specification of the model in particular the specification of random effects and the priors applied. Some may find it disturbing that the two analyses can give such different results, although it highlights the extent to which these may be influenced by the aspects under the control of the analyst. It is disappointing that a drug that systematically reduces the incidence of serious complications following cardiac surgery, such as the need for a reoperation for bleeding and transfusion, has become the subject of statistical and methodological scaremongering without due consideration for the patients.
1. Hutton B, Joseph L, Fergusson D, Mazer CD, Shapiro S, Tinmouth A. BMJ 2012;345:e5798 doi: 10.1136/bmj.e5798
2. Fergusson DA, Hebert PC, Mazer CD, Fremes S, MacAdams C, Murkin JM, et al. A comparison of aprotinin and lysine analogues in high-risk cardiac surgery. N Engl J Med 2008;358:2319-31.
3. DeAnda A, Spiess BD. Aprotinin revisited. J Thorac Cardiovasc Surg 2012;144:998-1002
4. Bosco JLF, Silliman RA, Thwin SS, Geiger AM, Buist DSM, Prout MN, et al. A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies. J Clin Epidemiol 2009;doi:10.1016/j.jclinepi.2009.03.001
5. Howell NJ, Senanayake EL, Freemantle N, Pagano D. Putting the record straight: Aprotinin is safe and effective. Results from a mixed treatment meta-analysis of trials of Aprotinin including the BART study. Journal of Thoracic and Cardiovascular Surgery, published ahead of print.
Competing interests: None declared
University College London, Rowland Hill Street, London NW3 2PF
Click to like:
We read with interest the systematic review by Brian Hutton et al , but there are several statistical issues on network meta-analysis we are concerned about.
Basically, network meta-analysis holds three methodological assumptions : 1) like standard meta-analysis, it is assumed that different trials in every pair-wise comparison in the network are sufficiently homogeneous; 2) the similarity of the relative effects of treatment should be held across the entire set of trials; and 3) the relative effects of direct and adjusted indirect comparisons should be consistent when direct and indirect data are combined together. These three issues of comparability concern different levels of decisions for a research synthesis of clinical trials. The trial similarity assumption for adjusted indirect comparison requires the prerequisite of homogeneity assumption; and the consistency assumption is relevant only if both the homogeneity assumption and the similarity assumption are valid . In this systematic review, the consistency was tested by splitting the nodes but the homogeneity and the similarity were neglected. It is unclear whether these assumptions are fulfilled or not in this study.
This systematic review set a very broad inclusion criteria. No restrictions were set for many essential factors like surgical history, surgical urgency, surgical type, and drug dose. By checking the included studies, we find that the patient baseline characteristics vary greatly among different studies, especially for drug dose. This makes it highly possible that the data analysis results would change when stratify the data by the above factors. We kindly suggest the authors 1) testing the homogeneity by traditional chi-square test or I-square test ; 2) assessing the similarity using subgroup analysis or meta-regression; and 3) splitting the nodes in the network model according to dosage to provide a more detailed safety profile of the treatments compared.
1. Brian Hutton, Lawrence Joseph, Dean Fergusson, C David Mazer, Stan Shapiro, Alan Tinmouth. Risks of harms using antifibrinolytics in cardiac surgery: systematic review and network meta-analysis of randomised and observational studies. BMJ 2012;345:e5798
2. Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 2009;338:b1147.
3. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011): The Cochrane Collaboration; 2011: www.cochrane-handbook.org
Competing interests: None declared
The Second Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou, Gansu Province, China, NO.1 Dongguan West Road, Chenguan District, Lanzhou, Gansu Province, China
Click to like: