Rapid Responses to:

PAPERS: Eric Lim, Ziad Ali, Ayyaz Ali, Tom Routledge, Lyn Edmonds, Douglas G Altman, and Stephen Large Indirect comparison meta-analysis of aspirin therapy after coronary surgery BMJ 2003; 327: 1309 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

BMJ save us

Simon Grant   (8 December 2003)

Hype over substance

Saul G Myerson   (9 December 2003)

The trials differ too much to make indirect comparisons

Christopher J Cates   (9 December 2003)

Authors' Reply

Eric Lim   (12 December 2003)

Independent Prospective Inclusion Criteria are vital for unbiased Meta-analysis

Joel Dunning, On Behalf of the EBM Cardiothoracic Journal Club at Wythenshawe Hospital   (22 December 2003)

Aspirin after coronary bypass surgery - medium versus low dose improves graft patency

Thomas Hohlfeld   (7 January 2004)

First principles or evidence based critique?

Eric Lim   (9 January 2004)

Can we replace uncertainty with certainty ?

Joel Dunning   (13 January 2004)

Reservations with Indirect Comparisons

Steven A. Julious   (1 September 2005)

BMJ save us 8 December 2003
Simon Grant, Consultant Cardiologist Calderdale Royal Hospital HX3 0PW Send response to journal: Re: BMJ save us	EDITOR-‘Medium dose treatment (with aspirin) may be better (at maintaining saphenous vein graft patency) than low doses’ (BMJ News and front cover Dec 2003). On the other hand it may not. Lim et al’s article is a good review of this potentially important question.1 However, the correct interpretation of their results is that the available literature does not tell us the answer. The confidence intervals of the trials they reviewed are are wide and overlap even when aggregated. The p values do not achieve the generally accepted levels of significance. The data (allowing for the differences in the trials) might generate the hypothesis to drive a new trial but are inconclusive in themselves. I was surprised therefore to see the article’s conclusion given pride of place on the BMJ’s front cover; I am used to colourful pharmaceutical sales aids showing me non significant differences but hoped the BMJ might protect me! 1. Lim E, Ali Z, Ali A, Routledge T, Edmonds L, Altman DG, Large S. Indirect comparison metanalysis of aspirin therapy after coronary surgery. BMJ 2003;327:1309-11. Competing interests: None declared
Hype over substance 9 December 2003
Saul G Myerson, Clinical lecturer in cardiovascular medicine John Radcliffe Hospital, Oxford Send response to journal: Re: Hype over substance	I agree wholeheartedly with Dr. Grant above. I find it hard to understand how such a poor quality piece of research is published. let alone given pride of place. It does the BMJ no favours at all if it wishes to earn a reputation as a publisher of quality papers. This is a meta-analysis of studies of aspirin vs. placebo trials. There is not a single trial comparing doses and the authors use dubious means to make this comparison. Even allowing for this, the end result was not significant in a total of over 3000 patients. To report a "trend" rather than "no significant difference" is creative reporting at best. The authors completely ignore a large body of literature suggesting no or little difference between outcome with different doses in native coronary arteries. Why this should be so different post-bypass surgery is unclear. Do the editors read these papers with a critical eye before they choose to publicise them, or do they see the potential for media hype alone? Competing interests: None declared
The trials differ too much to make indirect comparisons 9 December 2003
Christopher J Cates, General Practitioner Manor View Practice, Bushey, WD23 2NN Send response to journal: Re: The trials differ too much to make indirect comparisons	The purpose of randomisation in a clinical trial is to try to ensure that the two groups of patients in the trial are on average the same, with the exception of the treatment under study. There are no trials that have randomised patients to medium or low dose aspirin so the authors of this meta-analysis have made indirect comparisons between the results of different doses of aspirin compared to placebo in different trials. How do we check whether the trials are similar enough to make indirect comparisons? One way to do this is to examine the event rates in the placebo arms of all the trials, as they should be broadly similar if all the trials include patients, outcomes and co-interventions that are comparable. The raw data are helpfully included in this paper, and using the table in the full text version of the paper it is possible to compare the rate of patients having events in the placebo arm of each trial. A Chi square test can be used to see if the event rates vary more than would be expected by chance alone. In this case the variability is well beyond the play of chance (p=0.0004). It is possible that the relative risk of patients having a graft occlusion might be stable in spite of the variation in rates in the control group, but this does not seem to be the case. When the effects of treatment are examined in the medium dose aspirin group (using the patient event rates shown in the meta-analysis) there is a big discrepancy between the relative risks (RR =0.39 for Gavaghan and RR = 0.79 for Goldman). When these two trials are compared using this effect measure the Chi square test shows that this is again beyond the play of chance (p = 0.04). This heterogeneity is not mentioned by the authors or incorporated in the calculations of the Relative Risk Ratio for patient events. The authors point out that coronary angiography was carried out at 10 days in the Sanz trial. They do not mention that the Gavaghan trial also assessed patients at this stage (as well as the results shown for 12 months). In the Sanz trial at 10 days the graft occlusion rate is 18% in the placebo arm but in the Gavaghan trial it is only 6% at the early stage. Again this suggests that the two trials are not comparable. The conclusion that medium dose aspirin may have an advantage over lower dose aspirin is dependent on a non-significant trend based mostly on the data derived from the Sanz, Gavaghan and Goldman trials, and in my view these trials are not similar enough to make any meaningful indirect comparisons between medium and low doses of aspirin. P.S. The data in the table in the full version of this paper is not the same as in the abridged version, could the authors clarify which is correct? Competing interests: None declared
Authors' Reply 12 December 2003
Eric Lim, Specialist Registrar in Cardiothoracic Surgery Papworth Hospital, Cambridge CB3 8RE Send response to journal: Re: Authors' Reply	Editor, Ascertaining the correct dose of antiplatelet therapy is vitally important, because dosage adjustment is simple, inexpensive and has the potential to bring about large effects at a population level. It is true that the available literature does not give us a definitive answer, and conventional levels of significance has not been achieved with p=0.10. But one should not rely on hypothesis testing and take P<0.05 as the sole criterion when making inferences.(1) The point of our paper was to argue that just because the two doses are not seen to be significantly different it does not follow that they are equivalent.(2) The evidence base is limited and the confidence intervals are wide, but the data offer some suggestion that medium dose may be better. Both Dr Grant and Dr Myerson believe that our paper should not have been published because there was not a statistically significant difference between the doses. This argument might have had some validity had we suggested that medium dose was superior to low dose, but it has none at all when we are arguing that the data do not support a conclusion of equivalence. Dr Myerson asserts that there were over 3000 patients. There was in fact less than half this number of whom just 385 had events, this is not a large number. The word “trend” is given in inverted commas but it is Dr Myerson’s term - we did not use the word. Who is being creative here? Trying to extrapolate results of a 'large body of evidence' in native coronary arteries to patients undergoing coronary surgery is injudicious. Incisions into the coronary arteries and conduits exposes collagen and stimulates platelet aggregation under conditions of intense platelet activation due to trauma and extracorporeal circulation, these events do not occur naturally in native coronary arteries. Indirect comparison meta-analysis is not “dubious” but a valid statistical technique(3). Of course, additional assumptions are made when comparing two sets of trials, as we explained in some detail. Whether the indirect analysis is justified is a matter of judgment. The same applies to the marginally significant heterogeneity between the two medium dose trials. Dr Cates draws our attention to the occlusion rates in the placebo arms of the trial by Gavaghan being 6% (average of 7 days) compared to Sanz of 18% (average of 10 days) in the early stage. However, the early occlusion rates of the placebo arm in Goldman’s study (average 9 days) were visually estimated to be 14% (figure 4 of his paper), results that are broadly comparable with Sanz. Also, the baseline characteristics appear similar in terms of coronary risk factors as a measure of the risk for graft occlusion (we accept that information on smoking was sparse). The tables provided in the full manuscript are the correct version. In summary, there is a lack of evidence that low and medium dose regimens are equivalent and a suggestion that medium dose may be superior. It is important not to confuse the issue by reference to lack of conventional statistical significance, unfamiliarity with indirect comparison meta-analysis, or trying to invoke the results from trials that clearly do not apply to this subset of patients. The uncertainty can only be resolved by head to head comparison. Eric Lim [1] & Douglas G Altman [2] [1] Department of Cardiothoracic Surgery, Papworth Hospital, Cambridge CB3 8RE, [2] Cancer Research UK/NHS Centre for Statistics in Medicine, Institute for Health Sciences, Oxford OX3 7LF References 1. Altman DG, Machin D, Bryant TN, Gardner MJ. (eds) Statistics with confidence. 2nd Edition: BMJ Books, 2000. 2. Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ 1995;311:485. 3. Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 2003; 326: 472-4. Competing interests: None declared
Independent Prospective Inclusion Criteria are vital for unbiased Meta-analysis 22 December 2003
Joel Dunning, RCS Research Fellow ,and Editor of Best Evidence in Cardiothoracic Surgery in the ICVTS Manchester Royal Infirmary, On Behalf of the EBM Cardiothoracic Journal Club at Wythenshawe Hospital Send response to journal: Re: Independent Prospective Inclusion Criteria are vital for unbiased Meta-analysis	Lim and collegues addressed a very important issue in their article on the Dose of aspirin post CABG(1). Cardiac Surgeons are often of the opinion that 300-325mg of aspirin is the optimal dose and indeed this was the conclusion of the ACCP Consensus Conference of 2001 (2). Lim et al are to be congratulated for attempting to address this issue and for publicising it. However in order to convince non-specialists by the method of meta- analysis extreme rigour to good methodology must be applied. This is repeatedly advocated by the Cochrane Collaborative, NICE and many other professional systematic review bodies. Central to this is the application of strict and sensible inclusion criteria for studies into meta-analysis prior to performing the statistics. It is therefore very disappointing to see that no consideration was given to the timing of angiography as an inclusion criteria. From first principles it is very clear that 9 days of aspirin therapy will not reduce the relative risk of graft occlusion in the same way that a year of aspirin might do. This is at the heart of the methodological flaw of this paper as it can be seen in the forest plot that the only study with an inferior relative risk reduction for aspirin is the study by Sanz et al , and yet this study contributes 47% of the weighting to this arm of the study for the low dose aspirin group. It can furthermore be seen in this forest plot that if this study had been excluded no difference would have been seen between the low dose and medium dose groups. The full findings of this paper can therefore easily be ascribed to treatment effect with time as, in the low dose group, 50% of the relative risk comes from patients who have had Aspirin for 9 days and 50% from patients who have had aspirin for 180 days and all the data for medium dose aspirin comes from patients who have had aspirin for 365 days. This paper should serve as a warning to those who read meta-analyses that it is not always the complex statistics that readers should look at, but the basics of how papers have been selected for analysis that is the corner-stone of a good meta-analysis. 1)Eric Lim, Ziad Ali, Ayyaz Ali, Tom Routledge, Lyn Edmonds, Douglas G Altman, and Stephen Large Indirect comparison meta-analysis of aspirin therapy after coronary surgery BMJ 2003; 327: 1309-0 2)Stein P, Dalen J, Goldman S, Theroux P. Antithrombotic therapy in patients with saphenous veins and internal mammary arterybypass grafts. Chest 2001;119:278S–82S. Competing interests: None declared
Aspirin after coronary bypass surgery - medium versus low dose improves graft patency 7 January 2004
Thomas Hohlfeld, Clinical Pharmacologist University of Duesseldorf, Moorenstrasse 5, 40225 Duesseldorf, Germany Send response to journal: Re: Aspirin after coronary bypass surgery - medium versus low dose improves graft patency	Editor, the recent meta-analysis by Lim and coworkers concludes that medium dose aspirin (300-324 mg/day) more successfully reduces graft occlusion and events after bypass surgery than low dose aspirin (< 150 mg/day).(1) This contradicts the common view that low dose aspirin is at least as effective for coronary prevention than higher doses, as reinforced by the meta-analysis of the Antithrombotic Trialist´s Collaboration.(2) Hence, aspirin´s appropriate dose remains a complex issue. The authors correctly state that the biological basis of the impaired response to aspirin in these patients is not clear and that the meta- analysis approach leads to uncertaincy. Recent results from our laboratory, which appeared while Lim´s analysis was already in press, provided additional evidence. We found that oral aspirin (100 mg), administered daily after coronary surgery, was not only ineffective in preventing thromboxane-dependent platelet activation and thromboxane formation within 10 days after surgery.(3) A remarkably attenuated antiplatelet effect was also seen when aspirin was added in vitro to platelets, indicating a pharmacodynamic mechanism of aspirin ´resistance´ (type 2-resistance).(4) Moreover, a chemically different NSAID (indomethacin) was also significantly less effective. An increase of the concentration of aspirin restored platelet inhibition. The molecular mechanism of the reduced platelet responsiveness to aspirin is currently under investigation. Our recent work, therefore, strongly supports the dose-dependency suggested by Lim and coworkers. From this and the widespread discussion on aspirin resistance, it becomes increasingly evident that an optimum antiplatelet effect is not always achieved by low-dose aspirin. Some groups of patients may require higher daily doses. Those who underwent coronary surgery are an example. It should not be overseen, however, that the response to aspirin within this (3) and probably other patient populations is highly heterogeneous. For a rational therapy with aspirin, we need practical strategies which allow to determine the optimum dose of aspirin for individual patients. Probably not only after bypass surgery. References (1) Lim E, Ali Z, Ali A, Routledge T, Edmonds L, Altman DG, Large S. Indirect comparison meta-analysis of aspirin therapy after coronary surgery. BMJ 2003; 327:1309 (2) Antithrombotic Trialist's Collaboration. Collaborative meta- analysis of randomised trials of antiplatelet therapy for prevention of death, myocardial infarction, and stroke in high risk patients. BMJ 2002; 324:71-86 (3) Zimmermann N, Wenk A, Kim U, Kienzle P, Weber A-A; Gams E, Schrör K, Hohlfeld T. Functional and biochemical evaluation of platelet aspirin resistance after coronary artery bypass surgery. Circulation. 2003; 108:542-547 (4) Weber A-A, Przytulski B, Schanz A, Hohlfeld T, Schrör K. Towards a definition of aspirin resistance: a typological approach. Platelets 2002; 13:37-40 Competing interests: None declared
First principles or evidence based critique? 9 January 2004
Eric Lim, SpR Cardiothoracic Surgery Papworth Hospital Send response to journal: Re: First principles or evidence based critique?	Editor, It is disappointing to read Dr Dunning’s critique of our meta- analysis and even more disappointing when a ‘proponent’ of evidence based medicine refers to ‘first principles’. Let us leave ‘first principles’ aside and talk about evidence. The temporal profile of the effects of aspirin on graft occlusion is behind the current debate with regards to our choice of including the trial by Sanz who performed angiography at a mean of 10 days. The question of the temporal profile of the effects of aspirin therapy is best answered by examining the early and late angiography performed in both the Gavaghan(1) and Goldman(2) trials. There are 2 important points: 1) The rate of graft occlusions is highest in the first 9 days. From the table provided you can see that a significant number of occlusions occur in the first 7-9 days, as compared to the remaining 356- 358. To quote Goldman (2) ‘The frequency of new graft occlusions in the patients with patent grafts at 9 days was not improved by aspirin administration during the subsequent year’ 2) The effects of aspirin are established by the first 9 days. When comparing relative risks, time as an isolated factor is not important, but rather any change in the relative cumulative graft occlusion over time. Figure 4 of the Goldman paper, shows very nicely that the difference at day 9 days mirrors the difference at day 367. The crude relative risk comparing aspirin to placebo for graft occlusion at 9 days was 0.47 in the Goldman trial increasing to 0.70 at one year and 0.26 at 7 days in the Gavaghan trial increasing to 0.50 at one year. (The relative risks at a year differ from our original tables, because we now include data from patients in all aspirin combinations of the Goldman trial, and rounded percentages instead of actual numbers from the Gavaghan trial.) This suggests that including Sanz in our meta-analysis would, if anything, provide a bias in favour of superior results for the low dose group, and that the results of low dose aspirin may be even poorer if all low dose trials had angiography at one year. In fact, the relative effect of aspirin diminishes with time, and this is counterintuitive to the impression of Dr Dunning, and another example of why we should not apply ‘first principles’ based critique. If you take away Sanz, the difference between the doses is indeed lowered, but the confidence interval becomes much wider. This does not provide support of equivalence, but rather increases uncertainty about the estimated difference between the two dosing groups.(3) The prospective inclusion of Sanz was a deliberated process to specifically evaluate low and medium dose aspirin on vein graft patency. We consider our inclusion criteria to be much more stringent than others who have come to a similar conclusion from meta-analyses that did not discriminate between the doses of 50-325mg, studies that included arterial grafts (where aspirin has no effect), consensus statements and omission of relevant valid randomised trials.(4) We thank Dr Hohlfeld for his letter, although he rather overstates our conclusion. We would like to mention that based on his previous work, we are also evaluating the biological effects of low and medium dose aspirin therapy on platelet aggregation after coronary surgery in Cambridge, UK. Eric Lim [1] & Douglas G Altman [2] [1] Department of Cardiothoracic Surgery, Papworth Hospital, Cambridge CB3 8RE, [2] Cancer Research UK/NHS Centre for Statistics in Medicine, Institute for Health Sciences, Oxford OX3 7LF Table of total occlusions Goldman (9 days, 367 days) Aspirin arms* 7%, 15.8% Placebo arm* 15%, 22.6% Gavaghan (7 days, 365 days) Aspirin arm 1.6%, 5.8% Placebo arm 6.2%, 11.6% * Data derived from Figure 4 of Goldman et al. References (1) Gavaghan T, Gebski V, Baron D. Immediate postoperative aspirin improves vein graft patency early and late after coronary artery bypass graft surgery. A placebo-controlled, randomized study. Circulation 1991;83(5):1526-1533. (2) Goldman S, Copeland J, Moritz T, Henderson W, Zadina K, Ovitt T, Doherty J, Read R, Chesler E, Sako Y. Saphenous vein graft patency 1 year after coronary artery bypass surgery and effects of antiplatelet therapy. Results of a Veterans Administration Cooperative Study. Circulation. 1989;80:1190-1197. (3) Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311:485. (4) Dunning J, Das S. What is the optimal dose of aspirin after discharge following coronary bypass surgery. Interactive Cardiovascular and Thoracic Surgery 2003. Article 216 Competing interests: None declared
Can we replace uncertainty with certainty ? 13 January 2004
Joel Dunning, RCS Research Fellow Manchester Royal Infirmary Send response to journal: Re: Can we replace uncertainty with certainty ?	I thank Lim et al for their very comprehensive reply in response to my doubts about the validity of the inclusion of the Sanz trial into their meta-analysis. Mr Lim and collegues have certainly carefully considered many of the factors leading to the heterogeneity between the 5 studies included in the meta-analysis, and as they state in their article there are indeed many reasons why each of the studies have such different findings. This paper reminds me of another paper by David Taggart, Douglas Altman et al [1] Where Bilateral internal mammary artery(BIMA )usage was compared to left internal mammary artery(LIMA) use by systematic review and meta-analysis. However although the results were statistically significant, Taggart et al warned against widespread adoption until a randomized controlled study of adequate power had been performed. He went further by providing the power calculation in that article, indicating that a study of 4200 patients was needed. In addition he has now received substantial funding to perform this study to provide us with the answer. If a similar power study is calculated for this paper then if a 20% placebo vein graft occlusion rate is assumed, low dose aspirin would reduce this to 15% and medium dose aspirin to 11% and thus 2300 patients would be required to have an 80% chance of finding a statistically significant result, in a prospective RCT in the modern era. Eric Lim has a prolific publication history and Papworth is a highly respected unit, thus perhaps instead of advocating widespread use of 325mg of Aspirin currently, we should take on board all the caveats mentioned in the article, state that there is no statistically significant difference between low dose and medium dose aspirin and call for Mr Lim et al at Papworth to head a well funded multicentre trial on this important topic, just as David Taggart did in Oxford for BIMA Vs LIMA. [1]Taggart DP, D’Amico R, Altman DG. Effect of arterial revascularisation on survival: a systematic review of studies comparing bilateral and single internal mammary arteries. Lancet 2001 : 358: 870- 874. Competing interests: None declared
Reservations with Indirect Comparisons 1 September 2005
Steven A. Julious, Senior Lecturer in Medical Statistics Medical Statistis Group, University of Sheffield, S5 7AU Send response to journal: Re: Reservations with Indirect Comparisons	I do have reservations about indirect comparisons.  To highlight my concerns I have constructed the following table from the paper for the placebo responses   Table 1.  Placebo responses by year paper was published Year Published Occlusion Rate Event Rate Comparison To 1984 0.35 0.63 Low Dose 1989 0.23 0.44 High Dose 1990 0.18 0.33 Low Dose 1991 0.11 0.30 High Dose 1993 0.09 0.17 Low Dose   From this table it is evident that the placebo response improves for both endpoints with time.  This effect is known as placebo creep.  There are a number of plausible reasons that could be conjectured as to why we observe such responses.  One reason could be due to most trials being in fact adjunct trials with concomitant therapies or forms of care in addition to the investigative treatments.  The end result could be that as these concomitant treatments improve over time then so does the observed response of the investigative treatments randomised on top of them (in this context placebo creep may also equate to a narrowing of the active treatment effect over placebo). The consequence of placebo creep is that although each individual trial will provide an unbiased assessment of treatment response within a time caution should be exercised when making comparisons across trials (and time).  In the presence of placebo creep it is not enough just to assess whether different studies are clinically or demographically the same as trials could have similarly representative populations and yet have observe different effects.  Where appropriate meta regression methods should be applied to account for time.  Where this is not appropriate comment should be made as to how not allowing for placebo creep could bias the results. Competing interests: None declared