Association of onset to balloon and door to balloon time with long term clinical outcome in patients with ST elevation acute myocardial infarction having primary percutaneous coronary intervention: observational study
BMJ 2012; 344 doi: https://doi.org/10.1136/bmj.e3257 (Published 23 May 2012) Cite this as: BMJ 2012;344:e3257All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
During an automated survey of MEDLINE abstracts for potential errors in percent-ratio calculations, we noticed the 2012 BMJ paper by Shiomi et al(1) stood out from the rest as the only high impact paper where all the reported percent-ratio pairs in the abstract had discrepancies when their values were extracted from the text and recalculated algorithmically. Because this error-checking algorithm is not without its own error rate, we felt compelled to investigate further and analyzed the full-text as well.
Out of the 27 ratios reported within their paper, 23 of them do not match the percent values immediately next to them within the text (see recalculated values in Table 1). The reported relative risk reduction (RRR) numbers match the percents associated with them. We understand their statistical design includes a survival analysis, which may take into account that some of the patient population has been censored due to gaps in follow-up data or death by unrelated causes, and this may have been taken into account when calculating the percentages.
However, only the ratio can be used to recalculate p-values, assuming no censoring. If we go by the percents, it is not clear what their underlying ratio was and if we use the ratios to estimate significance, the recalculated RRR and p-values do not match (see Table 2). In three cases, it would change the conclusion regarding whether a result was significant or not by the criteria used by the authors (2-tailed test, p<0.05). Because our algorithm scanned all MEDLINE abstracts, we were able to determine that the vast majority of other studies (>97%) that reported patient survival data and risk ratios similar to the values reported here did not have detected discrepancies. Thus, we at least feel confident that this pattern of reporting data is not the norm (i.e., ratio-percent pairs reported immediately next to each other are normally intended as paired values).
It may be that a key piece of information (e.g., number of patients censored) was not made explicit in the text, and this may be the source of the discrepancies between ratios and percents and in our p-value recalculations. Nonetheless we feel that this is important for the authors to clarify.
References
1 Shiomi, H. et al. Association of onset to balloon and door to balloon time with long term clinical outcome in patients with ST elevation acute myocardial infarction having primary percutaneous coronary intervention: observational study. Bmj 344, e3257, doi:10.1136/bmj.e3257 (2012).
Competing interests: No competing interests
Re: Discrepancies in reported vs calculated percent-ratio pairs in the Shiomi et al 2012 BMJ paper
We are afraid that Jonathan D. Wren has ignored the fact that the presentations of results were chosen by the study design and clinical implications, as well as misunderstanding the definition of “relative risk” in limited sense.
In his letter, he pointed out the discrepancies between our reported relative risk reductions (RRR) based on the Kaplan-Meier estimate and their calculated RRR based on the contingency table without taking censoring into account.
According to the journal style, we presented event rates using the Kaplan-Meier estimate, number of events/total number of patients, RRR for each comparison in our paper. In randomized clinical trials (RCTs), event rates calculated by contingency tables were considered robust because the censored observations should be included in the denominator based on the intention-to-treat (ITT) principle. Considering relatively frequent occurrences and larger differences between groups in censoring rates in observational studies, event rates using the Kaplan-Meier estimate, instead of a contingency table method without taking censoring into account, should reflect real world clinical practice. Therefore, we presented the event rates using the Kaplan-Meier estimate as “risks” in our observational study. As a result, calculated RRRs using the Kaplan-Meier estimate were different from RRRs using the contingency table method. In our study, for example, the cumulative incidence of a composite of death/congestive heart failure at 3 years was 13.5% (123 events/964 patients at day 0) in the OTB time <=3 h group and 19.2% (429 events/2427 patients at day 0) in the OTB time > 3h group (Log-rank P<0.001), and RRR was 29.7% (1-rerative risk = 1-13.5%/19.2%). The definition of RRR was not fixed in the contingency table method, and “relative risk” or RRR are applicable to other metrics of “risk.” The majority of published papers in major journals report results of RCTs, and minority of papers present the exact numbers of events and cohort patients in their abstract, however, we preferred to report them to show the credibility of our report and such exact numbers should be a benefit to live readers, especially those who want to conduct systematic reviews. Unfortunately, the author's created automatic program to detect errors of MEDLINE abstract could not take into account the core of clinical studies.
Hiroki Shiomi MD, PhD
Assistant Professor of Department of Cardiovascular Medicine,
Kyoto University Graduate School of Medicine, Kyoto, Japan
Takeshi Morimoto, MD, MPH
Professor of Department of Clinical Epidemiology,
Hyogo College of Medicine, Hyogo, Japan
Yoshihisa Nakagawa, MD
Director of Division of Cardiology,
Tenri Hospital, Nara, Japan
Takeshi Kimura, MD
Professor of Department of cardiovascular Medicine,
Kyoto University Graduate School of Medicine, Kyoto, Japan
Competing interests: No competing interests