Rivaroxaban: can we trust the evidence?BMJ 2016; 352 doi: https://doi.org/10.1136/bmj.i575 (Published 03 February 2016) Cite this as: BMJ 2016;352:i575
All rapid responses
Updates on the validity of ROCKET-AF conclusion based on issues with point-of-care INR results: Defence or Defiance?
I believe that interested readers would like to be informed on the recent published letter from Dr JR Powell regarding his concerns with point-of-care INR readings used in the ROCKET-AF and the study author's response:
However I do not believe NEJM has published any letters from Thomas Marcinicak mentioned in this BMJ article. No free access to datasets has been offered as far as I can tell.
For some, the veil of secrecy that remains even when significant allegations of the study validity has been openly voice, will only result in a slow erosion of confidence from lingering doubts
Competing interests: No competing interests
We read with interest the recent BMJ editorial “Rivaroxaban:Can we trust the evidence”.1 which questions the conclusions from the ROCKET AF clinical trial on Rivaroxaban 2. Cohen raises questions about the accuracy of the INRs used for monitoring patients in the warfarin control arm and suggests that they may have compromised the validity of the published conclusions. The ROCKET investigators have defended their conclusions after re assessing the data in light of these concerns 3. The device used for INR testing in the warfarin control arm was not identified in the original publication but subsequently has been revealed as the INRatio Point of Care (POC) device. This device was the subject of a safety recall notice by FDA in 2014 due to “clinically significantly lower” readings compared to laboratory values, with the fault going back to 2002 (before the Rocket AF trial started). The BMJ editorial points out that a falsely low INR reading could mean that patients in the control arm may have had their warfarin dose unnecessarily increased leading to a greater risk of bleeding.
To ensure accuracy, POC INR testing should be subjected to the same level of quality control (QC) and quality management systems that laboratory INR testing has to meet. Appropriate and effective External Quality Assessment (EQA) for POC INR testing devices is possible 4 and there has been a POC INR EQA programme available in the UK for 20 years. In the UK NEQAS for Blood Coagulation EQA programmes we provide suitable material for several different devices and have over 4500 users enrolled in the programmes (January 2016). The POC device used in the warfarin control arm of the ROCKET AF study 2 is one which we evaluated to assess our EQA material suitability in 2010 but found we were unable to provide any suitable EQA material for this technology. To our knowledge no other programme provides EQA for this device. The lack of EQA and internal quality control (IQC) makes it difficult to assess the reliability of INR results obtained with this device.
Laboratory based INR testing in many countries is subject to accreditation, often against international ISO standards (ISO 15189) 5 which require participation in regular EQA as part of the quality management system. Currently the vast majority of POC INR testing in many countries is not subject to such accreditation and not all POC sites participate in EQA even when this is available. Although such participation for POC sites is not currently mandated by regulation, we believe it should be obligatory. Non-participation despite availability could have serious medico-legal implications in the event of untoward incidents. If the use of INR methods in the laboratory or in point of care was restricted to those for which satisfactory participation in EQA can be demonstrated, the issues of clinically significant problems in INR testing such as those which have led to the questioning of the ROCKET AF study could be more easily avoided.
POC INR testing has been in use for over two decades and in our view is safe, providing continuous QC assessment is in place in order to ensure that the INR results produced are valid and accurate 6.There are several published national and international guidelines stating the essential requirement for QC to ensure reliable results 7-9 when using POC INR devices. It would be inappropriate to view all POC devices for INR testing to be inferior to the laboratory measurements as this is not usually the case. POC devices can be accurate and precise giving reliable results in a very quick turnaround time. The key is to ensure the accuracy and precision of all INR results, whether derived from a POC device or from a laboratory method, with the use of both internal control and external quality assessment.
1.Cohen D. Can we trust the evidence? BMJ 2016,352,179-180
2. Patel MR, Mahaffey KW, Garg J, Pan G, Singer DE, Hacke W, Breithardt G, Halperin JL, Hankey GJ, Piccini JP, Becker RC, Nessel CC, Paolini JF, Berkowitz SD, Fox KAA, Califf RM, and the ROCKET AF Steering Committee, for the ROCKET AF Investigators* N Eng J Med 2011:365; 883-891
3.Patel MR, Hellkamp AS, Fox KAA, and the ROCKET AF executive committee and investigators. Point–of–Care Warfarin monitoring in the ROCKET AF Trial. N Eng J Med 2016:374-378
4 Kitchen DP, Kitchen S, Jennings I, et al. Point of Care INR testing devices: performance of the Roche CoaguChek XS and XS Plus in the UK NEQAS BC external quality assessment programme for healthcare professionals: four years’ experience. J Clin Pathol 2012;65(12):1119–1123
5 The British Standards : Medical laboratories- Requirements for quality and competence ISO 15189
6. Kitchen S, Kitchen DP , Jennings I, Woods TAL, Walker ID , Preston FE. Point of Care INRs : UK NEQAS experiences demonstrates necessity for proficiency testing of three different monitors. Thromb Haemost 2006 : 96; 590-6.
7.CLSI Point –of –Care monitoring of anticoagulation therapy: Approved guideline. CLSI document POCT14-A Wayne PA:Clinical and laboratory standards institute 2004
8. Keeling D, Baglin T, Tait C, et al; British Committee for Standards in
Haematology. Guidelines on oral anticoagulation with warfarin -fourth edition. Br J Haematol 2011;154(3):311–324
9. Jennings I, Kitchen D, Keeling D, et al; BCSH Committee. Patient self-testing and self-management of oral anticoagulation with vitamin K antagonists: guidance from the British Committee for Standards in Haematology. Br J Haematol 2014;
Competing interests: No competing interests
I would like to add my concern to that of D. Cohen regarding the quality of evidence of the ROCKET-AF trial that led to regulatory approval of rivaroxaban for stroke prevention in non-valvular atrial fibrillation.
In the warfarin-treated control group, the mean Time in Therapeutic Range (TTR) was reported to be 55% - significantly less than the 62% and 64% reported respectively in the apixaban (ARISTOTLE) and dabigatran (RE-LY) trials. In the methods section of the published reports of all the above trials it is stated that TTR was calculated according to Rosendaal’s linear interpolation method. However, in the Supplementary Appendix (Table 5) available online with the full text of the ROCKET-AF report at NEJM.org it is stated that “Center TTR is calculated using total number of international normalized ratio (INR) values in target range from all
warfarin subjects within a center divided by total number of INR values from all warfarin subjects within the center”. This percentage is a measure of anticoagulation quality entirely different from Rosendaal’s TTR and may yield values which deviate significantly from the mean of individual TTRs. This discrepancy is confusing and raises further questions concerning the reliability of the ROCKET-AF anticoagulation data.
All individual patients’ data should be made public, if our trust in the findings of this pivotal trial is to be restored. The campaign for open access to RCT data should be intensified.
1. D. Cohen. Rivaroxaban: can we trust the evidence? BMJ 2016;352:i575
2. Rosendaal FR, Cannegieter SC, van der Meer FJ, Briet E. A method to determine the optimal intensity of oral anticoagulant therapy. Thromb Haemost 1993;69:236-239
Last accessed 12-2-2016
Competing interests: No competing interests
It is with concern that we read the Editorial in the BMJ (1) regarding the ROCKET-AF trial comparing rivaroxaban (Xarelto®) with a warfarin treated control group. It has been brought forth that the warfarin-managed group had their INR measured on a coagulometer, which has some serious drawbacks. Here we will focus on the point of care coagulometer (POCT) used in the trial; discuss the impact on the results and the requirements for the use of a POCT in trials.
Generally, in order to provide solid evidence in terms of non-inferiority (or even superiority) of rivaroxaban compared with warfarin, the latter needs to have a sufficiently high time within therapeutic target range (TTR) – preferably ≥70%. In the ROCKET-AF trial (2) the TTR in the warfarin group was merely 55%, which reflects a suboptimal quality of oral anticoagulant therapy (3). Using patient self-testing and self-management a higher quality of treatment can be achieved for selected patients. (4) High TTR has been linked to improved efficacy and safety outcomes.(5)
In a previous paper we concluded, that the precision of the POCT coagulometers was generally adequate for clinical use, but their performance in terms of accuracy has to be viewed in the context of the inherent inaccuracies in INR measurements. The overall accuracy of POCT coagulometers seems, in this respect, to be generally acceptable for everyday clinical practice and they may be used in a clinical setting.(6) We also showed that the HemoSense (INRatio®) had not been investigated in any high quality studies and the precision and accuracy of that specific coagulometer was lower compared to some of the other coagulometers on the market. Therefore, one may wonder from an analytical perspective why the investigators of the ROCKET-AF study choose to use HemoSense (INRatio®) in a clinical trial?
A potential underestimation of the INR value compared with a laboratory standard is the concern. This could potentially lead to a higher risk of bleeding in the warfarin group and subsequently misleadingly favour rivaroxaban over warfarin in a direct comparison on safety outcomes. The authors of the ROCKET-AF trial have responded that INR values were merely low in patients with certain conditions in terms of e.g. chronic inflammation, acute inflammation, or hematocrit levels out of range. Furthermore, it is advocated, that this did not influence the conclusions of the study regarding safety endpoints.(7) It should be emphasized that a statistical insignificant result may still be regarded as a clinical relevant result. Additionally, it can be seen that the previous statistically significant and clinically relevant reduction in intracerebral haemorrhage for the rivaroxaban arm was not observed in the group with the conditions mentioned above. For the outcome of major bleeding the subgroup of patients with none of the conditions favoured treatment with rivaroxaban compared with warfarin, hazard ratio 0.87 (95% CI, 0.70-1.08); while patients with any of the conditioned displayed a hazard ratio of 1.18 (95% CI, 0.98-1.42).7 These results are counterintuitive with the hypothesis that patients with the mentioned conditions could have received a higher dose of warfarin resulting in an increased risk of bleeding. It could also indicate that:
1) This subgroup is not the patients causing problems in relation to INR measurements,
2) The analytical error does not affect the over all conclusion.
However, the overall statement that the results of the coagulometer (and conditions) had no influence on the outcome is somewhat speculative, and further enhances the doubt on the conclusions from the trial that has already been raised.(1)
It has been argued that a quality control of these coagulometers is necessary, which can be performed using several different setups.(8) Hereby, flaws in INR measurements than can be attributed to analytical variations would probably be detected, but these quality controls would not predict flaws attributable to the biological deviations in the individual patient as those described above.
In a clinical trial there must be transparency regarding which type of coagulometer is used, and the quality of INR measurement should be high and precise. This could have been fortified in the ROCKET-AF trial by using a more validated coagulometer and quality control of the INR values, e.g. by using INR measured in a certified laboratory.
We recommend that coagulometers used in a clinical study should be those with the evidence of the best performance in terms of accuracy and precision. Otherwise, the obtained results should be interpreted with caution and may limit the clinical impact in terms of changing treatment strategies.
1 D Cohen. Rivaroxaban: can we trust the evidence? BMJ 2016;352:i575
2 Patel MR, Mahaffey KW, Garg J, et al. Rivaroxaban versus warfarin in nonvalvular atrial fibrillation. N Engl J Med 2011;365:883-91.
3 Christensen TD, Larsen TB, Hasenkam JM. Results of Vitamin K antagonist treatment in patients with atrial fibrillation depends on the type of management. Circulation 2012;126:e45.
4 Nilsson H, Grove EL, Larsen TB, et al. Sex differences in treatment quality of self-managed oral anticoagulant therapy: 6,900 patient-years of follow-up. PLoS One 2014;9:e113627.
5 De Caterina R, Husted S, Wallentin L, et al. Vitamin K antagonists in heart disease: current status and perspectives (Section III). Position paper of the ESC Working Group on Thrombosis--Task Force on Anticoagulants in Heart Disease. Thromb Haemost 2013;110:1087-107.
6 Christensen TD, Larsen TB. Precision and accuracy of point-of-care coagulometers used for self-testing and self-management of oral anticoagulation therapy. Journal of Thrombosis and Haemostasis 2012;10:251-260.
7 Patel MR, Hellkamp AS, Fox KAA, et al. Point-of-Care Warfarin Monitoring in the ROCKET AF Trial. N Engl J Med 2016 Feb 3 [Epub ahead of print].
8 L Poller. Precision and accuracy of CoaguChek S and XS monitors: The need for external quality assessment. Thromb Haemost 2009;101:419–421.
Competing interests: Peter Brønnum Nielsen has been on the speaker bureaus for Boehringer Ingelheim. Torben Bjerregaard Larsen has served as an investigator for Janssen Scientific Affairs, LLC and Boehringer Ingelheim and has been on the speaker bureaus for Bayer, BMS/Pfizer, Roche Diagnostics, Takeda and Boehringer Ingelheim. Thomas Decker Christensen has been on the speaker bureaus for AstraZeneca, Boehringer Ingelheim, Pfizer, Takeda, Bristol-Myers Squibb and Roche Diagnostics.
Previous articles by des spence and a rapid response last week re the real risks of af make this article even more scaring. We like most GP routinely review our pts with af and advice nearly all of them to be anti coagulated. Anecdotal evidence with all the bias this bring in Hints that just in our practice unexplained significant drops in haemaglobin without overt acute bleeds are not rare in those on Noacs and this product is the CCG favourite due to its cost. Indeed were all those in our CCG put on a noac the savings using this product are I understand about £300,000 compared to other Noacs. As the CCG has no spare money we really need a true comparison between products and also need for treatment vs risks especially in those with chads score of only 1 -over a lifetime of treatment . I remain a firm believer in preventing strokes but ultimately it is patient choice.
Competing interests: No competing interests
The BMJ must continue its campaign for open access to RCT data, without fear or favour (http://www.bmj.com/content/350/bmj.h3337).
Sadly, in an intensely competitive Pharma world, the companies are unlikely to throw open their most intimate secrets.. but they could confirm how accurately their INR data was lab-validated ??
At least the Rocket-AF triallists have reviewed the effect of the recalled device, thus - http://www.nejm.org/doi/full/10.1056/NEJMc1515842?query=featured_home
Competing interests: No competing interests
We at the Project On Government Oversight have been writing about this issue in the United States and applaud The BMJ for drawing attention to it.
The situation The BMJ describes poses a test of the system. Having employed the suspect devices in the Xarelto trial, can the people who conducted that trial be relied upon to review their own work objectively? Having apparently overlooked the problem in their original vetting of the drug for patients with atrial fibrillation, can the regulators be counted on to conduct thorough and impartial investigations now?
As we reported in November, the brand of blood-testing device used in the Xarelto trial was the subject of two FDA warning letters before the trial began. And, as we’ve reported, one of the co-chairmen of ROCKET AF’s executive committee—the group responsible for designing the trial—is today deputy commissioner of the FDA and President Obama’s nominee to head the agency.
A footnote: Though the FDA issued a recall notice in December 2014, the recall did not require that the systems be withdrawn from the market. They remain in use. The manufacturer “is working diligently to develop a software upgrade” for the devices to “address the potential for inaccuracies,” a spokeswoman for the company told us in November.
For more coverage of this subject, see “Drug Problems: Duke Reassessing Data From Trial Led By FDA Nominee” and links to related articles at this URL:
Project On Government Oversight
Competing interests: I've written about this topic and continue to cover it as a journalist for the Project On Government Oversight (www.pogo.org).
I would like to say if Hercule Poirot ever existed he would have exclaimed: "Thank God! Someone has been using their little grey cells...."
May I congratulate Deborah Cohen and the BMJ investigative team on their time, effort, perseverance and bravery in reviewing the safety profile of Rivaroxaban. BMJ too has been instrumental last year in promoting awareness about the issue in analyses withheld by the makers of dabigatran involving the frequency of bleeding.
While there are many physicians have been heavily promoting the role of anticoagulation in non-vulvular atrial fibrillation, often NOAC (Novel Oral Anti-Coagulants) are also either openly or subtly touted as the panacea with safety profile similar or superior to warfarin, but the "bonus" of not requiring regular monitoring. In the same period there too has been a growing number of clinicians who reported concerns regarding the safety profile of these drugs in the real world, particularly with regard to what is considered "non-major" haemorrhagic events and the lack of immediate antidotes.
News with concerns about published trials involving NOACs reminds clinicians that much of the basis of what we know about the role and efficacy of oral anticoagulation upon many conditions is based on studies involving warfarin and so far (as I understand) many published NOACs studies are non-inferiority efficacy trials compared to warfarin, without the necessary power to assess their safety profile like those specifically performed by the manufacturers. NOACs thus should not be promoted as the first line drugs without further reanalysis of their safety profile.
I note with curiosity that NEJM has published a correspondence from the researchers of ROCKET-AF on the same day as this investigative feature in BMJ, and yet there is no apparent news so far about the letter submitted to NEJM for publication by Thomas Marcinicak. Keeping in mind the allegations surrounding the conduct of the editorial process involved in various controversial publications regarding the efficacy of knee arthroscopy and unpublished letters from editors of an orthopaedic journal ("thus, we are shocked when the debate is stifled." Ref 1) not so long ago, I shall look forward to see in what circumstances the Marcinicak letter is published (if at all, due to the lack of space) and (hopefully) the tirade of letters of opposing views.
Again I am reminded of the quote: Freedom is the oxygen without which science cannot breathe. (David Sanorff)
1. Could the New England Journal of medicine Be Biased Against Arthroscopic Knee Surgery? http://dx.doi.org/10.1016/j.arthro.2014.02.013
Competing interests: I have previously voiced concerns regarding safety profile of the NOACs