Primary Care

Reliability of international normalised ratios from two point of care test systems: comparison with conventional methods

BMJ 2003; 327 doi: http://dx.doi.org/10.1136/bmj.327.7405.30 (Published 03 July 2003) Cite this as: BMJ 2003;327:30
  1. Leon Poller, coordinator (ecaa{at}man.ac.uk)1,
  2. Michelle Keown, research scientist1,
  3. Nikhil Chauhan, research statistician1,
  4. Anton MHP van den Besselaar, senior scientist2,
  5. Armando Tripodi, senior scientist3,
  6. Caroline Shiach, consultant haematologist4,
  7. Jorgen Jespersen, chairman5
  1. 1European Concerted Action on Anticoagulation Central Facility, School of Biological Sciences, University of Manchester, Manchester M13 9PT
  2. 2Haemostasis and Thrombosis Research Centre, Leiden University Medical Center, Leiden, Germany
  3. 3A Bianchi Bonomi, Haemophilia and Thrombosis Centre, IRCCS Maggiore Hospital, University of Milan, Milan, Italy
  4. 4Department of Haematology, Manchester Royal Infirmary, Manchester
  5. 5Department for Thrombosis Research, University of Southern Denmark and Department of Clinical Biochemistry, Ribe County Hospital, Esbjerg
  1. Correspondence to: L Poller
  • Accepted 13 May 2003

Abstract

Design Comparison of the INRs from the two systems with a “true” INR on a conventional manual test from the same sample of blood.

Setting 10 European Concerted Action on Anticoagulation centres.

Participants 600 patients on long term dosage of warfarin.

Main outcome measures Comparable results between the different methods.

Results The mean displayed INR differed by 21.3% between the two point of care test monitoring systems. The INR on one system was 15.2% higher, on average, than the true INR, but on the other system the INR was 7.1% lower. The percentage difference between the mean displayed INR and the true INR at individual centres varied considerably with both systems.

Conclusions Improved international sensitivity index calibration of point of care test monitors by their manufacturers is needed, and better methods of quality control of individual instruments by their users are also needed.

Introduction

Demands for warfarin have greatly increased in recent years for a range of clinical states including atrial fibrillation.1 As a consequence,centres providing oral anticoagulants throughout the world are overwhelmed by demands formonitoring systems; many patients may not receive this treatment because of limited facilities.2

Innovative testing procedures at the point of care have been introduced to determine the prothrombin time for whole blood samples. These procedures do not need the technical expertise of traditional methods because the tests use unmeasured samples of blood.3

One of two monitors—CoaguChek (Roche Diagnostics, Basel)—which we studied is being introduced throughout the United Kingdom with widespread promotion in the national media. Most large centres in the United Kingdom have limited but increasing numbers of patients using CoaguChek. In Germany, 50 000 to 60 000 patients are already in self testing or self dosage programmes using CoaguChek.1

Point of care test monitors must give dependable international normalised ratios (INR) because the safety and effectiveness of warfarin depends on keeping patients within target INR ranges. Thrombotic events increase at INRs less than 2.0 and bleeding complications increase at INRs greater than 4.5.4

We used two point of care test monitoring systems which are widely marketed in the European Union—CoaguChek Mini, and TAS PT-NC (RapidPointCoag)—to test 600 long term patients who take warfarin at 10 centres taking part in the study. We compared INRs displayed on the monitors with “true” INRs found by conventional manual prothrombin time testing with World Health Organization standard thromboplastin on the same samples of blood. We also compared the INRs displayed on the two systems with each other and considered the clinical implications of discrepancies. We coded our results because we assessed only two systems of several currently available.

Materials and methods

The monitoring systems consisted of an instrument with a uniquely numbered batch of thromboplastin test strips or cards. The manufacturers of the two systems (CoaguChek Mini and TAS PT-NC5) provided their systems to each of the 10 European Concerted Action on Anticoagulation centres.6 The CoaguChek Mini system uses rabbit thromboplastin and the TAS PT-NC uses human placental thromboplastin. We determined true INRs for all samples of blood by manual tests with WHO human standard (rTF/95)7 and WHOrabbit plain standard (RBT/90).8

With their consent, we took non-citrated venous whole blood from 60 patients stabilised on long term oral anticoagulants at each centre. We tested each sample on both monitor systems within 15 seconds of collection and recorded the INRs which the systems displayed.

With minimum delay, we drew the rest of the sample into a 4.5 ml vacuum container containing 105 mmol/l of sodium citrate, centrifuged it at 2500 g for 10 minutes, and transferred the plasma into a plastic tube. We stored plasmas at room temperature and tested them, within six hours of collecting blood, using the recommended manual technique for conventional prothrombin time testing with both WHO reference thromboplastins in a standardised sequence.6 Representatives of all 10 centres developed and practised this standardised procedure at a workshop before the study started.

True INRs with both WHO thromboplastin standards were obtained for each plasma. The INRs were calculated from the local prothrombin time (PT), the mean normal prothrombin time (MNPT) based on 20 fresh normal samples from each centre, and the international sensitivity index (ISI) for each WHO thromboplastin standard as follows INR = (PT/MNPT)ISI.68

With a paired t test, we compared the mean INR displayed by the monitors with the mean of the true INRs of plasma from the same samples tested with WHO thromboplastin standard from the same species. We calculated P values and confidence intervals for the difference in mean INR at each centre. We also compared the overall mean INR for results at all 10 centres. We constructed Bland-Altman plots of the difference between displayed INR and true INR at different levels of treatment.

We classified an absolute deviation of INR of more than 50% from the true INR as aberrant and recorded the number of tests at each centre which gave aberrant results for both systems. We compared the mean INR displayed by the two systems using paired t tests; we also compared mean true INR derived with the two WHO thromboplastin standards.

Results

Of the 600 samples of blood tested by the two monitoring systems (coded A and B), we excluded 64 according to WHO protocol because the INRs were outside the 1.5 to 4.5 range with the relevant WHO thromboplastin immunoreactive standard.9

Since different local populations of 60 patients were tested at all 10 centres, mean INR, both true and displayed, differed at the centres. The overall mean INR displayed by the monitors of the 536 samples remaining after exclusions was considerably higher with system A than with system B (overall mean difference 21.3%).

With system A, the difference between the mean displayed INR and the true INR of the local samples varied between 0% and 34.6% at the 10 centres (table 1). At nine centres, the mean displayed INR on system A was significantly higher than the true INR, for the same samples of blood (P < 0.001). The overall difference between the mean displayed INR and the true INR was 15.2%. At seven of the 10 centres, mean differences in INRs were more than 10%, which is clinically relevant according to WHO's guidelines.9 The limits of agreement, which give a measure of the variability of individual INR results, ranged from −0.70 to 1.47 units (fig 1).

Table 1

Comparison of INR displayed by system A with true INR at the 10 centres

View this table:
Fig 1
Fig 1

Bland-Altman plots of differences in INR plotted against the mean INR displayed by (a) monitoring system A and (b) monitoring system B, and “true” INRs for the 536 samples

The difference between the displayed mean and true INR was less with system B. Mean displayed INRs were, however, 7.1% lower than true INRs (table 2). Six of the 10 centres gave statistically significant differences in meandisplayed INR—between 19.0% lower to 3.5% higher—compared with the true INR, and at four centres mean results exceeded the 10% limit.9 The limits of agreement (fig 1) ranged from −1.24 to 0.87 units.

Table 2

Comparison of INR displayed by system B with “true” INR at the 10 centres

View this table:

Relation to intensity of anticoagulation

Although the Bland-Altman plots (fig 1) show greater deviation with higher INRs, the percentage difference from the true INR is not simply related to the INR (tables 1 and 2).10 Tables 1 and 2 show the variability of individual differences between displayed INRs and true INRs within centres with the two monitor systems.

Differences between true INRs

The overall mean difference between true INR within the range 1.5 to 4.5 was relatively small (2.8%) with the two different thromboplastin standards; the mean INR was 2.52 with human and 2.59 with rabbit.

Using the WHO thromboplastin standards from different species with the manual prothrombin time test results in a well established minor INR difference.11 Our results showed that only a small contribution to the differences in INR with the two point of care test systems comes from the different routes of international sensitivity index calibration (human and rabbit immunoreactive plasma) used in the two monitor systems.

Aberrant results

With system A (table 1), monitors at eight of the 10 centres gave at least one aberrant result (more than 50% deviation from the true INR), with a total incidence of 28 (5.2%). A single instrument at one centre accounted for 10 of these. With system B (table 2), 12 (2.2%) results were aberrant. In all, four of the aberrant results, from different samples, differed by more than 2.0 units (fig 1).

Discussion

The two whole blood point of care test systems gave INRs which differed by 21.3%—a considerable clinical discrepancy. The systems were tested on the 536 patients treated with warfarin remaining in the analysis after exclusions at 10 centres.9

Also of concern is the considerable disagreement between the overall displayed INR with system A and the true INR and this occurred to a lesser extent with system B. Even withsystem B, however, the percentage difference in mean INR at four of the centres exceededWHO's 10% limit for clinical relevance.9 With systemA, displayed results of INR from nine of the 10 centres were greater than the true INR, but the results of system B showed the opposite trend with most mean displayed INRs less than the true INRs.

Another problem is the inconsistency of variations between centres between the mean displayed INR and the true INR. On the same samples of blood, the centre which gave the greatest percentage difference from mean true INR with system A (34.6%), gave one of the smallest percentage differences from true INR (2.3%) with system B.

Although Bland-Altman plots (fig 1) show a trend to greater deviation from the true INR above 3.0, the variation in percentage differences from true INR at the 10 centres was not due simply to the degree of coagulation defect (tables 1 and 2).

Point of care test prothrombin time testing is being increasingly introduced to meet the growing demand for warfarin in the United Kingdom and worldwide. The two monitoring systems that we studied are the most widely used in the European Union. One of these systems is becoming widely adopted with the general trend to increased community management of dosage of warfarin because these monitors avoid the need for hospital attendance and this system may have an application in self dosage.12

All 20 instruments of the two types used in this study were assembled at one centre before the start and tested by the same operator on the same set of whole blood samples. Variation in displayed INRs between instruments of the same type was observed, but it was not as great as found in this report.13 This study, however, reflects not only the instrumental error but also the added variation introduced by 10 different operators.

The clinical relevance of these discrepancies to dosage of warfarin is important (fig 2). The effect on dosage of warfarin may be that with system A less warfarin is prescribed than with system B to achieve target INRs. This might result in a tendency to increased bleeding with system B or alternatively less protection from thrombosis with system A as it is necessary to maintain patients within target INR intervals to minimise bleeding and further thrombosis.4

Fig 2
Fig 2

Differences in mean INR plotted against “true” INR; (a) monitoring system A, (b) monitoring system B

Only a few of the discrepancies between displayed INRs and the true INRs on the two types of monitor can be attributed to the different species for international sensitivity index calibration with rabbit or human thromboplastin standards: the difference between the mean true INRs found using the two thromboplastin WHO thromboplastin standards of rabbit and human origin with the traditional manual prothrombin time test was small (2.6%). The two manual prothrombin time test INRs also agreed far better overall despite the fact that they also incorporated the inherent error of using different species of thromboplastin.

We also found aberrant displayed results—that is, results of INR exceeding 50% deviation from the true INR—on a small number of samples with both monitor systems. This previously unrecognised problem may be important, as only a single test would normally be performed and users are not likely to be aware of an aberrant result. One of the monitor systems states that if an unexpected result occurs with a test, it should be repeated, but this is unsatisfactory as it places the onus on the user.

Point of care test prothrombin time whole blood monitors are convenient and simple, and claims have been made that they are more reliable than laboratories performing conventional prothrombin time testing.1418 Only two randomised cross over studies of such point of care test prothrombin time systems have been reported, but none of the clinical studies compared the results displayed by the monitor with true INR on the same blood samples tested with the WHO thromboplastin standard and the manual prothrombin time technique.19 20

Van den Besselaar previously reported, in a single monitor study, a statistically but not clinically significant difference in mean INR from reference values with a WHO thromboplastin standard using the manual prothrombin time technique with the CoaguChek Mini system.21

What is already known on this topic

Whole blood point of care test prothrombin time monitors are convenient and simple andclaims have been made that they are more reliable than laboratories doing conventional international normalised ratio (INR) monitoring

What this study adds

The INRs obtained using manual tests with two thromboplastin standards gave better agreement than the INRs from the monitoring systems despite the thromboplastins coming from different species

Manufacturers need a more practical way of calibrating the international sensitivity index of their systems but better methods of quality control are needed to check performance of individual monitors and operators

The manufacturers of the two systems make considerable efforts to ensure the reliability and safety of their monitors. Nevertheless the results indicate that additional steps in calibrating the international sensitivity index and quality control are essential to ensure the reliability of displayed INR of these point of care test prothrombin time whole blood monitor systems. Several other types of point of care test monitors for prothrombin time testing are currently marketed, and they may share similar problems.

As users cannot adjust the INR displayed by the monitors, calibration of the international sensitivity index of a monitor to derive INRs has to be the responsibility of the manufacturer. Because of the large numbers of monitors in use and the complexity of the recommended procedure, calibration of international sensitivity index for all individual instruments would not be possible.22 Furthermore, to be reliable, such calibrations need to be on a multicentre basis. A minimum of three centres is required to calibrate the TAS PT-NC and five for the CoaguChek Mini.23

Manufacturers of monitors therefore need a less complex and demanding procedure for international sensitivity index calibration. We have developed such a system using lyophilised plasmas certified by European Certified Action on Anticoagulation, which has been validated in a multicentre exercise and needs to be used with each of the two point of care test prothrombin time monitor systems studied in this report.5 24 25 Nevertheless, even with the simplified procedure, calibrating all instruments will still not be feasible, and since calibration does not check performance of operators, quality control of individual monitors and their users is also necessary. Recommendations for such quality control have been made by the European Concerted Action on Anticoagulation as existing national or regional systems of external quality control could not be expected to cope with the massive numbers of point of care test monitors in use.26 The European Concerted Action on Anticoagulation has developed and validated a simplified system of calibrating the international sensitivity index for each of the two monitor systems and has also proposed a method for the quality control of individual monitors.

Acknowledgments

Editorial by Murray and Greaves

We thank J Conard, D Dias, N Egberg, J A Iriarte, I Kontopoulou-Griva, and B Otridge. We also thank Roche Diagnostics and Bayer AG for the loan of the CoaguChek and TAS instruments and the donation of test strips and cards and WHO Biologicals for the supply of RBT/90 and rTF/95 thromboplastin standards. We also thank G Anthi, M Clerici, H Fitzgerald, MH Horellou, J Meeuwisse-Braun, EM Norberg, L Söderblom, K Overgaard, and M Vacas Rius for scientific help. Contributors: LP, AMHPvdB, AT, CS, and JJ are members of the European Concerted Action on Anticoagulation Steering Group. LP organised and coordinated the study and drafted the report with MK. AMHPvdB, AT, MK, CS, and JJ were participants in the study, assisted in the planning and design of protocols, training of other participants and in the preparation of the final report. NC performed statistical planning and analysis and assisted in preparation of the final report. LP is guarantor.

The study was supported by European Community Standards Measurements and Testing Programme (Grant No SMT4-CT98-2269) and an additional grant from Manchester Thrombosis Research Foundation, a registered charity.

Footnotes

  • Competing interests None declared.

References

View Abstract