Intended for healthcare professionals

Rapid response to:

Practice Practice Pointer

Your results may vary: the imprecision of medical measurements

BMJ 2020; 368 doi: https://doi.org/10.1136/bmj.m149 (Published 20 February 2020) Cite this as: BMJ 2020;368:m149

Rapid Response:

Using reference change values (RCV) to assess changes in analyte concentrations – not as easy as it looks

McCormack and Holmes have developed an application to make it easier for medical practitioners and patients to assess changes in analyte concentrations when monitoring the same individual over time, an initiative we applaud. This is achieved by providing reference change values (RCV) which are based on estimates of biological variation data and analytical variation, assuming that pre-analytical variation is negligible. The biological variation data used as basis for this application are mostly taken from the EFLM Biological Variation Database (1), which delivers real-time updated biological variation data for numerous analytes, resulting from systematic reviews and appraisal of published biological variation studies by the Biological Variation Data Critical Appraisal Checklist (BIVAC) (2). However, we observe for some analytes inconsistencies between the estimates reported by McCormack and Holmes and those published in the EFLM Biological Variation Database, as exemplified for HDL cholesterol, at 7.5% vs 5.7%, respectively.

Furthermore, there are two additional caveats that must be taken into account when using this application. Firstly, an inappropriate formulae for RCV is applied. Estimates of analytical and biological variation can be quantified as standard variations (SD) or as coefficient of variation (CV). The concept of the RCV was introduced by Harris and Yasaka (3), calculated as: RCV(SD)=Z*√(2)*√(SDA^2 * SDI^2). This formulae is only applicable to SDs, i.e. changes in units, with a presupposition that the expected change between samples follows a normal distribution. However, the difference in percent between normally distributed variables is not a normally distributed variable itself (4). The percent difference between measurements M1 and M2 is defined as (M2-M1)/M1=M2/M1 -1, and if M1 and M2 are normally distributed, M2/M1 is not. Thus, when using analytical (CVA) and within-subject biological variation (CVI) estimates quantified as CVs, the following RCV(CV) formulae should be applied:
SDA^2 = ln(CVA^2 +1)
SDI^2 = ln(CVI^2 +1)
RCV(CV)=100%*(exp(±Z*√2*√(SDA^2 +SDI^2) )-1)

To exemplify, using triglycerides for which McCormack and Holme in their application report:
RCV(SD)=±100%*1.64*√(2)*√(0.025^2 * 0.205 ^2)=±47.9%,
based on the biological and analytical variation estimates of 20.5% and 2.5%, respectively, at a 95% confidence level. If applying these in a RCVCV formulae as is appropriate, we get the following:
SDA^2 = ln(CVA^2 +1)=ln(0.025^2 +1)=0.0006248048
SDI^2 = ln(CVI^2 +1)=ln(0.205^2 +1)=0.04116594
RCV(CV)=100%*(exp(±1.64*√2*√(0.0006248048 +0.04116594))-1) = (-37.8%, 60.7%)

Thus, when using these RCVs to monitor a patient, there is clearly a consequence of applying incorrectly or correctly calculated RCVs. The consequence of using CV-based estimates in a RCV(SD) formulae is the greatest when analytical or biological estimates are high, ie. over 10% if based on rough estimates.

Secondly, the calculator as a standard approach applies a two-sided Z-score. It is often assumed that a Z-score of 1.96 for P <0.05 (and sometimes also 2.58 for P <0.01) is appropriate for any clinical scenario. However, these Z-scores are bidirectional (or two-tailed or two-sided), and this infers that the difference between the two serial results can be either an increase or a decrease. However, in many clinical situations, the decision-making is the assessment of a significant fall (for example, reduction of HbA1c after treatment for diabetes mellitus), or a significant rise (for example, an increase in serum troponin after acute chest pain). Thus, unidirectional (one-tailed or one-sided) Z-scores must be used in such clinical situations to facilitate correct interpretation; these are 1.64 for P <0.05 and 2.33 for P <0.01. Correct definitions of the clinical decision-making context and the major differences between the term “change” and “rise or fall,” and their synonyms, are required for correct calculation of appropriate RCV. In addition, clinical decisions are often taken using probabilities less than 95% and the chosen Z scores therefore must reflect this.

It is also important to note that RCV only address how likely it is that a certain change can be explained by analytical and biological variation, but not the probability that a real change in the patient´s health state has occurred. It has been suggested that a better tool for understanding and interpretation of measured differences in monitoring is needed and that the concepts of sensitivity, specificity, likelihood ratios, and odds used for diagnostic test evaluations should also be applied to monitoring by substituting measured concentrations with measured differences (5).

References
1. Aarsand AK, Fernandez-Calle P, Webster C, Coskun A, Gonzales-Lao E, Diaz-Garzon J, Jonker N, Minchinela J, Simon M, Braga F, Perich C, Boned B, Roraas T, Marques-Garcia F, Carobene A, Aslan B, Barlett WA, Sandberg S. The EFLM Biological Variation Database. https://biologicalvariation.eu/. Accessed March 2020.
2. Aarsand AK, Røraas T, Fernandez-Calle P, Ricos C, Díaz-Garzón J, Jonker N, Perich C, González-Lao E, Carobene A, Minchinela J, Coşkun A, Simón M, Álvarez V, Bartlett WA, Fernández-Fernández P, Boned B, Braga F, Corte Z, Aslan B, Sandberg S. The Biological Variation Data Critical Appraisal Checklist: A Standard for Evaluating Studies on Biological Variation. Clin Chem 2018; 64:501-514
3. Harris EK, Yasaka T. On the calculation of a “reference change” for comparing two consecutive measurements. Clinical Chemistry. 1983;29:25–30.
4. Fokkema MR, Herrmann Z, Muskiet F a J, Moecks J. Reference change values for brain natriuretic peptides revisited. Clinical Chemistry. 2006;52:1602–3.
5. Petersen PH, Sandberg S, Iglesias N, et al. ‘Likelihood-ratio’ and ‘odds’ applied to monitoring of patients as a supplement to ‘reference change value’ (RCV). Clin Chem Lab Med 2008;46:157-164.

Competing interests: No competing interests

23 March 2020
Thomas Røraas
Mathematician
Sverre Sandberg, Aasne K. Aarsand
Norwegian Organisation for Quality Improvement of Laboratory Examinations (NOKLUS)
Haraldsplass Deaconess Hospital, Bergen, Norway