Your results may vary: the imprecision of medical measurements

One of the major problems with our imprecise diagnostic tests is the assumption that disease is either present or absent using man-made standards and interpretation of results.

Diagnostic testing that requires the interpretation of whether disease is present or absent, reflects a fundamental problem with the yes/no testing approach used to determine if someone has cancer, heart disease or some other problem (sensitivity) or doesn’t (specificity); and just as importantly whether our treatment for the patient is working or not [1].

For too long physicians and insurance companies have looked at people as either having a disease or not. Medicine is after all the practice of healthcare, not disease care. The entire approach leads to problems with coverage of pre-existing conditions, as well as the consequence of dealing with failures of that approach, when those told they do not have a health problem, later die from it [2,3].

Qualitative and semi-quantitative methods of looking for disease, excludes the understanding that changes in the health of a patient occurs across a continuum – a “Health-Spectrum” - and that patients do not suddenly wake up one day with cancer, heart disease, or some other health problem [4].

The ability to quantitatively measure changes in the Health-Spectrum of a patient is more accurate, consistent and reproducible; and it allows us to measure actual treatment results without guessing [5,6] - thereby decreasing healthcare costs and time while saving lives.

These measurements are only possible when the calibration of the equipment we use is made against a known standard, which does not flux – but is constant [5,6]. Only by using such constants, will these measurements be accurate, consistent and reproducible – across the Health-Spectrum of the patient – eliminating the “results may vary” error introduced using man-made standards and interpretation [6].

References:

1. Fleming RM, Fleming MR, Chaudhuri TK. Are we prescribing the right diets and drugs for CAD, T2D, Cancer and Obesity? Int J Nuclear Med Radioactive Subs 2019;2(2):000115.
2. https://www.areyoudense.org/
3. https://www.youtube.com/watch?v=RTHEtRtiB3k
4. Fleming RM, Fleming MR. The Importance of Thinking about and Quantifying Disease like Cancer and Heart Disease on a “Health-Spectrum” Continuum. J Compr Cancer Rep 2019;3(1):1-3 (Article ID 100011).
5. Fleming RM, Fleming MR, Chaudhuri TK, McKusick A. FMTVDM Quantitative Imaging Replaces Current Qualitative Imaging for Coronary Artery Disease and Cancer, Increasing Diagnostic Accuracy and Providing Patient- Specific, Patient-Directed Treatment. Cardio Open 2019;4(3).
6. Fleming RM, Fleming MR, Dooley WC, Chaudhuri TK. Invited Editorial. The Importance of Differentiating Between Qualitative, Semi-Quantitative and Quantitative Imaging – Close Only Counts in Horseshoes. Eur J Nucl Med Mol Imaging. DOI:10.1007/s00259-019-04668-y. Published online 17 January 2020 https://link.springer.com/article/10.1007/s00259-019-04668-y

Dear Editor

Measurement variation is profoundly important in clinical medicine and its implications largely overlooked.

A number of us in in University of Birmingham have an interest in measurement variation and its clinical implications - which include misdiagnosis and incorrect inferences about the effects of treatment during monitoring leading to incorrect clinical decisions. There are some situations where chance variation in measured parameters may be the main influence on clinical decisions.

In response to the observation that there is little medical student teaching on measurement variation I would like to point out that I do teach Graduate Entry Students in University of Birmingham about measurement variation. I use a simple exercise with a hypothetical patient to illustrate the effects of variation. This can easily be reproduced and I would be happy to share the teaching materials.

In view of the papers cited in this article can give the impression that the “reference change value” (RCV) is a new concept. However, the statistical model that supports it was introduced in the 70s of last century by E. K. Harris and collaborators [1] and it is noteworthy that, more than 40 years later, such an interesting parameter has not yet moved into the routine interpretation of laboratory results.
Although practical difficulties have been adduced, that there are, to present this information properly to the clinician, the reason could really be in the lack of confidence in the robustness of this parameter.
To enter this information in the laboratory report, the laboratory professional must start by selecting the value for the biologic variation (CVi) that best matches their patient population among those published, which sometimes differ substantially, or assume that the median of all of them or the value reported in a meta-analysis represents them.
Next it should be assumed that this CVi is the same for all patients, which is generally not true. This means that patients with intra-individual variability lower than the value set for CVi will have a high risk that a significant change will go unnoticed, while patients with greater variability will have a high risk of detecting changes that have not taken place.
Finally, it should be assumed that the test results compared are independent. Although this may be considered true for many magnitudes if the time elapsed between determinations is sufficiently long, the results tend to be correlated when sampling is frequent as occurs with hospital patients. The existence of correlation implies that the probability of detecting a significant change will depend on the time elapsed between determinations.
Taking these conditions into account, the laboratory must assess whether the estimated RCV for a test is sufficiently reliable and can be safely applied in the patients investigated before flagging a result as different from the previous one in the laboratory report. The concept of the value of the reference change is attractive and can even be used as an instructive diversion but it can pose a risk to patients if it is not used rigorously.

Reference
1. Harris EK, Brown SS. Temporal changes in the concentrations of serum constituents in healthy men: Distributions of within-person variances and their relevance to the interpretation of differences between successive measurements. Annals of Clinical Biochemistry, 1979:16, 169–76

Dear Editor,

As a senior medical student, your article made me reflect on the lack of undergraduate teaching around understanding and communicating biological variability in relation to medical test measurements.

It strikes me that medical schools could have a key role in preparing newly qualified doctors to more objectively interpret test results. As McCormack and Holmes say, biological variability cannot be fixed, but an understanding of its effect can help us make informed decisions. By incorporating a taught session into clinical training, whole cohorts of newly qualified doctors would have a familiar framework to help judge how meaningful certain test results may be.

Junior doctors may also be rightly placed to help spread awareness of the issue within medical teams. Junior doctors are the most frequent people to order and follow up patient tests in the hospital setting, supported by seniors (Ericksson et al., 2018). The surrounding discussions within teams may be an ideal opportunity to raise the issue of test reference change values. It might help to rationalise the significance of certain results and more critically consider whether further testing is always required. Given current pressures on laboratory overuse, this would surely be a welcome change (Zhi et al., 2013).

I think targeting such teaching to medical school communication skills sessions could be effective. In my centre we rehearse and receive feedback on simulated patient interactions over a variety of set scenarios. There are existing initiatives to prepare medical students to communicate with ‘e-patients’ (patients who have access to online information) by adding modules or new scenarios to existing curricula (Herrmann-Werner et al., 2019). It would be feasible to extend a task and practice discussing uncertainty or insignificance in test results.

By adequately preparing medical students to interpret and communicate uncertainty in results, we can expect better care as they progress to be junior doctors: more confident test interpretation, more appropriate test ordering and more effective patient interactions.

References
Ericksson, W., Bothe, J., Cheung, H., Zhang, K., and Kelly, S. (2018). Factors leading to overutilisation of hospital pathology testing: the junior doctor’s perspective. Aust. Health Rev. 42, 374.
Herrmann-Werner, A., Weber, H., Loda, T., Keifenheim, K.E., Erschens, R., Mölbert, S.C., Nikendei, C., Zipfel, S., and Masters, K. (2019). “But Dr Google said…” – Training medical students how to communicate with E-patients. Med. Teach. 41, 1434–1440.
Zhi, M., Ding, E.L., Theisen-Toupal, J., Whelan, J., and Arnaout, R. (2013). The Landscape of Inappropriate Laboratory Testing: A 15-Year Meta-Analysis. PLoS ONE 8, e78962.