Lessons from biases in electronic health record data: the importance of clinical vigilance with negative test results
We congratulate Agniel et al1[1] on their paper showing that the timing and repetition of testing in a hospital acute care environment is frequently more predictive of the outcome than the actual test results. This is a textbook illustration of selection bias, in that ill patients are more likely to receive testing. We have been studying the same phenomenon in primary care, primarily in the context of cancer diagnosis, also using large databases of electronic health care records. The mere fact a test has been conducted, irrespective of the actual result, predicts cancer. This additional risk is only partly eliminated by a negative test result, leaving the negative test group still at a higher risk than those untested. For example, male primary care patients with a normal platelet count (<400 x109/l) have a cancer risk of 4.1% in the next year;[2] the risk is 5.1% for those with platelet count in the ‘high-normal’ range of 375-399 x109/l,[3] suggesting – like Agniel et al describe - that the test result also adds some information to the selection element. These risks are above the 3% NICE threshold for urgent referral for cancer, and well above the 1% threshold that patients would choose.[4] Similarly, we have found a normal primary care haemoglobin result to be associated with colorectal cancer (odds ratio 1.5; p=0.001),[5] a normal chest x-ray to be associated with lung cancer (odds ratio 6.9; P<0.001),[5] and we are currently seeing a similar pattern with normal inflammatory markers in primary care.
Agniel et al emphasise the importance of this for researchers – it also has important clinical implications. Diagnosis in primary care can be challenging; many early symptoms of cancer are non-specific and low risk. Clinicians may use ‘routine’ blood tests in such patients for reassurance, assuming negative tests represent the absence of disease. Clinicians may weigh the results of an ‘objective’ test more highly than one’s own ‘subjective’ clinical judgement; yet these findings demonstrate that clinicians’ judgement to perform a test, based on experience, intuition, history and examination, probably has a higher predictive value than the test result. This phenomenon demonstrates the need for clinical vigilance with negative test results, and has implications for how clinicians safety net and explain negative test results to patients.
References
1. Agniel D, Kohane IS, Weber GM. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ 2018;361 doi: 10.1136/bmj.k1479
2. Bailey SE, Ukoumunne OC, Shephard EA, et al. Clinical relevance of thrombocytosis in primary care: a prospective cohort study of cancer incidence using English electronic medical records and cancer registry data. Br J Gen Pract 2017 doi: 10.3399/bjgp17X691109
3. Ankus E, Price SJ, Ukoumunne OC, et al. Cancer incidence in patients with a high normal platelet count: a cohort study using primary care data. Family Practice 2018:cmy018-cmy18. doi: 10.1093/fampra/cmy018
4. Banks J, Hollinghurst S, Bigwood L, et al. Preferences for cancer investigation: a vignette-based study of primary-care attendees. The Lancet Oncology 2014;15(2):232-40. doi: 10.1016/s1470-2045(13)70588-6
5. Hamilton W. Earlier diagnosis of colorectal, lung and prostate cancer (thesis). Bristol 2005
Competing interests:
No competing interests
11 May 2018
Jessica C Watson
NIHR Doctoral Research Fellow
Sarah Bailey, Fergus Hamilton, Willie Hamilton, Luke Mounce
Rapid Response:
Lessons from biases in electronic health record data: the importance of clinical vigilance with negative test results
We congratulate Agniel et al1[1] on their paper showing that the timing and repetition of testing in a hospital acute care environment is frequently more predictive of the outcome than the actual test results. This is a textbook illustration of selection bias, in that ill patients are more likely to receive testing. We have been studying the same phenomenon in primary care, primarily in the context of cancer diagnosis, also using large databases of electronic health care records. The mere fact a test has been conducted, irrespective of the actual result, predicts cancer. This additional risk is only partly eliminated by a negative test result, leaving the negative test group still at a higher risk than those untested. For example, male primary care patients with a normal platelet count (<400 x109/l) have a cancer risk of 4.1% in the next year;[2] the risk is 5.1% for those with platelet count in the ‘high-normal’ range of 375-399 x109/l,[3] suggesting – like Agniel et al describe - that the test result also adds some information to the selection element. These risks are above the 3% NICE threshold for urgent referral for cancer, and well above the 1% threshold that patients would choose.[4] Similarly, we have found a normal primary care haemoglobin result to be associated with colorectal cancer (odds ratio 1.5; p=0.001),[5] a normal chest x-ray to be associated with lung cancer (odds ratio 6.9; P<0.001),[5] and we are currently seeing a similar pattern with normal inflammatory markers in primary care.
Agniel et al emphasise the importance of this for researchers – it also has important clinical implications. Diagnosis in primary care can be challenging; many early symptoms of cancer are non-specific and low risk. Clinicians may use ‘routine’ blood tests in such patients for reassurance, assuming negative tests represent the absence of disease. Clinicians may weigh the results of an ‘objective’ test more highly than one’s own ‘subjective’ clinical judgement; yet these findings demonstrate that clinicians’ judgement to perform a test, based on experience, intuition, history and examination, probably has a higher predictive value than the test result. This phenomenon demonstrates the need for clinical vigilance with negative test results, and has implications for how clinicians safety net and explain negative test results to patients.
References
1. Agniel D, Kohane IS, Weber GM. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ 2018;361 doi: 10.1136/bmj.k1479
2. Bailey SE, Ukoumunne OC, Shephard EA, et al. Clinical relevance of thrombocytosis in primary care: a prospective cohort study of cancer incidence using English electronic medical records and cancer registry data. Br J Gen Pract 2017 doi: 10.3399/bjgp17X691109
3. Ankus E, Price SJ, Ukoumunne OC, et al. Cancer incidence in patients with a high normal platelet count: a cohort study using primary care data. Family Practice 2018:cmy018-cmy18. doi: 10.1093/fampra/cmy018
4. Banks J, Hollinghurst S, Bigwood L, et al. Preferences for cancer investigation: a vignette-based study of primary-care attendees. The Lancet Oncology 2014;15(2):232-40. doi: 10.1016/s1470-2045(13)70588-6
5. Hamilton W. Earlier diagnosis of colorectal, lung and prostate cancer (thesis). Bristol 2005
Competing interests: No competing interests