QRISK or Framingham for predicting cardiovascular risk?BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b2673 (Published 07 July 2009) Cite this as: BMJ 2009;339:b2673
- Rod Jackson, professor of epidemiology,
- Roger Marshall, associate professor of biostatistics,
- Andrew Kerr, cardiologist and clinical senior lecturer,
- Tania Riddell, senior research fellow,
- Sue Wells, senior lecturer in clinical epidemiology and quality improvement
In the linked study (doi: 10.1136/bmj.b2584), Collins and Altman assess the performance of the QRISK cardiovascular risk prediction algorithm in a primary care setting in the United Kingdom,1 and compare QRISK2 3 with equivalent Framingham algorithms.4 5
The QRISK algorithm is based on the largest risk prediction study ever undertaken and highlights a potential use of large scale electronic health record systems.2 3 6 In just a few years, a small team has linked electronic health records from several million people to produce a cardiovascular risk prediction algorithm that is more accurate and better validated than previous ones. Although prediction algorithms are available for many conditions, most are based on small numbers, are poorly validated, infrequently updated, and not generalisable. Moreover, most prediction algorithms are weak predictors and are not used regularly.
The first QRISK prediction algorithm was generated by retrospectively extracting data on risk factors and subsequent cardiovascular events for almost two million people from the QRESEARCH primary care database of more than 10 million patients covering about 7% of the population of the United Kingdom.2 It was validated by the developers in another large database,3 and a year later they published an updated and improved algorithm, QRISK2, which included several additional predictors.6
Collins and Altman1 now provide an independent evaluation of the first (QRISK1) algorithm and compare its performance with three Framingham algorithms.4 5 They conclude that on every performance measure, QRISK1 is better than Framingham. Unfortunately, because of the timing of publications, they were unable to compare QRISK2 with a recently modified version of the Framingham algorithm recommended by the National Institute for Health and Clinical Excellence (NICE) in 2008. This modified Framingham algorithm includes adjustments for family history and ethnic origin.7 Although not an independent evaluation, Hippisley-Cox and colleagues have now compared their QRISK2 algorithm with the NICE modified Framingham algorithm and again QRISK performs better.6
Direct comparisons between QRISK and Framingham are perhaps a little unfair because Framingham algorithms have not been calibrated to the UK population, although this is a relatively easy mathematical adjustment.8 However, an algorithm’s ability to discriminate between patients who will have an event and those who will not cannot be so easily improved and this is where QRISK has a slight edge on Framingham. More importantly, because QRISK2 performs better than QRISK1, further improvements are likely in future iterations.
But a closer look at the Collins and Altman evaluation provides a sobering message about the current state of cardiovascular risk prediction.1 Our figure⇓ uses scaled rectangles to re-present some of their data, and it illustrates more clearly the modest discrimination performance of both algorithms at recommended treatment thresholds.9 QRISK would classify one in 10 men in the UK as high risk—that is, having a 10 year cardiovascular risk above the threshold recommended by NICE for treatment with statins.7 However only 30% of the subsequent cardiovascular events in men occurred in this high risk group. In contrast, the Framingham algorithm would classify about twice as many men in the UK (one in five) as being at high risk, although this larger high risk group does not include twice as many of the men who had a cardiovascular event during follow-up (it included only 50%). Substantially fewer women were identified as high risk (about 4% by QRISK and 5% by Framingham), with surprisingly little overlap between the two high risk groups. These high risk groups included only 18% (QRISK) and 17% (Framingham) of women who subsequently had a cardiovascular event.
Almost 80% of participants in QRISK had some missing risk prediction variables, which suggests that QRISK could be improved given more complete data. Furthermore, it indicates that most UK adults have not had a formal documented cardiovascular risk assessment, as recommended by NICE,7 and that the quality of cardiovascular risk management in the UK (as elsewhere) is suboptimal.
Although UK general practices using the EMIS electronic health record system will have free access to an integrated QRISK calculator, commercial restrictions on the use of the algorithm in other systems are a concern. Cost may become a barrier to the development of effective electronic decision support using QRISK algorithms. Our experience has taught us that developing and implementing a computerised cardiovascular risk assessment and decision support system is a highly specialised task. Three features are crucial to their success: automatic provision of decision support as part of clinician workflow; provision of recommendations rather than just assessments; and provision of support at the time and location of decision making.10 We have shown that decision support incorporating these features significantly increases cardiovascular risk assessment,11 but substantial time, experimentation, and wide collaboration are needed.
A QRISK based algorithm should replace the currently recommended Framingham based algorithm for estimating cardiovascular risk in the UK. With increased use, the quality of data will improve and updated prediction algorithms should be more accurate. However, QRISK is just the first of many continuously updatable prediction algorithms that will become available worldwide as electronic health record systems replace current paper based systems. The planned UK General Practitioner Extraction Service, for example, should soon be capturing data relevant to risk prediction from most of the population.12 We believe that freely sharing these algorithms is the best way to facilitate their effective implementation.
Cite this as: BMJ 2009;339:b2673
Competing interests: None declared.
Provenance and peer review: Commissioned; not externally peer reviewed.