Prediction models for cardiovascular disease risk in the general population: systematic review
BMJ 2016; 353 doi: https://doi.org/10.1136/bmj.i2416 (Published 16 May 2016) Cite this as: BMJ 2016;353:i2416
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Damen and colleagues (1) present an interesting overview, with recommendations for future practice, of prediction models for cardiovascular disease (CVD) risk. Unfortunately, although they make much of so-called “validation” of these scores in research articles, they say nothing about its relevance to current clinical practice.
In predictive modeling, as reviewed by these authors, historical data are used to predict a known future. Each model produces a risk score on a scale of 0-1 (or 0-100%), but each outcome is either an event or a non-event, so the score cannot ever be “valid”, except in an extreme situation. As in many of the papers reviewed, typically the basic definition of “validation” is stretched to include directional agreement of observed relative frequencies and predicted risks across ordinal categories of predicted risk, but there remains an underlying problem that, when any score is used in clinical practice, we are predicting an unknown future. How can we possibly validate this?
To move to practical issues, I would argue that almost any risk score is invalid, in the sense implied by Deman et al, even when applied contemporarily in the same population from which the data used in the score were derived. A prime reason for this is that the historical data, used for any particular score, were derived in a period when “background” risks were much higher than now: at least 10 years ago if 10-year risks were derived. Hence the score inevitably has built-in obsolescence in that it tends to over-estimate risk. Alternatively, it may be that a score under-estimates within its own parent population, even at a time concurrent with data capture, due to the healthy cohort effect. In theory, these problems may be overridden by exhaustive inclusion of prognostic variables, but this is unlikely to be possible in CVD. So, when considering future clinical practice, external validation is only useful for testing discrimination (avoiding bias from self-testing) and internal validation is only useful for testing calibration (to ensure that the model used fits adequately within its own data).
Denman et al mention GLOBORISK (2) in passing, dismissing this new approach to CVD risk scoring as not adding anything new. I disagree. This is an innovative, pragmatic approach to risk prediction which aims to produce a CVD risk score that is fit to (clinical) purpose, using contemporary data to address the problems of built-in obsolescence and the healthy cohort effect. In addition, it provides a general method to predict CVD risk globally. Importantly GLOBORISK satisfies two important criteria for a new approach to risk scoring, similar to recommendations made by the authors: making better use of available evidence (than have previous risk models) and tailoring (to current needs).
References
1. Damen JA, Hooft L, Schuit E, Debray TP, Collins GS, Tzoulaki I, Lassale CM, Siontis GC, Chiocchia V, Roberts C, Schlüssel MM, Gerry S, Black JA, Heus P, van der Schouw YT, Peelen LM, Moons KG. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016 May 16;353:i2416. doi: 10.1136/bmj.i2416.
2. Hajifathalian K, Ueda P, Lu Y, Woodward M, Ahmadvand A, Aguilar-Salinas CA, Azizi F, Cifkova R, Di Cesare M, Eriksen L, Farzadfar F, Ikeda N, Khalili D, Khang YH, Lanska V, León-Muñoz L, Magliano D, Msyamboza KP, Oh K, Rodríguez-Artalejo F, Rojas-Martinez R, Shaw JE, Stevens GA, Tolstrup J, Zhou B, Salomon JA, Ezzati M, Danaei G. A novel risk score to predict cardiovascular disease risk in national populations (Globorisk): a pooled analysis of prospective cohorts and health examination surveys. Lancet Diabetes Endocrinol. 2015 May;3(5):339-55. doi: 10.1016/S2213-8587(15)00081-9.
Competing interests: I co-designed the GLOBORISK cardiovascular risk score. I am a consultant to Amgen.
Prediction models for cardiovascular risk. Focus on high risk patients since the first consultation
Johanna A A G Damen et al examine the different model for prediction of cardiovascular risk and show they are useful but require external validation to make them of more practical value.
We have found that two simple concepts, the Pulse Mass Index and the Pulse Mass Pressure Product, can be very helpful to evaluate the risk of cardiometabolic diseases, and potentially, the relative risk of long term mortality right from the beginning, when we examine a patient in the first consultation.
The Pulse Mass Index (Resting Heart Rate or RHR multiplied by BMI and divided by 1730), has a high correlation with the Framingham Risk Score as we published at the end of the last century (Lancet. Volume 353, Number 9156. 13 March 1999). If a patient has a Pulse Mass Index over 1.3 (eg BMI 30 by RHR 75 equals 2250 divided by 1730 is 1.3), this patient has a high probability to be at high cardiovascular risk. To simplify, if the simple product of RHR by BMI is 2250 or over, there should be a complete evaluation of the cardiovascular risk. The normal reference product of 1730 represents a round of BMI 24 multiplied by RHR 72.
The RHR reflects the metabolic rate and should be done at least after five minutes rest and after two hours fasting, without influence of smoking or drugs. The RHR increases postprandial, and also in proportion to the level of abdominal and apple type obesity.
Interestingly, we found that if there is a relation of three to one between pulse and BMI (eg, 72 to 24) and this theoretical relation was maintained proportional as BMI increases, then the enlarged mortality becomes predictable—eg, for a BMI of 33 and a theoretically corresponding pulse of 99 (1/3), the pulse×mass index (33×99•1730) is 1•9 or almost twofold, corresponding with the known doubling of mortality with this BMI. The same tendency is found for every increase of BMI and pulse.
We could confirm this trend in all groups in the study of Body Mass Index and Mortality among 1.46 Million White Adults by Amy Berrington de Gonzalez et. al. (N Engl J Med 2010; 363:2211-2219), and also in the study of Hazard Ratios for All-Cause Mortality According to BMI at Survey vs Maximum Lifetime BMI (Proc Natl Acad Sci USA, online January 4, 2016) using data from the 1988–2010 National Health and Nutrition Examination Surveys (NHANES) that were linked to death records through 2011.
Besides the Pulse Mass Index as a first approach, we can also evaluate the Pulse Mass Pressure Product in the first consultation. The normal values of this product of systolic blood pressure or SBP by RHR by BMI is round 200,000 (115x72x24). If this product is 300,000 or over (50 percent elevation), the is a high probability of abnormal fasting blood sugar or GlycoHbA1c, that means, these patients have a high probability of Diabetes or Pre-diabetes, or metabolic syndrome, according to our long time observations.
I strongly recommend to evaluate both the Pulse Mass Index and the Pulse Mass Pressure Product in every patient right in the first consultation, to have a first impression of their cardiovascular and metabolic risk, and also an approximate impression of the relative risk of death at the long term.
And focus on these patients for the long term prevention of cardiovascular diseases.
Prof. Enrique Sánchez Delgado, MD
Internal Medicine-Clinical Pharmacology and Therapeutics
Hospital Metropolitano Vivian Pellas, Managua
Competing interests: No competing interests
Re: On validation of cardiovascular risk scores
We highly appreciate the comment of Dr. Woodward on our systematic review on prediction models for cardiovascular disease (CVD) risk in the general population (1). Dr. Woodward addresses the development and validation of a relatively new prediction model in this field, the so-called GLOBORISK model. At first sight this is just another development of a model to predict CVD, that can be added to the list of 363 prediction models already described in our systematic review. As already stated in a previous comment (2), the GLOBORISK model has indeed some features that make this model an interesting candidate for further research. GLOBORISK is during its development directly stratified by sex and cohort (from various countries across the globe) and can therefore more easy be tailored to different but similar countries as those included in the development set. This tailoring is even possible using only routinely available country specific data on overall CVD risk and on population averages of the predictors. The authors clearly describe how this updating should be done, and demonstrate this for 11 countries from various continents. As discussed in our review, to our knowledge yet no prediction models are developed from data specifically documented from African or South American countries, which makes GLOBORISK an interesting candidate for this gap.
GLOBORISK builds on the assumption that associations between predictor effects and cardiovascular disease are similar across Western and Asian populations (3). This is a unique assumption which, to our knowledge, has never been used in previous models. There is conflicting evidence though for this assumption (4-8). Gijsberts and colleagues describe for example that the association between total cholesterol levels and CVD risk differs between race/ethnicity groups (4).
External validation studies, in our view, are therefore still necessary to investigate whether this assumption truly holds, and to study the transportability of the model to populations other than the cohorts used for the development of GLOBORISK. Furthermore, head-to-head comparisons using individual participant data, are in our view still warranted to compare GLOBORISK to other existing and frequently advocated and validated CVD risk prediction models, such as the Pooled Cohort Equations (9), Framingham prediction models (10 11), and SCORE (12).
References
1. Damen JA, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 2016;353:i2416.
2. Moons KG, Schuit E. Prediction of cardiovascular disease worldwide. The lancet Diabetes & endocrinology 2015;3(5):309-10.
3. Hajifathalian K, Ueda P, Lu Y, et al. A novel risk score to predict cardiovascular disease risk in national populations (Globorisk): a pooled analysis of prospective cohorts and health examination surveys. The lancet Diabetes & endocrinology 2015;3(5):339-55.
4. Gijsberts CM, Groenewegen KA, Hoefer IE, et al. Race/Ethnic Differences in the Associations of the Framingham Risk Factors with Carotid IMT and Cardiovascular Events. PLoS ONE 2015;10(7):e0132321.
5. Singh GM, Danaei G, Farzadfar F, et al. The age-specific quantitative effects of metabolic risk factors on cardiovascular diseases and diabetes: a pooled analysis. PLoS ONE 2013;8(7):e65174.
6. Woodward M, Huxley H, Lam TH, et al. A comparison of the associations between risk factors and cardiovascular disease in Asia and Australasia. Eur J Cardiovasc Prev Rehabil 2005;12(5):484-91.
7. Yusuf S, Hawken S, Ounpuu S, et al. Effect of potentially modifiable risk factors associated with myocardial infarction in 52 countries (the INTERHEART study): case-control study. Lancet 2004;364(9438):937-52.
8. Hurley LP, Dickinson LM, Estacio RO, et al. Prediction of cardiovascular death in racial/ethnic minorities using Framingham risk factors. Circulation Cardiovascular quality and outcomes 2010;3(2):181-7.
9. Goff DC, Jr., Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;129(25 Suppl 2):S49-73.
10. D'Agostino RB, Sr., Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117(6):743-53.
11. Wilson PW, D'Agostino RB, Levy D, et al. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97(18):1837-47.
12. Conroy RM, Pyorala K, Fitzgerald AP, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 2003;24(11):987-1003.
Competing interests: No competing interests