Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2

Competing interests: JHC is director of QResearch (database used for developing QRISK2) and ClinRisk (which makes open and closed source software available to help the reliable implementation of QRISK)

Editor, Hippisley Cox et al(1), and Christiaens (2), have published important empirical data and conceptual reasoning, respectively, in relation to cardiovascular risk prediction. Hippisley Cox have strengthened the creaky foundations of risk prediction in relation to ethnic minority groups, but their vision is that the challenges are mathematical. By contrast, Christiaens questions the emphasis on absolute risk reduction, which favours older people, and suggests using relative risks within each age group-an approach which is, arguably, ageist. Very recently, the BMJ published Cooper and O’Flynn’s summary of NICE Guidance on risk assessment of cardiovascular disease, which places considerable emphasis on combining mathematically predicted risk and clinical judgement(3).

Before turning to the specifics of the new published papers, I would like to emphasise some basic principles, building on my recent rapid response to Cooper and O'Flynn (http://www.bmj.com/cgi/eletters/336/7655/1246#197036). Risk prediction tools are developed in populations and the outcome is a probability between 0% and 100%. As I have discussed recently, epidemiological data apply extremely well to populations, although some refinement is required when extrapolating from one population to another.(4) Disease occurs in individuals and is either 0% or 100%. Population data are not directly applicable to individuals, so it is not surprising that risk prediction is poor at the individual level. Conceptually, there is a mismatch between the clinical need of individual patients, and the available data. The sciences of individuals-genetics, toxicology, pathology, etc-need to realise the promise of personalised medicine so we can predict on individuals based upon their own data. Clinical medicine has become over- reliant upon epidemiology.

This fundamental limitation is a strong argument for clinical judgement, and individual clinical testing, following the screening test that risk prediction actually provides. Unfortunately, the screening test has become the definitive test-the tail is wagging the dog. Doctors have to decide whether the population derived risk is pertinent to the individual. The South Asian Health Foundation (http://www.sahf.org.uk) convened a conference in 2006 to review risk assessment modelling. The conference concluded that modelling is a screening tool requiring considerable clinical judgement before applying results to individuals, and this particularly applied to South Asians. . Even more clinical judgement is required in regard to ethnic minority and other populations where cardiovascular cohort studies are sparse(5).

The NICE guidance suggests the estimated risk for South Asian men is multiplied by 1.4. Hippisley Cox et al provide new data and new methods that improves on this pragmatic approach, but still need improvement. Contrary, to the Hippisley Cox et al's claim, theirs is not what we generally understand to be a prospective cohort study, and possibly it would be better described as a retrospective cohort. One of the problems these designs face is that the researchers are not in control of the measurement of exposure variables at baseline. One of the key variables is ethnicity.

Terminology for ethnic groups has been changing rapidly in the UK since 1993, when entry into this cohort started. The importance attached to collecting ethnicity data in the NHS has also shifted from apathy in 1993 to moderate interest now. The proportion of people with valid ethnic group codes in Hippisley Cox et al's study is, unsurprisingly, small. Population selection biases are almost inevitable. Firstly, that the Read codes used for ethnicity were only introduced around about 2002. Prior to that, a variety of Read codes were (and still are) used, comprising of a mixture of racial and ethnicity terms, that were not in accord with UK census categories (http://www.pcel.info/ethnicity/). Presumably, ethnic status was described using the Read codes given in the paper. If so, ethnic minority groups would be recent entrants to the cohort (around about 2002 and onwards), while others were entered throughout the period of the study. In diseases where the incidence and case fatality is changing rapidly, as for cardiovascular diseases, this will create period (cohort) effects. Secondly, repeat attenders at the practice are more likely to have an ethnic code, particularly if they are also being entered into a chronic disease register, either because they have the disease, a relevant comorbidity (such as diabetes or impaired glucose tolerance), or relevant risk factors. So, higher risk people are more likely to have ethnic codes. Since people without a specific ethnic group codes were placed into the White category, the potential bias is self-evident.

The authors acknowledge ethnic group missclassification, but conclude that this would reduce the size of apparent ethnic group differentials. We cannot be complacent about missclassification. It is very difficult to demonstrate that missclassification is nondifferential, and this is unlikely as illustrated above. Even if it is, missclassification in confounding variables can produce substantial spurious associations reflected in surprisingly high relative risks. The diabetes prevalence in the White reference group is very low indeed, and we would expect 3 or 4%, and higher if some of these people are non-White minorities. This is one potential indicator of the impact of period effects and missclassification.

Several researchers have been applying for funding for multi-ethnic cardiovascular cohort studies for many years, including myself. Fortunately, a very large cardiovascular cohort study has now been assembled, in Southall-it is called Lollipop, and is publishing cross- sectional data already-Professor Jaspal Kooner is the principal investigator.(6) We can foresee prospective analyses in a few years time.

The authors have proposed that their database be linked to the 2001 census to extract ethnic codes (and possibly other information). This approach has been implemented in Scotland.(7) In Fischbacher et al's study of myocardial infarction in South Asians compared to non-South Asians there was a marked variation in incidence-about 60% excess in men, and about 80% in women. These excesses are commensurate with those reported by Hippisley Cox et al.

The idea that South Asians have an excess of 40% CVD is, of course, simplistic. First, this figure only applies to CHD, not stroke, where the excess of mortality is about 100%. Second, mortality data, most recently examined by Wild et al(8) show excesses almost exactly in line with those of Hippisley Cox et al. In Bhopal et al's predictions the Framingham equations were fairly good for CHD but very poor for stroke.(9) In developing and testing prediction tools it might be better to work with each of these outcomes.

How do we apply these kind of findings in clinical practice? We have recently reported on a clinic and community-based cardiovascular disease risk factor control programme in Scotland.– Khush Dil.(10) To our knowledge, no other cardiovascular prevention project has reported outcomes in South Asians. The project doubled the calculated Framington risk prediction. Why so? This was guided by 1991 England and Wales mortality data, showing that the SMR for CHD in men was 142 in Indians, 148 in Pakistani and 151 in Bangladeshi and even higher for stroke e.g. 281 for Bangladeshi men.(11) Risk prediction tools, however, did not show such excesses. This decision was vindicated by the even higher inequalities in 2001 mortality data (8) and data from Fischbacher et al's linkage study.(7) The multifaceted intervention of Khush Dil was association with beneficial change in behaviour, biochemistry and intermediate clinical outcomes. In future work of this kind I would advocate QRISK2, and I would modify the predicted risk for each ethnic group. I would, however, recommend prediction to be a component of a detailed clinical assessment that tailored the intervention to the individual.

These recent observations call for rigorous trials. First, trials need to demonstrate the cost-effectiveness of cardiovascular screening compared with clinical care without this screening instrument and/or with other instruments. Second, trials need to show the effectiveness of risk factor change programmes, especially in S. Asians, including the practical value of risk prediction tools. Third, the costs and benefits of the absolute approach compared to the relative risk approach advocated by Christiaens et al(2) need to be compared.

Reference List

(1) Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008; 336(7659):1475- 1482.

(2) Christiaens T. Cardiovascular risk tables. BMJ 2008; 336(7659):1445-1446.

(3) Cooper A, O'Flynn N. Risk assessment and lipid modification for primary and secondary prevention of cardiovascular disease: summary of NICE guidance. BMJ 2008; 336(7655):1246-1248.

(4) Bhopal R. Causes in epidemiology: the jewels in the public health crown. J Public Health 2008;http://jpubhealth.oxfordjournals.org:80/cgi/content/full/fdn052?ijkey=7f3AYAC9bFNbzu2&keytype=ref, fdn052.

(5) Ranganathan M, Bhopal R. Exclusion and Inclusion of Nonwhite Ethnic Minority Groups in 72 North American and European Cardiovascular Cohort Studies. PLoS Med 2006; Vol 3(3):0001-0008, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1324792.

(6) Kooner JS, Chambers JC, guilar-Salinas CA, Hinds DA, Hyde CL, Warnes GR et al. Genome-wide scan identifies variation in MLXIPL associated with plasma triglycerides. Nat Genet 2008; 40(2):149-151.

(7) Fischbacher CM, Bhopal R, Povey C, Steiner M, Chalmers J, Mueller G et al. Record linked retrospective cohort study of 4.6 million people exploring ethnic variations in disease: myocardial infarction in South Asians. BMC Public Health 2007; 7(1):142.

(8) Wild SH, Fischbacher C, Brock A, Griffiths C, Bhopal R. Mortality from all causes and circulatory disease by country of birth in England and Wales 2001-2003. J Public Health (Oxf) 2007; 29(2):191-198.

(9) Bhopal R, Fischbacher C, Vartiainen E, Unwin N, White M, Alberti G. Predicted and observed cardiovascular disease in South Asians: application of FINRISK, Framingham and SCORE models to Newcastle Heart Project data. J Public Health 2005; 27:93-100.

(10) Mathews G, Alexander J, Rahemtulla T, Bhopal R. Impact of a cardiovascular risk control project for South Asians (Khush Dil) on motivation, behaviour, obesity, blood pressure and lipids. J Public Health 2007; 29(4):388-397.

(11) Gill PS, Kai J, Bhopal R.S, Wild S. Health Care Needs Assessment: Black and Minority Ethnic Groups. The epidemiologically based needs assessment reviews. In: Stevens A et al, editor. Abingdon: Radcliffe Medical Press Ltd, 2007.

Competing interests: None declared

Congratulations to Professor Hippisley-Cox and her team for breaking the shackles of Framingham. My own calculations suggest there is still more than fine tuning to be done though, which in turn raises some other basic issues.

Using discriminant analysis is an efficient way to predict a dichotomous criterion,without requiring transformations. I used the Nottingham teams predictor variables with a sample from New Zealand to produce the results below:

True Negatives 65.7%, True Positives 66% (a noticeable improvement on Framingham's true positives of 54.5% although its true negatives were 75.9%) but the classification/misclassification ratios yielded true to false negative predictions of 73.6 to 1 and true to false positive predictions of 1 to 19.6. Note the change of direction. Both Framingham and Nottingham predict who will not have a CVD incident with only a small error but predict who will have a CVD incident with a lot of error.

There are other variables that can be used to produce similar results. I included diabetes as a variable and Maori because of a known ethnic factor here in NZ and then did a stepwise discriminant calculation. In the presence of these new variables both cholesterols were eliminated as predictors as were blood pressure and smoking but it yielded a true positive prediction of 65.4% with a misclassification rate of 1 to 19.2. No real difference from Nottingham. One has to recognise the canonical correlation here is only 0.141 and because of the size of the sample almost any difference from zero is significant but only 2% of the total variance is accounted for(R squared or 1 - Wilks Lambda = 0.02. Putting it another way as there were only two categories to be predicted (have or have not had CVD incident)the baseline probability is not zero but is 0.5 with a true positive misclassification rate of 1 to 19.2!. Thus the advantage of using Framingham, Nottingham or Discriminant Function is only around 15% above chance and with a misclassification rate similar to chance. Furthermore when investigators use ROC curves to confirm their findings they seem to be unaware that the predictive figure given is a weighted average of the success with both true negatives and true positives, this is alright if the numbers or percentages are the same for both groups. But when the incidence of CVD incidents in the population is so relatively small the ROC results often don't represent the results for true positives at all.

Now the elimination of cholesterols, blood pressure and smoking as predictors by the introduction of diabetes and an ethnic factor suggests that what small correlation there is results from the processes of an illness as a specfic factor and something more holistic associated with ethnic differences rather than being specifically tied to cholesterol etc.. And this is only suggestive. What all three approaches show is that we really dont have an explanation here for the cause of cardiovascular events. At best what we use are some pre-conditions and fairly loose ones at that.

Given the weakness of the evidence it seems uneccessary waste, discomfort for some of our patients and perhaps even danger for others, to base a national prevention scheme on such a flimsy structure.

I am most grateful to Professor Rod Jackson and his team for giving me access to their Predict database. The analysis package was SPSS. Author's email: marshrw@hotmail.com .

Competing interests: None declared

1. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A, Brindle P. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ. 2008 Jun 28;336(7659):1475-82. Medline 18573856
2. http://www.health-heart.org/why.htm [a homocysteine based hypothesis as to cause]
3. Rhein HM. Vitamin D deficiency is widespread in Scotland. BMJ. 2008 Jun 28;336(7659):1451. Medline 18583649
4. Giovannucci E, Liu Y, Hollis BW, Rimm EB. 25-hydroxyvitamin D and risk of myocardial infarction in men: a prospective study. Arch Intern Med. 2008 Jun 9;168(11):1174-80. Medline 18541825
5. Obeid OA, Mannan N, Perry G, Iles RA, Boucher BJ. Homocysteine and folate in healthy east London Bangladeshis. Lancet. 1998 Dec 5;352(9143):1829-30. Medline 9851391

Competing interests: None declared