QRISK - authors response
Re: Derivation and validation of QRISK, a new cardiovascular disease
risk score for the United Kingdom: prospective open cohort study.
Thank you for giving us the opportunity to respond to letters on
QRISK. We have tried to address all the substantive points in full and
also present additional analyses as requested.
1.1 Additional analyses on statin usage and cholesterol
Milne, Lewis, Simpson and Cooney all raise concerns about the
inclusion of patients on statins in our analysis and ask whether this
might have biased our results. Peto, Cooney and Tunstall-Pedoe all
highlight the low hazard ratio for cholesterol ratio. Several people also
contacted us individually to request additional analyses to investigate
this in more detail. We found the discussions helpful and have now
conducted those analyses.
Of the 1,283,174 patients in the original derivation cohort, only
14,005 patients (1.09%) had already been prescribed statins at baseline.
After removing these patients from the cohort, there were a further
118,009 (9.3%) patients who were prescribed statins during the ten year
study period. However the vast majority of these patients started
treatment towards the end of the observation period. For example, of the
total 8,182,404 person years of observation, only 256,387 person years
(3.13%) were exposed to statin treatment. This means that more than 96% of
our study observation time was uncontaminated by statin usage and so our
population was largely statin naïve.
Table 1 shows the adjusted hazard ratios for model A (the additional
models to appear in the BMJ print version or from julia.hippisley-
• Model A includes all patients apart from those prescribed statins
at baseline. Patients who began statins during follow up are included.
This is our preferred model.
• Model B is the same as model A except it adjusts for statin usage
as a time varying covariate. The issue with this approach is how such a
model with time varying covariates could be applied to a real prospective
• Model C is the same as model A except the outcome is coronary heart
• Model D is the same as model A except that the outcome is
Table: Comparison of hazard ratios for cardiovascular disease using four different models. All models exclude statin users at baseline.
We have improved our implementation of multiple imputation for
missing data. Our revised model for imputation includes more variables in
addition to those originally included. We have added binary variables for
diagnosis of hypertension and incident diabetes and continuous variables
for the number of prescriptions for aspirin, statins and antihypertensive
treatments for each patient during the study period. When the revised
imputed datasets were used in the main analysis, the estimated hazard
ratios for the cholesterol ratio term in males and females increased to
more biologically plausible values given our combined outcome of
cardiovascular disease. We think this addresses the concerns raised by
Peto, Cooney and Tunstall-Pedoe and improves the analysis. Multiple
imputation has rarely been used in derivation of risk scores and has a
distinct advantage in terms of using a larger and more representative
sample than the analysis of only those subjects who had complete records
of all data including serum cholesterol. The adjusted hazard ratios for
the ratio of cholesterol/HDL in the complete case analysis was 1.20 (95%
CI 1.17 to 1.22) in females and 1.25 (95% CI 1.23 to 1.27) in males. The
analysis using only subjects with complete records is available from the
authors on request.
There was little difference between models for any of the hazard
ratios for cholesterol/HDL ratio for males or females which suggests that
use of statins has not significantly biased our hazard ratios. Models D
shows the hazard ratios for coronary heart disease as an outcome and
demonstrates that these are broadly similar to those obtained in other
studies for prediction of coronary heart disease mortality1 including
Framingham. The calibration statistics and the predictions of those at
risk by age, sex and deprivation were very similar to those in our
original analysis (results available from the authors). Overall, the
exposure to statin was low (3% of the person years) and the results from
each of the different models are very similar suggesting that this is not
an important source of bias in our analysis. Model A is our preferred
final model which will be used in our risk calculator http://www.qrisk.org
In their own research MacLeod et al. found that the addition of
material deprivation didn’t improve discrimination of Framingham and
consider the ROC curve areas for QRISK and Framingham are similar.
Romanens and Ferns make the same point. However, the deficiencies of the
ROC statistic are well known2 which is why we presented the D statistic
which is more appropriate for use with survival data3. This does
demonstrate a significant improvement for QRISK compared with the other
The performance of a risk prediction equation will always be specific
to contexts of time, place and means and definitions of data collection
and outcomes. Our aim is to provide a means to equitably implement a
cardiovascular disease prevention programme in the context of contemporary
UK practice, rather than to elucidate the aetiology of cardiovascular
disease. QRISK gives us a means for deriving, implementing and if
necessary updating an ‘in-house’ equation based on general practitioners
contemporary data which is at least as good if not better than Framingham
in it’s discrimination and better calibrated to identify major
cardiovascular events in the United Kingdom. It will also address problems
with Framingham which under- estimates risk in people from deprived
backgrounds4 5, those on anti-hypertensive treatment and people with
positive family histories.
1.3 Equity and over-prediction
Lewis has misunderstood – we modelled Townsend scores as a continuous
variable in the risk score and only presented the results by fifth for
ease of interpretation. We used area based deprivation scores derived at
output area (125 households) since there is no generally accepted and
reliably recorded individual measure of deprivation.
MacLeod et al. consider that over-prediction by Framingham is not a
problem because the inclusion of more people is likely to result in more
benefit than harm. The bar for intervention has been fixed nationally at a
20% 10 year cardiovascular disease risk by the statin Health Technology
Assessment http://guidance.nice.org.uk/TA94#documents . This threshold
effectively sets the additional national resource (workload, advice,
treatment) that will be allocated to people at increased risk. The issue
becomes how best to allocate this resource on an equitable basis taking
account of health needs – in this case indicated by cardiovascular risk –
which is determined by age, sex, blood pressure and social deprivation
amongst other things. We agree all new technologies run the risk of
exacerbating relative inequalities, but this is not a reason for not
seeking to reduce this by our social interventions.
We disagree with Macleod et al who consider that simply including
more people, including socially deprived men, will reduce inequality.
Equity is a relative not an absolute issue. In the most deprived compared
to the least deprived fifth of social deprivation, there is a three-fold
increase in the proportion of women identified at high risk and a 30%
excess among men. In the most affluent fifth, QRISK indicates that
Framingham overestimates risk by a factor of 2. These are substantial and
important differences particularly for women and they will be to some
extent ameliorated by a prevention programme driven by QRISK but are
likely to be further exacerbated if Framingham is retained. QRISK ensures
a more equitable allocation of the national resource than Framingham.
QRISK prioritises the more socially deprived, particularly women, older
ages and people with on antihypertensive treatment more equitably than
1.4 Home advantage
We agree with Macleod et al that QRISK has a home advantage but one
which is likely to be shared by other EMIS practices which together cover
60% of the practices in England and Wales. If QRISK performs as well in a
similar database from another major computer system (THIN) it will
demonstrate generalisability to most of the national population. MacLeod
et al suggest that THIN is also home territory, it is, but only in the
sense that primary care is the very territory that a cardiovascular
disease risk score is meant to be used in.
We included ASSIGN in our analysis as its use of social deprivation
in risk estimation led us to consider whether it was suitable for use in
England and Wales. The reason why ASSIGN may overestimate risk in England
and Wales is because the Scottish ASSIGN cohort represents a higher risk
population. It had a mean serum cholesterol of 6.2mmols/l which is
higher than the mean of 5.4mmols/l in the Health Survey for England and
Wales (1998) and 5.5mmols/l in the Health Survey for Scotland (1998) and
5.7mmols/l in the QResearch dataset. Similarly the proportion of smokers
was much higher in ASSIGN (42.4% of men) compared to 34% and 30%
respectively in the Health Surveys for England and Scotland and 28% for
QResearch. ASSIGN may well be representative of the Scottish Heart Health
Extended Cohort population – a cohort recruited in the 1980’s because
Scotland had particularly high cardiovascular mortality rates. However it
may not represent the whole population of modern-day Scotland since most
of the participants came from urban areas of North Glasgow and Edinburgh,
with baseline measurements collected between 1986-2001 compared with 1995
to 2007 in our study.
We thank Professors Richard Peto and Patrick Royston for advice on
additional analyses presented here and their interpretation.
1. Diverse Populations Collaborative Group. Prediction of mortality
from coronary heart disease among diverse populations: is there a common
predictive function? Heart 2002;88:222-228.
2. Cook N. Use and Misuse of the Receiver Operating Characteristic Curve
in Risk Prediction. Circulation 2007;115:928-935.
3. Royston P, Sauerbrei W. A new measure of prognostic separation in
survival data. Statistics in Medicine 2004;23:723-748.
4. Brindle P, McConnachie A, Upton M, Hart C, Davey-Smith G, Watt G. The
accuracy of the Framingham risk-score in different socioeconomic groups: a
prospective study. British Journal of General Practice 2005;55(520):838-
5. Tunstall-Pedoe HW, M for the SIGN group on risk estimation. By
neglecting deprivation, cardiovascular risk scoring will exacerbate social
gradients in disease. Heart 2006;92:307-310.
The authors of the original paper
Competing interests: No competing interests