Predicting risk of upper gastrointestinal bleed and intracranial bleed with anticoagulants: cohort study to derive and validate the QBleed scores
BMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g4606 (Published 28 July 2014) Cite this as: BMJ 2014;349:g4606
- Julia Hippisley-Cox, professor of clinical epidemiology and general practice,
- Carol Coupland, associate professor and reader in medical statistics
- Correspondence to: J Hippisley-Cox Julia.hippisley-cox{at}nottingham.ac.uk
- Accepted 1 July 2014
Abstract
Objective To develop and validate risk algorithms (QBleed) for estimating the absolute risk of upper gastrointestinal and intracranial bleed for patients with and without anticoagulation aged 21-99 years in primary care.
Design Open cohort study using routinely collected data from general practice linked to hospital episode statistics data and mortality data during the five year study period between 1 January 2008 and 1 October 2013.
Setting 565 general practices in England contributing to the national QResearch database to develop the algorithm and 188 different QResearch practices to validate the algorithm. All 753 general practices had data linked to hospital episode statistics and mortality data at individual patient level.
Endpoint Gastrointestinal bleed and intracranial bleed recorded on either the linked mortality data or the linked hospital records.
Participants We studied 4.4 million patients in the derivation cohort with 16.4 million person years of follow-up. During follow-up, 21 641 patients had an incident upper gastrointestinal bleed and 9040 had an intracranial bleed. For the validation cohort, we identified 1.4 million patients contributing over 4.9 million person years of follow-up. During follow-up, 6600 patients had an incident gastrointestinal bleed and 2820 had an intracranial bleed. We excluded patients without a valid Townsend score for deprivation and those prescribed anticoagulants in the 180 days before study entry.
Risk factors Candidate variables recorded on the general practice computer system before entry to the cohort, including personal variables (age, sex, Townsend deprivation score, ethnicity), lifestyle variables (smoking, alcohol intake), chronic diseases, prescribed drugs, clinical values (body mass index, systolic blood pressure), and laboratory test results (haemoglobin, platelets). We also included previous bleed recorded before entry to the study.
Results The final QBleed algorithms incorporated 21 variables. When applied to the validation cohort, the algorithms in women explained 40% of the variation for upper gastrointestinal bleed and 58% for intracranial bleed. The corresponding D statistics were 1.67 and 2.42. The receiver operating curve statistic values were 0.77 and 0.86. The sensitivity values for the top 10th of men and women at highest risk were 38% and 51%, respectively. There were similar results for men.
Conclusion The QBleed algorithms provided valid measures of absolute risk of gastrointestinal and intracranial bleed in patients with and without anticoagulation as shown by the performance of the algorithms in a separate validation cohort. Further research is needed to evaluate the clinical outcomes and the cost effectiveness of using these algorithms in primary care.
Introduction
Anticoagulants are used in the prevention and treatment of venous thromboembolism. They are also used to reduce risk of ischaemic stroke,1 especially among patients with atrial fibrillation. The use of anticoagulants is likely to increase in future, especially since guidelines from the UK National Institute for Health and Care Excellence encourage more systematic identification of patients at high risk of venous thromboembolism or stroke who might benefit from anticoagulation. For example, in 2010 NICE issued new guidance to improve the prevention of venous thromboembolism for patients, using cost effective interventions.2 In January 2014 NICE issued draft guidance on the management of atrial fibrillation, which included assessment of the risks of stroke as well as the risks and benefits associated with anticoagulation.3
New tools now exist to quantify the absolute risk of thrombosis4 as well as those at risk of ischaemic stroke in primary care, including patients with atrial fibrillation.5 Although the risk of stroke in patients with atrial fibrillation can be reduced by anticoagulation,1 many patients with atrial fibrillation are not currently prescribed anticoagulation even though it is incentivised in the general practice Quality and Outcomes Framework.6 This may reflect concerns about monitoring, or uncertainties about the potential balance of risks and benefits for an individual patient, including the potential adverse haemorrhagic effects of traditional anticoagulants such as warfarin. Novel oral anticoagulants (factor Xa inhibitors and direct thrombin inhibitors) exist that have the advantage of not requiring regular international normalisation ratio (INR) blood test monitoring.7 Data on safety and efficacy of the novel anticoagulants is still accumulating. However, a recent meta-analysis of three randomised controlled trials8 reported that the new anticoagulants are more effective at reducing all cause stroke and systemic embolism (relative risk 0.78, 95% confidence interval 0.67 to 0.92) in people with atrial fibrillation compared with warfarin. However, data on risks of major bleeding (0.88, 0.71 to 1.09) were inconclusive,8 with a suggestion of an increased risk of gastrointestinal bleeding (1.25, 0.91 to 1.72) compared with warfarin.8 Quantifying the absolute risk of bleed for an individual receiving anticoagulation treatment is important because it can help clinicians identify patients for whom the benefit of anticoagulation outweighs the risk of bleed, help inform the discussion with patients about the risks and benefits of anticoagulation, and allow clinicians to monitor more carefully patients at high risk to help reduce their risk. For example, this could be achieved by modifying known risk factors for bleed, including avoidance of the use of concurrent drugs known to increase the risk of bleed, such as non-steroidal anti-inflammatory drugs (NSAIDs) and antidepressants.9
Several clinical risk prediction rules have been developed to assess the risk of bleed. Among the first were the HEMOR2RHAGES,10 ATRIA scores,11 and RIETE and OBRI scores.12 More recently the HAS-BLED score13 has been developed and is recommended in the European Society of Cardiology guidelines.14 HAS-BLED has several advantages compared with previous scores as it does not require information unlikely to be available in clinical practice, such as genetic information. It has also has out-performed previous scores in identifying clinically relevant bleeding.15 The populations used to develop HAS-BLED, as with HEMOR2RHAGES,10 are unlikely to be representative of patients in UK primary care, having been recruited through an American insurance scheme10 or through secondary care as part of a clinical trial.11 13 The studies are based on small cohorts of patients with low numbers of events. The resulting scores are implemented using simple counting systems, do not include some important risk factors for bleed, and do not give an absolute risk of bleed. Although including more risk factors within a risk score generally improves risk stratification, it also makes tools such as HAS-BLED increasingly cumbersome to use in everyday clinical practice. Limitations of the ATRIA scores have been highlighted,16 including use of a cohort that had already received anticoagulants, exclusion of patients with risk factors, and counterintuitive results, including an apparent protective effect of previous bleed and concurrent use of antiplatelet treatment.16
We developed and validated a new set of risk prediction algorithms to predict risk of gastrointestinal and intracranial bleed in new users of anticoagulants (rather than existing users). We wanted to develop algorithms that quantify absolute risk of bleed, which can be communicated to patients to aid decision making in the consultation, automatically populated from the patients record and integrated into general practice computer systems, and updated readily over time, as the pattern of prescribing of both anticoagulant drug classes may change.
Methods
Study design and data source
We conducted a cohort study of a large primary care population using version 38 of the QResearch database (www.qresearch.org). QResearch is a large validated primary care electronic database containing the health records of 14 million patients registered from 753 general practices using the Egton Medical Information System (EMIS) computer system.17 Practices and patients contained on the database are nationally representative18 and similar to those on other primary care databases using other clinical software systems.19 We included all QResearch practices in England once they had been using their current EMIS system for at least a year (to ensure completeness of recording of morbidity and prescribing data), randomly allocating three quarters of practices to the derivation dataset and one quarter to the validation dataset.
Overall cohort selection
We identified an open cohort of patients aged 21-99 years at the study entry date, drawn from patients registered with eligible practices between 1 January 2008 and 1 October 2013. We used an open, rather than closed, cohort design, as this allows patients to enter the population throughout the whole study period rather than require registration on 1 January 2008, thus better reflecting the realities of routine general practice. We excluded patients without a valid postcode related Townsend deprivation score, and existing users of anticoagulants by excluding those prescribed anticoagulants in the 180 days before their study entry date. To help minimise potential biases we used an incident user design.20
For each patient we determined an initial entry date to the cohort, which was the latest of the following dates: 21st birthday, date of registration with the practice plus one year, date on which the practice computer system was installed plus one year, and the beginning of the study period (1 January 2008). For patients who had started an anticoagulant after their initial entry date, we assigned an index entry date that was the date of their first prescription of the anticoagulant during the study period.
We censored patients at the earliest date of the first upper gastrointestinal haemorrhage or intracranial haemorrhage, death, deregistration with the practice, last upload of computerised data, three months after their last prescription for anticoagulants (if applicable), five years after their study entry date, or the study end date (1 October 2013). For intermittent users of anticoagulants, if there was a gap of more than three months between the end of one prescription and the start of the next, we censored them after three months from the start of the gap even if the patient had future prescriptions. Patients who switched anticoagulants during the study period were censored one month after their last prescription before the switch.
Outcomes
Our two primary outcomes were incident gastrointestinal bleed and incident intracranial bleed occurring within five years of study entry. We used ICD-10 (international classification of diseases, 10th revision) codes to identify events that were recorded on either the linked hospital record or the mortality record. Supplementary table 1 lists the ICD-10 codes used for upper gastrointestinal bleeding and supplementary table 2 the codes used for intracranial bleed. Upper gastrointestinal bleed was defined as any upper gastrointestinal ulcer with perforation, bleed, or both; melena; haematemesis; laceration with bleed; varices with bleed; haemorrhagic gastritis; and other unspecified gastrointestinal bleeds. Intracranial bleed was defined as subarachnoid haemorrhage, intracerebral haemorrhage, subdural or extradural haemorrhage, and other unspecified intracranial bleeds.
The QResearch database is linked at individual patient level to both the hospital admissions data and the Office for National Statistics mortality records using a project specific pseudonymised National Health Service number. The recording of NHS numbers is valid and complete for 99.8% of patients with records on QResearch, 99.9% for Office for National Statistics mortality records, and 98.6% for hospital admissions records.21 22 We defined patients as having the outcome of interest if there was a record of the relevant ICD-10 code in either their hospital record or their linked mortality record. For example, for a patient to be classified as having been admitted to hospital due to bleeding, the ICD-10 code would be recorded as the primary diagnosis in the hospital admissions data. Similarly for a patient to have been considered to have died due to bleeding, the relevant ICD-10 code would be recorded as either the primary or the underlying cause of death in the Office for National Statistics mortality record. We did not include events that were recorded on the general practice record but not on either the hospital or mortality record as we wanted to ensure the events were likely to represent severe bleeding.
We separately quantified the risk of upper gastrointestinal bleed and intracranial bleed, as has been done in relevant clinical trials.8 These two events are potentially life threatening or life changing and may be preventable. We initially considered having a more general outcome of “any haemorrhagic event” but decided against this given the disparate nature of the events, the rarity with which some occur, the lack of relevant ICD-10 codes for some outcomes, and the difficulty of applying definitions requiring clinical adjudication to routinely collected data.
Exposure
The primary exposure of interest was use of anticoagulants, including warfarin, phenindione, dabigatran, rivaroxaban, or apixaban. We identified new users of anticoagulants during the study period by identifying those with at least one prescription issued after the date of initial entry to the study. We excluded patients who had been prescribed one or more prescriptions in the past 180 days before their study entry date. The remaining patients were classified as non-users of anticoagulants.
Predictor variables for consideration
Using similar definitions where relevant we compiled a list of risk factors (box) based on the September 2013 edition of the British National Formulary, literature, and existing risk prediction scores.
For clinical values, we used the most recently recorded before entry to the cohort (box). For new users of anticoagulants, we used the values recorded before they started anticoagulants. We identified medical conditions based on the patient’s primary care record, using standard definitions of Read codes where available. We considered the conditions to be present if they were entered in the patients’ record before entry to the cohort.
We identified new users of anticoagulants and users of other drugs thought to increase the risk of bleeding or that interact with anticoagulants (box). To qualify for being exposed to the relevant drug at study entry, participants had to have at least two prescriptions, with the most recent one within the preceding 28 days before entry to the cohort.
Risk factors associated with risk of bleeding
Personal and life style
Age (years) at study entry23
Self assigned ethnicity: white or not recorded, Indian, Pakistani, Bangladeshi, other Asian, Black African, Black Caribbean, Chinese, other (including mixed)
Smoking status: non-smoker, former smoker, light smoker (<10 cigarettes/day), moderate smoker (10-19 cigarettes/day), heavy smoker (≥20 cigarettes/day)
Alcohol intake (units/day): none, <1, 1 or 2, 3-6, 7-9, >913
Clinical values (most recent before study entry)
Body mass index
Systolic blood pressure
Haemoglobin <110 g/L
High or low platelet count (<150/μL or >480/μL)
Medical conditions thought to increase the risk of bleed
Treated hypertension13: (recorded diagnosis of hypertension and a current prescription for at least one antihypertensive)
Atrial fibrillation or flutter (including paroxysmal atrial fibrillation)
Chronic renal disease13: Read code indicating glomerular filtration rate <60 mL/min or recorded diagnosis of major renal disease, including renal transplant or dialysis
Chronic liver disease or chronic pancreatitis13: including cirrhosis, liver failure, liver fibrosis, portal hypertension, chronic hepatitis, autoimmune hepatitis, and chronic pancreatitis
Type 1 or type 2 diabetes (including diagnosis of diabetes or treatment with oral hypoglycaemic agents or insulin)
Ischaemic stroke or transient ischaemic attack13
Congestive cardiac failure (including left ventricular failure, right heart failure, pulmonary oedema)
Venous thromboembolism (including pulmonary embolism or deep vein thrombosis)
Cancer (any diagnosis of cancer, excluding non-melanoma skin cancer)
Peptic ulcer (including gastric, oesophageal, or duodenal ulceration)
Oesophageal varices or arteriovenous malformation
Previous bleed recorded in the general practice record (including intracranial, upper gastrointestinal, haematuria, or haemoptysis)
Drugs thought to increase bleeding risk or interact with anticoagulants
Derivation of the models
We used published methods to derive and validate models that could be applied to new users of anticoagulants.25 26 As in previous similar studies, we decided a priori to derive separate models in men and women since the predictors and their weightings are likely to be different. We therefore derived four models in total—gastrointestinal bleed and intracranial bleed for men and a corresponding set for women. We used multiple imputation to replace missing values for body mass index, systolic blood pressure, smoking, and alcohol intake, and used these values in our main analyses.27 28 29 30 We carried out five imputations. We used Cox’s proportional hazards models to estimate the coefficients for each risk factor for men and women separately, using Rubin’s rules to combine the results across the imputed datasets. We used fractional polynomials to model non-linear risk relations with continuous variables.31 We used backward selection procedures to eliminate variables, using a P value of 0.01 to obtain parsimonious models. We focused on predictor variables that have a clinical face validity and are included in similar scores and those that have a hazard ratio of <0.85 or >1.15 (for categorical exposures), selecting those that met these criteria for inclusion in the final model.
We examined interactions between predictor variables and age. We also tested for interactions between anticoagulants and drugs thought to increase the risk of a bleed, such as antiplatelets and other drugs listed previously. We included significant interaction terms in the final models where these were plausible.
We took the regression coefficients for each variable from the final models and used these as weights, which we combined with the baseline survivor function for each outcome evaluated at each of 1-5 years to derive absolute risk equations at each years’ follow-up.
Validation of the models
We applied each of the models obtained from the derivation cohort to the validation cohort and calculated predicted risks for each patient and measures of discrimination (D statistic32 and area under the receiver operating characteristic curve statistic) and calibration (comparing observed risk at five years with mean predicted risks by 10th of predicted risk), and displayed these as plots. Discrimination is the ability of the risk prediction model to differentiate between patients who do and do not experience an event during the study. This can be quantified by calculating the area under the receiver operating characteristic curve statistic; where a value of 1 represents perfect discrimination. The D statistic32 is also a measure of discrimination that is specific to censored survival data. As with the receiver operating characteristic curve, higher values indicate better discrimination. We also calculated an R2 measure specific to censored survival data—it measures explained variation, and higher values indicate more variation explained.33
We used the predicted risks in the QResearch validation cohort to define a threshold for the top 10% of patients at highest predicted risk of bleed. We carried out a separate analysis restricted to patients in the validation cohort with atrial fibrillation. We used all the available data on the QResearch database to maximise the power and also the generalisability of the results. We used STATA (version 13) for all analyses.
Results
Practices and patients
Overall, 753 practices in England met our inclusion criteria and all had been using their current computer system for at least one year. Of these, 565 were randomly assigned to the derivation dataset and 188 to the validation dataset. We identified 4 437 419 patients aged 21-99 years in the derivation cohort. Of these, 7870 (0.2%) had a missing Townsend score and 50 698 (1.2%) had at least one prescription for anticoagulants in the 180 days before study entry, leaving 4 378 851 eligible patients for analysis. We identified 1 373 253 patients aged 21-99 years in the validation cohort. Of these, 2041 (0.1%) had a missing Townsend score and 15 022 (1.1%) had at least one prescription for anticoagulants in the 180 days before their study entry date, leaving 1 358 231 eligible patients for analysis.
Table 1⇓ shows the characteristics of eligible patients prescribed warfarin, novel anticoagulants, and neither drug for both the derivation and validation cohort recorded before entry to the cohort. Of the 52 485 patients in the derivation cohort prescribed warfarin, 23% had a previous bleed. Of the 1641 patients prescribed novel anticoagulants, 27% had a previous bleed. Overall, 27% of patients prescribed warfarin had concurrent prescriptions for non-steroidal anti-inflammatory drugs, 11% for corticosteroids, and 37% for antiplatelets. The corresponding figures for patients prescribed novel anticoagulants were 39%, 10%, and 35%. Although the validation cohort was drawn from an independent group of practices, the baseline characteristics were generally similar to those for the derivation cohort.
Characteristics of patients at baseline comparing those not taking any anticoagulant, prescribed warfarin, and prescribed another type of anticoagulant except for warfarin
Incidence rates of bleed
Table 2⇓ shows the incidence of both types of bleed during follow-up in patients prescribed warfarin, new anticoagulants, or neither drug.
Incident haemorrhagic events in derivation cohort recorded on mortality or hospital records
Overall in the derivation cohort, we identified 21 641 patients with an incident upper gastrointestinal bleed and 9040 with an intracranial bleed, arising from 16.4 million person years of observation. The crude incidence rate for upper gastrointestinal bleed was 1.32 per 1000 person years. The crude incidence rate for intracranial bleeds was 0.55 per 1000 person years.
We standardised incidence rates to the overall age distribution of the derivation cohort. The age standardised incidence rate of upper gastrointestinal bleed per 1000 person years was 1.34 (95% confidence interval 1.32 to 1.36) in patients not prescribed anticoagulants, 5.83 (4.33 to 7.33) in those prescribed warfarin, and 2.69 (0.36 to 5.03) in those prescribed new anticoagulants. The corresponding figures for intracranial bleed per 1000 person years were 0.55 (0.54 to 0.56), 1.89 (1.46 to 2.32), and 1.49 (1.27 to 2.86).
Model development
Upper gastrointestinal bleed
Table 3⇓ shows the models for upper gastrointestinal bleed in men and women. Details of the fractional polynomial terms for age and body mass index are given in the footnote of table 3 and in figure 1⇓. The final model included 20 variables in addition to use of anticoagulants. These were age, body mass index, Townsend score, smoking status, ethnicity, alcohol intake, previous bleed, oesophageal varices, chronic liver disease or pancreatitis, atrial fibrillation, venous thromboembolism, congestive cardiac failure, treated hypertension, cancer, recent abnormal platelet counts (<150/μL or >480/μL), and current prescriptions for antiplatelets, non-steroidal anti-inflammatory drugs, corticosteroids, antidepressants, and anticonvulsants (phenytoin or carbamazepine).
Fig 1 Graphs showing fractional polynomial terms for age (hazard ratios compared with age 40) and body mass index (hazard ratios compared with body mass index 25) and age interactions for upper gastrointestinal bleed
Adjusted hazard ratios (95% confidence intervals) in derivation cohort for upper gastrointestinal haemorrhagic events recorded on mortality or hospital records
At the mean age, there was a fourfold increased risk of upper gastrointestinal bleed associated with new use of anticoagulants in men (adjusted hazard ratio 4.43, 95% confidence interval 3.32 to 5.92) and in women (3.89, 2.75 to 5.49). The risk of upper gastrointestinal bleed increased with age in both sexes (fig 1). Two interactions were included in the final model, which were between the age terms and anticoagulant use and between age and previous bleed. Both of these indicated higher hazard ratios for anticoagulant use and previous bleed among younger patients compared with older patients (fig 1).
In terms of concurrent prescribed drugs, the risks associated with antidepressants were higher than those associated with non-steroidal anti-inflammatory drugs and antiplatelets. In women, the adjusted hazard ratio for antidepressants was 1.57 (95% confidence interval 1.51 to 1.64) and for non-steroidal anti-inflammatory drugs was 1.16 (1.11 to 1.21). There was a suggestion of a dose response for both alcohol and smoking. Heavy and moderate smokers had higher risks than non-smokers and former smokers. There was a “U shaped” effect for alcohol intake, with lower risks associated with trivial, light, and moderate alcohol consumption compared with none, and higher risks associated with heavy and very heavy consumption.
Intracranial bleed
Table 4⇓ shows the adjusted hazard ratios and 95% confidence intervals for the final model for intracranial bleed. The final model included 15 variables in addition to use of anticoagulants, which were age, body mass index, Townsend deprivation score, smoking status, ethnicity, alcohol intake, previous bleed, oesophageal varices, chronic liver disease or pancreatitis, atrial fibrillation, treated hypertension, recent abnormal platelet counts (<150/μL or >480/μL), and current prescriptions for antiplatelets, antidepressants, and anticonvulsants (phenytoin or carbamazepine). The footnote of table 4 gives details of the fractional polynomial terms for age and body mass index. One interaction terms were included in the final model, which were between the age terms and anticoagulant use.
Adjusted hazard ratios (95% confidence interval) in the derivation cohort for intracranial haemorrhagic events recorded on mortality or hospital records
At the mean age, there was around a fourfold increased risk of intracranial bleed associated with new use of anticoagulants in men (adjusted hazard ratio 3.99, 95% confidence interval 1.86 to 8.55) and in women (3.62, 1.25 to 10.49). The risk of intracranial bleed increased with age in both sexes (fig 2⇓). The age interaction terms indicated that middle aged men and women tended to have higher hazard ratios associated with anticoagulation compared with younger and older patients (fig 2).
Fig 2 Graphs showing fractional polynomial terms for age (hazard ratios compared with age 40) and body mass index (hazard ratios compared with body mass index 25) and interaction between anticoagulant use and age for intracranial bleed
Calibration and discrimination in the validation cohort
For the validation cohort, we identified nearly 1.4 million patients contributing over 4.9 million person years of follow-up. During follow-up, 6600 patients had an incident upper gastrointestinal bleed and 2820 had an intracranial bleed. Table 5⇓ shows the performance of each model in the validation cohort.
Performance of each algorithm in validation cohort using incident events included in hospital or mortality data
For the upper gastrointestinal bleed outcome, the algorithms explained approximately 41% of the variation in women and 37% of the variation in men. The D statistic was 1.70 in women and 1.57 in men. The receiver operating characteristic curve value was 0.77 for women and 0.75 for men. For intracranial bleed, the algorithms explained approximately 58% of the variation in women and 53% of the variation in men. The D statistic was 2.40 in women and 2.19 in men. The receiver operating characteristic curve value was 0.85 for women and 0.81 for men.
To assist in the comparison with other bleeding risk scores we repeated the validation restricting the sample to men and women with atrial fibrillation (n=12 560) and we used the intracranial bleed outcome, which is comparable to that used in validation analyses of other scores.15 The receiver operating characteristic curve statistic for men and women combined was 0.65 (0.61 to 0.69).
Table 6⇓ shows the performance statistics of both algorithms in the full validation cohort for the 10% of men and women at highest predicted risk of each outcome. For example, for the 10% of men and women at highest predicted risk of an intracranial bleed (that is, those with a five year predicted risk score of 0.7% or higher), the sensitivity was 51%, and the observed risk at five years was 1.5%. For upper gastrointestinal bleed, the corresponding sensitivity was 38%, with an observed risk of 2.7% at five years.
Performance of each model in the validation cohort based on top 10% of patients at highest risk (n=135 823 patients). Values are point estimates (95% confidence intervals)
Figure 3⇓ displays the predicted and observed risks of bleed at five years across each 10th of predicted risk (1 representing the lowest risk and 10 the highest risk) for each of the four models. Overall the models were well calibrated.
Fig 3 Mean predicted risks and observed risks at five years by 10th of predicted risk applying QBleed risk prediction scores to all patients in QResearch validation cohort
Figure 4⇓ shows two clinical examples of using the web calculator at www.qbleed.org to calculate the absolute risk of bleed with and without anticoagulants in individual patients.
Fig 4 Example of risk of bleed with and without anticoagulants in two patients using the web calculator at www.qbleed.org
Discussion
We developed and validated two models (collectively known as the QBleed algorithms) to identify patients at high risk of gastrointestinal or intracranial bleed. The QBleed algorithms incorporate predictor variables that are associated with an increased risk of haemorrhagic events, including sociodemographic variables, lifestyle, morbidity, drugs, and laboratory test results such as abnormal platelet function. The algorithms can be applied to any adult in a primary care setting regardless of whether they have had a previous bleed and have the advantage of evaluating risk in new users of oral anticoagulants, which is likely to be the highest risk period for bleeding.
The QBleed algorithms are intended to help inform decisions within the consultation about the risks and benefits of patients in primary care using anticoagulants. For example, the doctor can assess the patient’s five year risk of stroke using QStroke5 or five year risk of thrombosis using QThrombosis4 against the risk of bleed with or without anticoagulation using QBleed. The doctor and patient also can review factors that might ameliorate the risks, such as amending concurrent drugs. Overall this could help the doctor and patient assess whether the balance of risks and benefits is likely to be favourable or not given the patient’s profile.
The QBleed algorithms provide an estimate of absolute risk of two types of bleed for each year for each of the next five years. The algorithms are designed to be used before anticoagulant treatment is started using information already available at the time, rather than to guide the continuing use of anticoagulants, taking into account changes in treatment during follow-up. The algorithms to quantify risk of intracranial bleed performed better than the algorithms to predict upper gastrointestinal bleed.
Comparison with other risk scores
We reviewed other clinical risk prediction tools designed to help quantify risk of bleed among patients prescribed anticoagulants, including HEMOR2RHAGES,10 ATRIA, RIETE, OBRI,12 and HAS-BLED,13 although these are all simple counting systems rather than tools to quantify absolute risk as in our study. They also predict a combined general “major bleed” outcome, which is not directly comparable to our study since they include events that cannot be identified from routinely collected data. The HAS-BLED alogorithm is a scoring system derived from a population of 3978 patients aged 18 or more with atrial fibrillation who had 53 haemorrhagic events during a one year follow-up period.13 The score includes the variables: hepatic or renal disease; use of aspirin or non-steroidal anti-inflammatory drugs or alcohol; older age (≥65 years); previous bleed; and uncontrolled hypertension, stroke, and labile international normalised ratio. The international normalised ratio test is taken while a patient is prescribed warfarin, therefore the HAS-BLED score is more applicable to patients with atrial fibrillation already prescribed warfarin than to new users of anticoagulants as in our study who may have additional indications for anticoagulation. One of the key problems with the HAS-BLED score is that the variables included in it nearly always scored parallel to the stroke risk (for example, CHADS2) score in patients with atrial fibrillation. This made it difficult to assess whether the risk of stroke or the risk of bleeding was higher. In contrast, QBleed and its associated risk scores (QStroke5 and QThrombosis4 for assessing stroke and thrombosis risk, respectively) use different variables, which are separately weighted and so are likely to give better assessment of the different risks. Also, HAS-BLED is a counting system resulting in a score between 0 and 9 rather than a measure of absolute risk of bleed as with QBleed. Although not directly comparable to HAS-BLED, the receiver operating characteristic curve value for QBleed (intracranial bleed) among patients with atrial fibrillation was 0.65 (95% confidence interval 0.61 to 0.69) in our validation cohort, which is slightly higher than the median C statistic from six cohort studies for HAS-BLED (0.61, 0.59 to 0.62).3
HEMOR2RHAGES is another scoring system, which was derived from a cohort of 3791 elderly patients aged 65-95 years with atrial fibrillation participating in an American insurance scheme.10 The study was small and included only 162 haemorrhagic events during follow-up. The score ranges from 0 to 12 and takes account of hepatic or renal disease, alcohol misuse, cancer, older age (≥75 years), previous bleed, reduced platelet count, hypertension, anaemia, excessive risk of falls, stroke, and genetic polymorphism. We were unable to apply it to our dataset as QResearch does not include some of the relevant variables such as genetic information. A recent review by the National Institute for Health and Care Excellence reported that the median C statistic for HEMOR2RHAGES from four cohort studies was 0.61 (95% confidence interval 0.56 to 0.65).3 Although these studies are not directly comparable to QBleed, the C statistics presented tended to be lower than the receiver operating characteristic curve value we report for QBleed (intracranial bleed) in patients with atrial fibrillation (0.65, 0.61 to 0.69).
Incidence rates and predictor variables
The incidence rates of events and the predictor variables included in our final models have good face validity compared with the literature. For example, our crude rate of hospital admission or death for upper gastrointestinal haemorhage (1.32 per 1000 person years) is similar to that reported by Button and colleagues in a Welsh study (1.34 per 1000 person years).23 It is marginally higher than the rates reported by Crooks and colleagues24 (0.87 per 1000 person years) and Rockall and colleagues34 (1.03 per 1000 person years), both of which used slightly different definitions. Our crude incidence rate of intracranial bleed (which includes intracerebral, subarachnoid , intradural, and extradural bleed) was 0.55 per 1000 person years, which is consistent with the incidence of intracerebral and subarachnoid bleeds of 0.4 per 1000 person years reported by Nilsson in Sweden35 and with the incidence rate for intracerebral bleeds of 0.25 per 1000 reported in a systematic review by Van Asch.36
We found that nearly all of the relevant available predictors included in HEMOR2RHAGES10 and HAS-BLED13 met the inclusion criteria for incorporation into our models. We found that increasing levels of material deprivation were associated with higher risk of bleed, including upper gastrointestinal bleed, supporting the findings of Button and colleagues.23 Similar to Tata and colleagues, we found that antidepressants were independently associated with an increased risk of upper gastrointestinal bleed.9 For some variables, which were tested but not included in our final model (such as rifampicin and quinidine), there were too few patients exposed and too few events to allow a robust analysis. We decided to include several variables in the models (such as peptic ulcer and oesophageal varices), although they have now been identified as a contraindication to the new anticoagulants. This information was published after the end of the study period and does not yet apply to warfarin. One of the advantages of developing risk tools using QResearch is that the risk model can be updated over time should new predictors be identified or existing predictors lose relevance.
Thresholds
We have not provided definite comment on what threshold of absolute risk should be used for intervention or avoiding anticoagulants as that would require consideration of the balance of risk and benefit for an individual and their choice and require cost-effectiveness analyses, which are outside the scope of this study. We have, however, provided analyses using the top 10% of absolute risk as the threshold of risk, which can be used to help inform future analyses. Sensitivity is important as it is a measure of how well the algorithm performs in finding cases that might be suitable for intervention. If the risk threshold is set too high, then the sensitivity will be low and a large number of patients with bleeds will be “missed” by the algorithm. Conversely, a high risk threshold is likely to result in a higher positive predictive value, which means a higher proportion of those identified are likely to go on to have a bleed. So, at the population level there is a balance to be struck between the sensitivity and positive predictive value of the score, which depends on the risk threshold selected, resources available, and likely risks and benefits of the interventions.
At the individual level, we envisage that the doctor and patient will review the absolute risk of intracranial or upper gastrointestinal bleed with and without anticoagulation. They can then compare this against the patient’s baseline risk of having a stroke or thrombosis and potentially the expected benefit to be derived from starting anticoagulation (although this is outside the scope of the current paper). They can then use this information to balance the estimated absolute individualised risks and benefits to inform the decision about anticoagulants.
Comparison between warfarin and new anticoagulants
We were interested to be able to quantify risk separately for new use of warfarin compared with new use of other anticoagulants (dabigatran, rivaroxaban, apixaban, or phenindione). However, it was apparent from initial analyses that the numbers of patients prescribed these new drugs are as yet insufficient as are the numbers of events over time to quantify the effects as separate variables with the risk model. We were able to compare the baseline characteristics of patients prescribed each category of drug (table 1) and to calculate age standardised incidence rates of events for our main outcomes separately for warfarin compared with the novel anticoagulants (table 2). The age standardised incidence rates of both types of bleed were higher with warfarin than with the new anticoagulants, but the confidence intervals overlapped. In light of the announcement by the US Food and Drug Administration on 30 December 2013,37 there is a need to further investigate potential safety issues with the new anticoagulants once more data are available. We plan to update the QBleed score when sufficient data are available to undertake direct comparisons between the older and newer agents, at which point we will be able to decide whether they should be presented as separate or combined variables.
Methodological considerations
The methods to derive and validate this model are the same as for a range of other clinical risk prediction tools derived from the QResearch database.4 25 26 38 39 The strengths and limitations of the approach have already been discussed in detail,4 19 38 40 41 42 including information on multiple imputation of missing data. In summary, key strengths include size, duration of follow-up, and representativeness, and lack of selection, recall, and respondent bias. UK general practices have good levels of accuracy and completeness in recording clinical diagnoses and prescribed drugs.43 We think our study has good face validity since it was conducted in the setting where most patients in the United Kingdom are assessed, treated, and followed up. Our database has linked hospital and mortality records for nearly all patients and is therefore likely to have picked up the majority of severe bleeds requiring admission to hospital or resulting in death thereby minimising ascertainment bias. We excluded patients without a valid deprivation score since this group may represent a more transient population where follow-up could be unreliable or unrepresentative. Their deprivation scores are unlikely to be missing at random so we did not think it would be appropriate to impute them.
We used an incident user design to help minimise the impact of the biases that can arise from non-randomised studies such as in the current study.20 In particular it can help reduce biases that occur when comparisons include patients who were already using the drug at the start of the study. An incident user design also helps to ensure that the analysis captures all events that occurred after the start of treatment and reduces the chance of bias that could otherwise occur from patients who stop treatments as a result of early side effects.20 There is no consensus on the duration of the treatment-free period before study entry to classify a patient as a new user. Ray and colleagues defined incident use of non-steroidal anti-inflammatory drugs as patients who had not been prescribed treatment in the 365 days before entry to the cohort.44 We selected 180 days to help ensure our methods were comparable to the recently published FDA protocol to assess safety of new anticoagulants.37
The present validation has been done on a completely separate set of practices and individuals to those used to develop the score, although the practices all use the same general practice clinical computer system (EMIS, the computer system used by 55% of UK general practices). An independent validation study would be a more stringent test and should be done, but when such independent studies have examined other risk algorithms,41 42 45 46 they have shown comparable performance compared with the validation in the QResearch database.25 38 40 We (or another academic team) intend to conduct a separate validation on an independent database using a different clinical system (Clinical Practice Research Datalink, CPRD) as part of a separate project.
Other limitations of our study were that we measured exposure to anticoagulants based on prescriptions issued rather than a more direct measure of adherence to reflect whether the patient had actually taken the anticoagulants. However, we think it would be unusual for patients to continue to request repeat prescriptions for drugs that they are not taking. Also we did not take account of the degree of anticoagulation achieved during follow-up by analysing measurements for international normalised ratios. This was because our intention was to develop a score that would inform the decision to start anticoagulants, which would be based on information already available to the doctor and the patient rather than information that could vary during follow-up. Other limitations of our study included the lack of formally adjudicated outcomes, information bias, and potential for bias due to missing data. Also we included only two severe haemorrhagic outcomes, but these account for most types of severe bleed likely to be associated with anticoagulation. We were unable to include some rarer haemorrhagic outcomes since these either do not have ICD codes or are not routinely recorded in the datasets available to us.
These QBleed models have been developed using data from general practices in England and include a postcode related deprivation score. The models are therefore not likely to be applicable for clinical use in international settings without some modification of the English specific risk factors, and validation in the setting in which they are intended to be used. We included the Townsend score in the QBleed models since there is a clear relation between risk of bleeding and deprivation, which is captured by this variable. If we omitted it, it would tend to underestimate bleeding risk in patients from deprived areas. The corresponding QStroke score (which is used to identify some of those at high risk of stroke who might benefit from anticoagulation) includes deprivation and so we considered it would be more consistent to include it in both scores. In terms of international use of QBleed, where the Townsend score is not available, a locally relevant deprivation score could be constructed or adapted for a particular country to have a range between −11 (affluent) and 7 (deprived) to correspond to the range of Townsend scores that could be used instead (subject to local validation).
Conclusion
We developed and validated a set of new algorithms to predict absolute risk of upper gastrointestinal and intracranial bleed in new users of anticoagulants. QBleed has some advantages compared with current risk scoring methods. QBleed also provides an accurate measure of absolute risk of bleed in the general population as shown by its performance in a separate validation cohort. Further research is needed to evaluate the clinical outcomes and cost effectiveness of using this algorithm in primary care.
What is already known on this topic
Methods to identify patients at increased risk of intracranial and gastrointestinal bleed are needed to identify those for whom interventions or further assessment may be required
Current risk scoring methods do not take account of some established risk factors or are difficult to implement
What this study adds
We developed new algorithms to quantify the absolute risk of intracranial and upper gastrointestinal bleeds, which include established risk factors and are designed to work in primary care
The QBleed algorithms provide valid measures of absolute bleed risk in the general population of patients, as shown by the performance in a separate validation cohort, with some advantages over other scoring methods
Further research is needed to evaluate the clinical outcomes and cost effectiveness of using these algorithms in primary care
Notes
Cite this as: BMJ 2014;349:g4606
Footnotes
We thank the practices who use EMIS (Egton Medical Information System) and contribute to QResearch, and we thank EMIS and the University of Nottingham for expertise in establishing, developing, and supporting the database.
A simple web calculator to implement the QBleed algorithms will be publically available alongside the paper (http://qbleed.org). It also has the open source software for download.
Contributors: JHC initiated the study; undertook the literature review, data extraction, data manipulation, and primary data analysis; and wrote the first draft of the paper. She is the guarantor. CC contributed to the design and to the analysis, interpretation, and drafting of the paper.
Funding: This study received no external funding.
Competing interests: Both authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: JHC is professor of clinical epidemiology at the University of Nottingham and co-director of QResearch, a not for profit organization, which is a joint partnership between the University of Nottingham and EMIS (leading commercial supplier of information technology for 55% of general practices in the United Kingdom). JHC is also director of ClinRisk, which produces software to ensure the reliable and updatable implementation of clinical risk algorithms within clinical computer systems to help improve patient care. CC is associate professor of Medical Statistics at the University of Nottingham and a consultant statistician for ClinRisk. This work and any views expressed within it are solely those of the authors and not of any affiliated bodies or organisations.
Ethical approval: The project has been independently reviewed in accordance with the QResearch agreement with Trent research ethics committee (reference 03/4/021).
Data sharing: The patient level data from the QResearch are specifically licensed according to its governance framework. See www.qresearch.org for further details. The QBleed algorithms will be published as open source software under the GNU Lesser Public License.
Transparency: The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.