Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort studyBMJ 2020; 371 doi: https://doi.org/10.1136/bmj.m3731 (Published 20 October 2020) Cite this as: BMJ 2020;371:m3731
- Ash K Clift, clinical research fellow1,
- Carol A C Coupland, professor of medical statistics in primary care2,
- Ruth H Keogh, professor of biostatistics and epidemiology3,
- Karla Diaz-Ordaz, associate professor of biostatistics3,
- Elizabeth Williamson, associate professor of medical statistics3,
- Ewen M Harrison, professor of surgery and data science4,
- Andrew Hayward, director5,
- Harry Hemingway, professor of clinical epidemiology6,
- Peter Horby, professor of emerging infectious diseases7,
- Nisha Mehta, clinical adviser8,
- Jonathan Benger, acting chief medical officer9,
- Kamlesh Khunti, professor of primary care, diabetes and vascular medicine10,
- David Spiegelhalter, professor of mathematics11,
- Aziz Sheikh, director4,
- Jonathan Valabhji, national clinical director for diabetes and obesity12,
- Ronan A Lyons, clinical professor of public health13,
- John Robson, clinical reader in primary care research and development14,
- Malcolm G Semple, professor in child health and outbreak medicine15,
- Frank Kee, professor of public health medicine16,
- Peter Johnson, national clinical director for cancer12,
- Susan Jebb, professor of diet and population health1,
- Tony Williams, consultant occupational physician17,
- Julia Hippisley-Cox, professor of clinical epidemiology and general practice1
- 1Nuffield Department of Primary Care Health Sciences, Radcliffe Observatory Quarter, Oxford OX2 6GG, UK
- 2Division of Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
- 3Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
- 4Usher Institute, University of Edinburgh, Edinburgh, UK
- 5UCL Institute of Epidemiology and Health Care, University College London, London, UK
- 6UCL Institute for Health Informatics, University College London, London, UK
- 7Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
- 8Office of the Chief Medical Officer, Department of Health and Social Care, London, UK
- 9NHS Digital, Leeds, UK
- 10Diabetes Research Centre, University of Leicester, Leicester, UK
- 11Winton Centre for Risk and Evidence Communication, Faculty of Mathematics, University of Cambridge, Cambridge, UK
- 12NHS England, London, UK
- 13Swansea University, Swansea, UK
- 14Centre for Primary Care and Public Health, Queen Mary University of London, London, UK
- 15Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Liverpool, UK
- 16Centre for Public Health, Queen’s University Belfast, Belfast, UK
- 17Association of Local Authority Medical Advisors, London, UK
- 18Imperial College London, London, UK
- Correspondence to: J Hippisley-Cox @juliahcox on Twitter) (or
- Accepted 23 September 2020
Objective To derive and validate a risk prediction algorithm to estimate hospital admission and mortality outcomes from coronavirus disease 2019 (covid-19) in adults.
Design Population based cohort study.
Setting and participants QResearch database, comprising 1205 general practices in England with linkage to covid-19 test results, Hospital Episode Statistics, and death registry data. 6.08 million adults aged 19-100 years were included in the derivation dataset and 2.17 million in the validation dataset. The derivation and first validation cohort period was 24 January 2020 to 30 April 2020. The second temporal validation cohort covered the period 1 May 2020 to 30 June 2020.
Main outcome measures The primary outcome was time to death from covid-19, defined as death due to confirmed or suspected covid-19 as per the death certification or death occurring in a person with confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in the period 24 January to 30 April 2020. The secondary outcome was time to hospital admission with confirmed SARS-CoV-2 infection. Models were fitted in the derivation cohort to derive risk equations using a range of predictor variables. Performance, including measures of discrimination and calibration, was evaluated in each validation time period.
Results 4384 deaths from covid-19 occurred in the derivation cohort during follow-up and 1722 in the first validation cohort period and 621 in the second validation cohort period. The final risk algorithms included age, ethnicity, deprivation, body mass index, and a range of comorbidities. The algorithm had good calibration in the first validation cohort. For deaths from covid-19 in men, it explained 73.1% (95% confidence interval 71.9% to 74.3%) of the variation in time to death (R2); the D statistic was 3.37 (95% confidence interval 3.27 to 3.47), and Harrell’s C was 0.928 (0.919 to 0.938). Similar results were obtained for women, for both outcomes, and in both time periods. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths within 97 days was 75.7%. People in the top 20% of predicted risk of death accounted for 94% of all deaths from covid-19.
Conclusion The QCOVID population based risk algorithm performed well, showing very high levels of discrimination for deaths and hospital admissions due to covid-19. The absolute risks presented, however, will change over time in line with the prevailing SARS-C0V-2 infection rate and the extent of social distancing measures in place, so they should be interpreted with caution. The model can be recalibrated for different time periods, however, and has the potential to be dynamically updated as the pandemic evolves.
Contributors: JHC, CC, AKC, RK, KDO, PH, and NM led study conceptualisation. All authors contributed to the development of the research question and study design, with development of advanced statistical aspects led by JHC, CC, RK, KDO, and AKC. JHC, AKC, CC, JB, and PJ were involved in data specification, curation, and collection. JHC and AKC developed, checked, or updated clinical code groups. JHC did the statistical analyses, which were checked by CC. JHC developed the software for the web calculator. All authors contributed to the interpretation of the results. AKC and JHC wrote the first draft of the paper. All authors contributed to the critical revision of the manuscript for important intellectual content and approved the final version of the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. JHC is the guarantor.
Funding: This study is funded by a grant from the National Institute for Health Research (NIHR) following a commission by the Chief Medical Officer for England, whose office contributed to the development of the study question, facilitated access to relevant national datasets, and contributed to interpretation of data and drafting of the report.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: JHC has received grants from the National Institute for Health Research Biomedical Research Centre, Oxford, John Fell Oxford University Press Research Fund, Cancer Research UK (grant number C5255/A18085) through the Cancer Research UK Oxford Centre, and the Oxford Wellcome Institutional Strategic Support Fund (204826/Z/16/Z) during the conduct of the study, is an unpaid director of QResearch, a not-for-profit organisation which is a partnership between the University of Oxford and EMIS Health who supply the QResearch database used for this work, and is a founder and shareholder of ClinRisk Ltd and was its medical director until 31 May 2019; ClinRisk produces open and closed source software to implement clinical risk algorithms (outside this work) into clinical computer systems; CC reports receiving personal fees from ClinRisk, outside this work; AH is a member of the New and Emerging Respiratory Virus Threats Advisory Group; PJ was employed by NHS England during the conduct of the study and has received grants from Epizyme and Janssen and personal fees from Takeda, Bristol-Myers-Squibb, Novartis, Celgene, Boehringer Ingelheim, Kite Therapeutics, Genmab, and Incyte, all outside the submitted work; AKC has previously received personal fees from Huma Therapeutics, outside of the scope of the submitted work; RL has received grants from Health Data Research UK outside the submitted work; AS has received grants from the Medical Research Council (MRC) and Health Data Research UK during the conduct of the study; CS has received grants from the DHSC National Institute of Health Research UK, MRC UK, and the Health Protection Unit in Emerging and Zoonotic Infections (University of Liverpool) during the conduct of the study and is a minority owner in Integrum Scientific LLC (Greensboro, NC, USA) outside of the submitted work; KK has received grants from NIHR, is the national lead for ethnicity and diversity for the National Institute for Health Applied Research Collaborations, is director of the University of Leicester Centre for Black Minority Ethnic Health, was a steering group member of the Risk reduction Framework for NHS staff (chair) and for Adult care Staff, is a member of Independent SAGE, and is supported by the NIHR Applied Research Collaboration East Midlands (ARC EM) and the NIHR Leicester Biomedical Research Centre (BRC); RHK was supported by a UKRI Future Leaders Fellowship (MR/S017968/1); KDO was supported by a grant from the Alan Turing Institute Health Programme (EP/T001569/1); no other relationships or activities that could appear to have influenced the submitted work. The views expressed are those of the author(s) and not necessarily those of the NIHR, the NHS, or the Department of Health and Social Care.
Ethical approval: The QResearch ethics approval is with East Midlands-Derby Research Ethics Committee (reference 18/EM/0400).
Data sharing: To guarantee the confidentiality of personal and health information, only the authors have had access to the data during the study in accordance with the relevant licence agreements. Access to the QResearch data is according to the information on the QResearch website (www.qresearch.org).
The lead author affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
Dissemination to participants and related patient and public communities: Patient representatives from the QResearch Advisory Board have advised on dissemination of studies using QResearch data, including the use of lay summaries describing the research and its findings.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.