Table 3

Overview of prediction models for prognosis of covid-19

Study; setting; and outcomePredictors in final modelSample size: total No of participants for model development set (No with outcome)Predictive performance on validationOverall risk of bias
using PROBAST
Type of validation*Sample size: total No of participants for model validation (No with outcome)Performance* (C index, sensitivity (%), specificity (%), PPV/NPV (%), calibration slope, other (95% CI, if reported))
Prognosis
Original review
 Bai et al9; data from China, inpatients at admission with mild confirmed covid-19; deterioration into severe/critical disease (period unspecified)Combination of demographics, signs and symptoms, laboratory results and features derived from CT images133 (54)UnclearNot applicableC index 0.95 (0.94 to 0.97)High
 Caramelo et al19; data from China, target population unclear; mortality (period unspecified)††Age, sex, presence of any comorbidity (hypertension, diabetes, cardiovascular disease, chronic respiratory disease, cancer)††UnknownNot reportedNot applicableNot reportedHigh
 Lu et al20; data from China, inpatients at admission with suspected or confirmed covid-19; mortality (within 12 days)Age, CRP577 (44)Not reportedNot applicableNot reportedHigh
 Qi et al21; data from China, inpatients with confirmed covid-19 at admission; hospital stay >10 days6 features derived from CT images‡‡ (logistic regression model)26 (20)Fivefold cross validationNot applicableC index 0.92High
 Qi et al21; data from China, inpatients with confirmed covid-19 at admission; hospital stay >10 days6 features derived from CT images‡‡ (random forest)26 (20)5 fold cross validationNot applicableC index 0.96High
 Shi et al37; data from China, inpatients with confirmed covid-19 at admission; death or severe covid-19 (period unspecified)Age (dichotomised), sex, hypertension478 (49)Validation in less severe cases66 (15)Not reportedHigh
 Xie et al7; data from China, inpatients with confirmed covid-19 at admission; mortality (in hospital)Age, LDH, lymphocyte count, SPO2299 (155)External validation (other Chinese centre)130 (69)C index 0.98 (0.96 to 1.00), calibration slope 2.5 (1.7 to 3.7)High
 Yan et al22; data from China, inpatients suspected of covid-19; mortality (period unspecified)LDH, lymphocyte count, high sensitivity CRP375 (174)Temporal validation, selecting only severe cases29 (17)Sensitivity 92, PPV 95High
 Yuan et al23; data from China, inpatients with confirmed covid-19 at admission; mortality (period unspecified)Clinical scorings of CT images (zone, left/right, location, attenuation, distribution of affected parenchyma)Not applicableExternal validation of existing model27 (10)C index 0.90 (0.87 to 0.93)High
Update 1
 Huang et al60; data from China, inpatients with confirmed covid-19 at admission; severe symptoms three days after admissionUnderlying diseases, fast respiratory rate >24/min, elevated CRP level (>10 mg/dL), elevated LDH level (>250 U/L)125 (32)Apparent performance onlyNot applicableC index 0.99 (0.97 to 1.00), sensitivity 91, specificity 96High
 Pourhomayoun et al61; data from 76 countries, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)UnknownUnknown10-fold cross validationNot applicableC index 0.96, sensitivity 90, specificity 97High
 Sarkar et al44; data from several continents (Australia, Asia, Europe, North America), inpatients with covid-19 symptoms; death v recovery (period unspecified) Age, days from symptom onset to hospitalisation, from Wuhan, sex, visit to Wuhan80 (37)Apparent performance onlyNot applicableC index 0.97High
 Wang et al42; data from China, inpatients with confirmed covid-19; length of hospital stayAge and CT features301 (not applicable)Not reportedNot applicableNot reportedHigh
 Zeng et al62; data from China, inpatients with confirmed covid-19; severe disease progression (period unspecified)CT features338 (76)Cross validation (number of folds unclear)Not applicableC index 0.88High
 Zeng et al62; data from China, inpatients with confirmed covid-19; severe disease progression (period unspecified)CT features and laboratory markers338 (76)Cross validation (number of folds unclear)Not applicableC index 0.88High
Update 2
 Al-Najjar et al63; data from South Korea, target population unclear; recovery from covid-19 (period unspecified)Birth year (age), sex, country, group, infection reason, confirmed date466 (40)Training test split193 (14)Sensitivity 43, specificity 98High
 Al-Najjar et al63; data from South Korea, target population unclear; mortality (period unspecified)Age, sex, country, region, infection reason, confirmed date463 (25)Training test split191 (7)Sensitivity 86, specificity 100 High
 Barda et al67; data from Israel, patients with confiremed covid-19; mortality (period unspecified)Age, sex, pack years, COPD, number of wheezing/dyspnea diagnoses, albumin, red cell distribution width, C reactive peptide, urea, lymphocyte, chloride, creatinine, high density lipoprotein, duration of hospital admissions, count of hospital admissions, count of ambulance rides, count of sulfonamide dispenses, count of anticholinergic dispenses, count of glucocorticoid dispenses, chronic respiratory disease, cardiovascular disease, diabetes, malignancy, hypertension735,000 (8251)Other (specify in column CL) January 29 to April 8 inclusion, follow-up until April 22 2020 (covid cases for external validation)3176 (87)C index 0.94 (0.92 to 0.96) , sensitivity 90 (83 to 96), PPV 17 (14 to 21)High
 Bello-Chavolla et al70; data from Mexico, confirmed covid-19 patients presenting at GP; 30-day mortalityAge, pregnancy, diabetes, obesity, pneumonia, CKD, COPD, immunosuppression12424 (1137)Training test split3105 (297)C index 0.80, Somer's D 0.60High
 Carr et al75; data from United Kingdom, inpatients with confirmed covid-19; progression to severe covid-19 (period unspecified)Age, National Early Warning Score (NEWS) 2, CRP, neutrophil, eGFR, albumin 452 (159)Temporal validation 256 (59)C index 0.73, sensitivity 46, specificity 87High
 Chassagnon et al77; data from France, inpatients with confirmed covid-19; composite, 4-day intubation or mortality Unclear383 (84)External validation (new centres, France)95 (26)Sensitivity 88, specificity 74High
 Colombi et al79; data from Italy, inpatients with confirmed covid-19; ICU admission or in-hospital (period unspecified)Age, cardiovascular comorbidities, median platelet count, CRP, visual assessment of well aerated lung %236 (108)Apparent performance onlyNot applicableC index 0.86 (0.81 to 0.90), sensitivity 72 (63 to 80), specificity 81 (73 to 88) PPV 70 (61 to 78), NPV 78 (72 to 83)High
Colombi et al79; data from Italy, inpatients with confirmed covid-19; ICU admission or in-hospital mortality (period unspecified)Age, cardiovascular comorbidities, median platelet count, LDH, CRP, software assessment of well aerated lung absolute volume, adipose tissue236 (108)Apparent performance onlyNot applicableC index 0.86 (0.81 to 0.90), sensitivity 75 (66 to 83), specificity 81 (73 to 88), PPV 70 (61 to 78), NPV 78 (72 to 83)High
 Das et al80; data from South Korea, inpatients with confirmed covid-19; ICU admission or in-hospital mortality (period unspecified)Age, sex, province, date of diagnosis, place of exposure to covid-193022 (61)Training test split604 (12)C index 0.97High
 Gong et al119; data from China, target population unclear; 15-day progression to severe covid-19Age, direct bilirubin, red cell distribution width, blood urea nitrogen, CRP, lactate dehydrogenase, albumin189 (28)External validation (new centres, China)165 (40)C index 0.85 (0.79 to 0.92), sensitivity 78, specificity 78 High
 Guo et al83; data from China, inpatients with confirmed covid-19; 14-day progression to severe covid-19 Age, chronic illness, neutrophil to lymphocyte ratio, CRP, D-dimer818 (24)External validation (new centres, China)320 (38)C index 0.78 (0.70 to 0.87)High
 Hu et al84; data from China inpatients with confirmed covid-19;in-hospital mortality (period unspecified)Age, high-sensitivity CRP, lymphocyte count, D-dimer183 (68)External validation (new centres, China)64 (31)C index 0.88, sensitivity 84, specificity 79 High
 Hu et al85; data from China, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)Modified Early Warning Score (MEWS): heart rate, systolic blood pressure, respiratory rate, body temperature, consciousnessNot applicableExternal validation only105 (19)C index 0.68 (0.58 to 0.77), sensitivity 68, specificity 65,
PPV 30, NPV 90
High
 Hu et al85; data from China, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)Rapid Emergency Medicine Score (REMS): mean arterial pressure, pulse rate, respiratory rate, oxygen saturation, GCS, ageNot applicableExternal validation only105 (19)C index 0.84 (0.76 to 0.91), sensitivity 89, specificity 70,
PPV 40, NPV 97
High
  Ji et al88; data from China, inpatients with confirmed covid-19; 10-day progression to severe COVID-19Comorbidity, age, lymphocyte count, lactate dehydrogenase208 (40)Internal validation by resampling (bootstrap)Not applicableC index 0.91 (0.86 to 0.94), sensitivity 95 (83 to 99), specificity 78 (71 to 84)High
 Jiang et al89; data from China, inpatients with confirmed covid-19; acute respiratory distress syndrome***Alanine aminotransferase, myalgias, haemoglobin, sex, temp, Na+, K+, lymphocyte count, creatinine, age, white blood count53 (5)10-fold cross validation Not applicableClassification accuracy 50%High
 Jiang et al89; data from China, inpatients with confirmed covid-19; acute respiratory distress syndrome***Alanine aminotransferase, myalgias, haemoglobin, sex, temp, Na+, K+, lymphocyte count, creatinine, age, white blood count53 (5)10-fold cross validation Not applicableClassification accuracy 80%High
 Jiang et al89; data from China, inpatients with confirmed covid-19; acute respiratory distress syndrome***Alanine aminotransferase, myalgias, haemoglobin, sex, temp, Na+, K+, lymphocyte count, creatinine, age, white blood count53 (5)10-fold cross validation Not applicableClassification accuracy 70%High
 Jiang et al89; data from China, inpatients with confirmed covid-19; acute respiratory distress syndrome***Alanine aminotransferase, myalgias, haemoglobin, sex, temp, Na+, K+, lymphocyte count, creatinine, age, white blood count53 (5)10-fold cross validation Not applicableClassification accuracy 70%High
 Jiang et al89; data from China, inpatients with confirmed covid-19; acute respiratory distress syndrome***Alanine aminotransferase, myalgias, haemoglobin, sex, temp, Na+, K+, lymphocyte count, creatinine, age, white blood count53 (5)10-fold cross validation Not applicableClassification accuracy 70%High
 Jiang et al89; data from China, inpatients with confirmed covid-19; acute respiratory distress syndrome***Alanine aminotransferase, myalgias, haemoglobin, sex, temp, Na+, K+, lymphocyte count, creatinine, age, white blood count53 (5)10-fold cross validation Not applicableClassification accuracy 80%High
 Levy et al96; data from USA, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)Age, serum blood urea nitrogen, emergency severity index, red cell distribution width, absolute neutrophil count, serum bicarbonate, glucoseUnknownLeave-one-out cross validationNot applicable C index 0.83High
 Levy et al96; data from USA, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)SOFA scoreNot applicableExternal validation onlyUnclearC index 0.73High
 Levy et al96; data from USA, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)CURB-65 scoreNot applicableExternal validation onlyUnclearC index 0.74 High
 Levy et al96; data from USA, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)SOFA+ scoreNot applicableExternal validation onlyUnclearC index 0.83High
 Liu et al98; data from China, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)Age, underlying disease status, helper T cells, helper T cells and suppressor T cells ratio340 (30)Apparent performance onlyNot applicableMcFadden pseudo R-squared 0.35High
 McRae et al100; data from China, inpatients with confirmed covid-19; in-hospital mortality (period unspecified)Age, sex, cardiac troponin I, CRP, procalcitonin, myoglobin160 (43)New centres in China, case series12 (unknown)C index 0.94 (0.89 to 0.99)High
 Singh et al108; data from USA, inpatients with confirmed covid-19; ICU-level care, mechanical ventilation or in-hospital mortality (period unspecified)Epic Deterioration IndexUnknownExternal validation only174 (61)C index 0.76 (0.68 to 0.84), sensitivity 39 PPV 80High
 Vaid et al112; data from USA, inpatients with confirmed covid-19; intubation, discharge to hospice care or mortalit (period unspecified)Sex, race, ethnicity, age, hypertension, atrial fibrillation, coronary artery disease, heart failure, stroke, chronic kidney disease, diabetes, asthma, COPD, cancer, heart rate, pulse, oximetry, respiration rate, temperature, systolic blood pressure, diastolic blood pressure, body weight, sodium, potassium, creatinine, lactate, white blood cells, lymphocyte percentage, haemoglobin, red blood cell distribution width, platelets, alanine, aminotransferase, aspartate, aminotransferase, albumin, total bilirubin, prothrombin time, partial thromboplastin time, PCO2, pH, CRP, ferritin, D-dimer, creatinine phosphokinase, lactate dehydrogenase, procalcitonin, troponin I1225 (37)External validation, new centres (USA)1830 (unknown)C index 0.84, sensitivity 86, specificity 82High
 Vazquez Guillamet et al113; data from USA, target population unclear; in-hospital mortality (period unspecified)Age, immunosuppression, COPD, congestive heart failure, BMI, sex, time to mechanical ventilation (days), length of hospital stay prior to hospital admission, PaO2/FiO2, Glasgow coma scale, maximum heart rate, maximum respiratory rate, minimum mean arterial blood pressure, maximum temperature, minimum albumin, minimum pH2122 (429)External validation, new centres (USA)1175 (154)C index 0.81, PPV 55, NPV 89High
  Vazquez Guillamet et al113; data from USA, target population unclear; mechanical ventilation >96 hoursAge, immunosuppression, COPD, congestive heart failure, BMI, sex, time to mechanical ventilation (days), length of hospital stay before hospital admission, PaO2/FiO2, Glasgow coma scale, maximum heart rate, maximum respiratory rate, minimum mean arterial blood pressure, maximum temperature, minimum albumin, minimum pH2167 (158)Training test split1063 (96)C index 0.81High
  Vazquez Guillamet et al113; data from USA, target population unclear; mechanical ventilation >96 hoursAge, immunosuppression, COPD, congestive heart failure, BMI, sex, time to mechanical ventilation (days), length of hospital stay prior to hospital admission, PaO2/FiO2, Glasgow coma scale, maximum heart rate, maximum respiratory rate, minimum mean arterial blood pressure, maximum temperature, minimum albumin, minimum pH1169 (141)Training test split619 (90)C index 0.78High
 Zhang et al116; data from China and United Kingdom, inpatients with confirmed covid-19; in hospital mortality (period unspecified)Age, sex, neutrophil count, lymphocyte count, platelet count, CRP, creatinine653 (20)External validation (new centres, different country)226 (77)C index 0.75, sensitivity 23, specificity 95, PPV 69, NPV 71 High
  Zhang et al116; data from China, inpatients with confirmed covid-19; ARDS, intubation or ECMO, ICU admission, in hospital mortality (period unspecified)Age, sex, chronic lung disease, diabetes mellitus, malignancy, cough, dyspnoea, immunocompromised, hypertension, heart disease, chronic renal disease, fever, fatigue, diarrhoea768 (72)Repeated five-fold cross validationNot applicableC index 0.80, sensitivity 9, specificity 99 PPV 53, NPV 91High
 Zhang et al116; data from China and United Kingdom, inpatients with confirmed covid-19; ARDS, intubation or ECMO, ICU admission, in hospital mortality (period unspecified)Age, sex, neutrophil count, lymphocyte count, platelet count, CRP, creatinine653 (58)External validation (new centres, different country)226 (97)C index 0.72, sensitivity 40, specificity 85, PPV 67, NPV 65High

ARDS=acute respiratory distress syndrome; BMI=body mass index; COPD=chronic obstructive pulmonary disease; covid-19=coronavirus disease 2019; CRP=C reactive protein; CT=computed tomography; ECMO=extracorporal membrane oxygenation; ICU=intensive care unit; LDH=lactate dehydrogenase; NPV=negative predictive value; PaO2/FiO2=the ratio of arterial oxygen partial pressure to fractional inspired oxygen; PCO2=partial pressure of carbon dioxide; PPV=positive predictive value; PROBAST=prediction model risk of bias assessment tool; SOFA=sequential organ failure assessment score; SPO2=oxygen saturation; Na+=sodium; K+=potassium.

  • * Performance is given for the strongest form of validation reported. This is indicated in the column “type of validation.” When a training test split was used, performance on the test set is reported. Apparent performance is the performance observed in the development data.

  • †† Outcome and predictor data were simulated.

  • ‡‡ Wavelet-HLH_gldm_SmallDependenceLowGrayLevelEmphasis, wavelet-LHH_glcm_Correlation, wavelet-LHL_glszm_GrayLevelVariance, wavelet-LLH_glszm_SizeZoneNonUniformityNormalized, wavelet-LLH_glszm_SmallAreaEmphasis, wavelet-LLH_glcm_Correlation.

  • *** Each model uses a different predictive algorithm.