Intended for healthcare professionals

CCBYNC Open access

Diagnostic prediction models for suspected pulmonary embolism: systematic review and independent external validation in primary care

BMJ 2015; 351 doi: (Published 08 September 2015) Cite this as: BMJ 2015;351:h4438
  1. Janneke M T Hendriksen, GP trainee1,
  2. Geert-Jan Geersing, general practitioner1,
  3. Wim A M Lucassen, general practitioner2,
  4. Petra M G Erkens, clinical epidemiologist3,
  5. Henri E J H Stoffers, general practitioner3,
  6. Henk C P M van Weert, professor of general practice2,
  7. Harry R Büller, professor of medicine4,
  8. Arno W Hoes, professor of general practice1,
  9. Karel G M Moons, professor of clinical epidemiology1
  1. 1Department of Epidemiology, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, 3508 GA Utrecht, Netherlands
  2. 2Department of General Practice, Academic Medical Center, Amsterdam, Netherlands
  3. 3Department of Family Medicine, CAHPRI School for Public Health and Primary Care, Maastricht University, Maastricht, Netherlands
  4. 4Department of Vascular Medicine, Academic Medical Center, Amsterdam, Netherlands
  1. Correspondence to: J M T Hendriksen j.m.t.hendriksen-9{at}
  • Accepted 31 July 2015


Objective To validate all diagnostic prediction models for ruling out pulmonary embolism that are easily applicable in primary care.

Design Systematic review followed by independent external validation study to assess transportability of retrieved models to primary care medicine.

Setting 300 general practices in the Netherlands.

Participants Individual patient dataset of 598 patients with suspected acute pulmonary embolism in primary care.

Main outcome measures Discriminative ability of all models retrieved by systematic literature search, assessed by calculation and comparison of C statistics. After stratification into groups with high and low probability of pulmonary embolism according to pre-specified model cut-offs combined with qualitative D-dimer test, sensitivity, specificity, efficiency (overall proportion of patients with low probability of pulmonary embolism), and failure rate (proportion of pulmonary embolism cases in group of patients with low probability) were calculated for all models.

Results Ten published prediction models for the diagnosis of pulmonary embolism were found. Five of these models could be validated in the primary care dataset: the original Wells, modified Wells, simplified Wells, revised Geneva, and simplified revised Geneva models. Discriminative ability was comparable for all models (range of C statistic 0.75-0.80). Sensitivity ranged from 88% (simplified revised Geneva) to 96% (simplified Wells) and specificity from 48% (revised Geneva) to 53% (simplified revised Geneva). Efficiency of all models was between 43% and 48%. Differences were observed between failure rates, especially between the simplified Wells and the simplified revised Geneva models (failure rates 1.2% (95% confidence interval 0.2% to 3.3%) and 3.1% (1.4% to 5.9%), respectively; absolute difference −1.98% (−3.33% to −0.74%)). Irrespective of the diagnostic prediction model used, three patients were incorrectly classified as having low probability of pulmonary embolism; pulmonary embolism was diagnosed only after referral to secondary care.

Conclusions Five diagnostic pulmonary embolism prediction models that are easily applicable in primary care were validated in this setting. Whereas efficiency was comparable for all rules, the Wells rules gave the best performance in terms of lower failure rates.


  • We thank AMUSE-2 project members R Oudega, H ten Cate, and M H Prins for their contribution to the design and initiation of the AMUSE 2 cohort. We thank Joris de Groot for his support with using the DeLong method and Peter Zuithoff and Karlijn Groenewegen for their statistical input.

  • Contributors: HEJHS, HCPMvW, HRB, AWH, and KGMM had the original idea for the study and were involved in writing the original study protocol. GJG, PMGE, and WAML were involved in data collection. JMTH and GJG drafted the first version of the manuscript, which was subsequently revised by the other authors. All authors participated in the final approval of the manuscript. JH, GJG, PMGE, and WAML had full access to all of the data in the study. JMTH and GJG are guarantors.

  • Funding: KGMM received a grant from The Netherlands Organization for Scientific Research (ZONMW 918.10.615 and 91208004). GJG is supported by a VENI grant from The Netherlands Organization for Scientific Research (ZONMW 016.166.030). All funding sources had no role in the design, conduct, analyses, or reporting of the study or in the decision to submit the manuscript for publication.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not required.

  • Transparency: The lead authors (the manuscript’s guarantors) affirm that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

  • Data sharing: Additional data are available on request from the corresponding author at j.m.t.hendriksen-9{at}

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

View Full Text