Intended for healthcare professionals

CCBYNC Open access
Research Methods & Reporting

Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

BMJ 2016; 352 doi: https://doi.org/10.1136/bmj.i6 (Published 25 January 2016) Cite this as: BMJ 2016;352:i6
  1. Andrew J Vickers, attending research methodologist1,
  2. Ben Van Calster, assistant professor2 3,
  3. Ewout W Steyerberg, professor3
  1. 1Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, 485 Lexington Avenue, New York, NY 10017, USA
  2. 2KU Leuven, Department of Development and Regeneration, Leuven, Belgium
  3. 3Department of Public Health, Erasmus MC, ‘s-Gravendijkwal, Rotterdam, Netherlands
  1. Correspondence to: A J Vickers vickersa{at}mskcc.org
  • Accepted 8 December 2015

Many decisions in medicine involve trade-offs, such as between diagnosing patients with disease versus unnecessary additional testing for those who are healthy. Net benefit is an increasingly reported decision analytic measure that puts benefits and harms on the same scale. This is achieved by specifying an exchange rate, a clinical judgment of the relative value of benefits (such as detecting a cancer) and harms (such as unnecessary biopsy) associated with models, markers, and tests. The exchange rate can be derived by asking simple questions, such as the maximum number of patients a doctor would recommend for biopsy to find one cancer. As the answers to these sorts of questions are subjective, it is possible to plot net benefit for a range of reasonable exchange rates in a “decision curve.” For clinical prediction models, the exchange rate is related to the probability threshold to determine whether a patient is classified as being positive or negative for a disease. Net benefit is useful for determining whether basing clinical decisions on a model, marker, or test would do more good than harm. This is in contrast to traditional measures such as sensitivity, specificity, or area under the curve, which are statistical abstractions not directly informative about clinical value. Recent years have seen an increase in practical applications of net benefit analysis to research data. This is a welcome development, since decision analytic techniques are of particular value when the purpose of a model, marker, or test is to help doctors make better clinical decisions.

Summary points

  • Prediction models, diagnostic tests, and molecular markers are traditionally evaluated using statistics such as sensitivity and specificity; such statistics do not tell us whether the model, test, or marker would do more good than harm if used in clinical practice

  • Decision analysis attempts to assess clinical value by incorporating …

View Full Text