Prognosis and prognostic research: validating a prognostic model

doi:10.1136/bmj.b605

From trainee to consultant, BMJ Group offers doctors around the world tailored information, special events, learning resources and recruitment services at every step along their career path.

... by doctors, for doctors, for patients
We are open for entries!

Online learning

BMJ Learning

High-quality CME / CPD for doctors and other healthcare professionals. BMJ Learning features hundreds of accredited, peer-reviewed learning modules in text, video, and audio formats. Find out more

Courses and Qualifications

BMJ Masterclasses

BMJ Masterclasses, led by experts, help clinicians to use the latest evidence and recent guidelines in practice and meet their CPD/CME requirements. Find out more

Exam Preparation

The leading provider of online exam preparation, helping over 167,000 healthcare professionals to pass their exams. Find out more

onExamination

Search all BMJ research articles:

From

Limit by

Our online table of contents is updated at least twice each day. Read all articles published in the last 7 days.
You can use bmj.com to help you with your continuing medical education. Find out about CME/CPD credits for BMJ articles
Keep up to date with cardiology: Access the latest cardiovascular medicine resources from across BMJ Group.
OPEN ACCESS: All research articles are freely available online, with no word limit. Find out more about the BMJ's open access policy. Submit your paper.
Find out how study types differ in our How to read a paper section.
Countdown to London 2012: BMJ Group's Olympics portal highlights latest Olympics and sports medicine-themed research, comment and learning

Access to the full text of this article requires a subscription or payment. Please log in or subscribe below.

Douglas G Altman, professor of statistics in medicine1,
Yvonne Vergouwe, assistant professor of clinical epidemiology2,
Patrick Royston, senior statistician3,
Karel G M Moons, professor of clinical epidemiology2

¹Centre for Statistics in Medicine, University of Oxford, Oxford OX2 6UD
²Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, Netherlands
³MRC Clinical Trials Unit, London NW1 2DA

Correspondence to: D G Altman doug.altman{at}csm.ox.ac.uk

Accepted 6 October 2008

Prognostic models are of little clinical value unless they are shown to work in other samples. Douglas Altman and colleagues describe how to validate models and discuss some of the problems

Prognostic models, like the one we developed in the previous article in this series,1 yield scores to enable the prediction of the risk of future events in individual patients or groups and the stratification of patients by these risks.2 A good model may allow the reasonably reliable classification of patients into risk groups with different prognoses. To show that a prognostic model is valuable, however, it is not sufficient to show that it successfully predicts outcome in the initial development data. We need evidence that the model performs well for other groups of patients.1 3 In this article, we discuss how to evaluate the performance of a prognostic model in new data.4 5

Summary points

Unvalidated models should not be used in clinical practice
When validating a prognostic model, calibration and discrimination should be evaluated
Validation should be done on a different data from that used to develop the model, preferably from patients in other centres
Models may not perform well in practice because of deficiencies in the development methods or because the new sample is too different from the original

Why prognostic models may not predict well

Various statistical or clinical factors may lead a prognostic model to perform poorly when applied to other patients.4 6 The model’s predictions may not be reproducible because of deficiencies in the design or modelling methods used in the study to derive the model, if the model was overfitted, or if an important predictor is absent from the model (which may be hard to know).1 Poor performance in new patients can also arise from differences between the setting of patients in the new and derivation …