General Practice

# Commentary: Markov models of medical prognosis

BMJ 1997; 314 (Published 01 February 1997) Cite this as: BMJ 1997;314:354
1. Andrew Briggs, research officera,
2. Mark Sculpher, research fellowb
1. a Health Economics Research Centre, Institute of Health Sciences, Wolfson College, Oxford OX2 6UD
2. b Health Economics Research Group, Brunel University, Middlesex UB8 3PH

## Commentary

Named after a turn of the century Russian mathematician, Markov models are used to describe stochastic processes–that is, random processes that evolve over time. Increasingly, such models are being applied in medical and health services research.1 Hogan et al use a Markov model to address an epidemiological question; such methods have also been used in clinical2 and economic evaluations.3 4

The most common application of Markov models in health is to characterise the possible prognoses experienced by a given group of patients. This entails modelling the progress over time of a notional group of patients through a finite number of health states. Patients are initially placed in one of the health states, and the probabilities of transition to the other states in the model are defined within a given time period, known as a Markov cycle.

Alternative forms of Markov models exist, differing in how transition probabilities are defined. Markov processes describe a model in which the state probabilities are allowed to vary as the number of Markov cycles increases. This is particularly appropriate for modelling life expectancy, in which the risk of death in each period clearly increases with age. A Markov chain, as used by Hogan et al, is when the transition probabilities of the model are assumed to be constant over time. An intuitive presentation of the model is given in figure 1 of their paper, and these state transition diagrams are commonly used to illustrate Markov models. The disease states of the model are represented by the faces and the arrows between the circles give the possible transitions.

Markov models can be analysed in several ways. One of the most common is known as cohort simulation. A hypothetical cohort of patients begins the model in any of the disease states and the cohort is then tracked for the duration of the model. The proportion of the cohort in any of the states at any point in time and the mean duration in each state can be calculated. An alternative approach, adopted by Hogan et al, is Monte Carlo simulation. Many hypothetical patients are passed individually through the model and their disease pathways recorded. The advantage of the Monte Carlo approach is that estimates of the duration dispersion can be obtained from the individual simulation data.

As with all models, Markov models are simplifications of the real world. An important limitation of Markov models is that they lack a memory. This is termed the Markovian assumption–that is, the probability of moving from one state to another is independent of the history of the patient before arriving in that state. This assumption is pertinent to the analysis presented by Hogan et al as they argue that the discrepancy between the predictions of their model and the observed data, in terms of the dispersion of the positive test results for otitis media with dispersion given in figure 2 can be explained by the existence of two groups of children characterised by high or low susceptibility to the disease. An alternative explanation might be that a history of acute events in otitis media with effusion affects the probability that such events recur, which the Markov model does not allow for. The extent to which the Markovian assumption is limiting in this case is a matter of clinical opinion, on which we are not qualified to judge. Although the limitations of the Markovian assumption should be kept in mind, the adept modeller will often find ways around the assumption using a combination of time dependent transition probabilities and distinct states for patients with different medical histories.

## Acknowledgments

Funding: None.

Conflict of interest: None.

## References

1. 1.
2. 2.
3. 3.
4. 4.
View Abstract