Markov models
Markov models represent a mathematic modelling method based on matrix calculus. A Markov model describes transitions of a cohort of patients between mutually exclusive and exhaustive health states in a series of subsequent cycles (i.e. time intervals between subsequent transitions from one state to another). This method makes it possible to model progression of a disease in time, which makes it especially useful for modelling of chronic diseases.
In a Markov model there is a finite number of possible states (e.g. Healthy, Diseased, Dead) with defined rules of transition between them. The important issue is that the chance of being in a specific state in a given cycle depends only on the state in which the patient was in the previous cycle and the rules of transition.
In Markov models the following aspects must be defined:
- possible states - reflecting the patient's fate; they must be exclusive (i.e. in a specific cycle the patient may be in one state only) and represent all possible events (e.g. the three basic states: Healthy, Diseased or Dead),
- cycle length - a model is a discrete representation of a continuous process; the shorter the cycle, the more realistic the model; however, short cycles are often impractical; for chronic diseases the assumed cycle length often equals one year,
- time horizon - a period of time, for which modelling is performed; for diseases affecting survival a lifetime horizon is usually applied,
- transition probabilities - based on available data the probability of transition from one state to another according to the rules of transition is calculated.
Utility depends on the time (the number of cycles) spent in a specific health state.
Types of Markov models
Two main types of Markov models are constructed:
- in cohort models the fate of a whole group of patients is modelled; probabilities changing with time and changing utilities are admissible; it is not possible to determine the distribution or variance of the results,
- in Monte Carlo simulation the fate of a single patient is modelled using a random number generator; this operation, repeated a sufficient number of times, should produce results similar to that of a cohort simulation; in Monte Carlo simulation not only the mean value, but also the variance of the results may be estimated. The time required for calculations is longer.