Categories: Mathematics, Stochastic analysis.

Markov process

Given a stochastic process \(\{X_t : t \ge 0\}\) on a filtered probability space \((\Omega, \mathcal{F}, \{\mathcal{F}_t\}, P)\), it is said to be a Markov process if it satisfies the following requirements:

  1. \(X_t\) is \(\mathcal{F}_t\)-adapted, meaning that the current and all past values of \(X_t\) can be reconstructed from the filtration \(\mathcal{F}_t\).
  2. For some function \(h(x)\), the conditional expectation \(\mathbf{E}[h(X_t) | \mathcal{F}_s] = \mathbf{E}[h(X_t) | X_s]\), i.e. at time \(s \le t\), the expectation of \(h(X_t)\) depends only on the current \(X_s\). Note that \(h\) must be bounded and Borel-measurable, meaning \(\sigma(h(X_t)) \subseteq \mathcal{F}_t\).

This last condition is called the Markov property, and demands that the future of \(X_t\) does not depend on the past, but only on the present \(X_s\).

If both \(t\) and \(X_t\) are taken to be discrete, then \(X_t\) is known as a Markov chain. This brings us to the concept of the transition probability \(P(X_t \in A | X_s = x)\), which describes the probability that \(X_t\) will be in a given set \(A\), if we know that currently \(X_s = x\).

If \(t\) and \(X_t\) are continuous, we can often (but not always) express \(P\) using a transition density \(p(s, x; t, y)\), which gives the probability density that the initial condition \(X_s = x\) will evolve into the terminal condition \(X_t = y\). Then the transition probability \(P\) can be calculated like so, where \(B\) is a given Borel set (see \(\sigma\)-algebra):

\[\begin{aligned} P(X_t \in B | X_s = x) = \int_B p(s, x; t, y) \dd{y} \end{aligned}\]

A prime examples of a continuous Markov process is the Wiener process. Note that this is also a martingale: often, a Markov process happens to be a martingale, or vice versa. However, those concepts are not to be confused: the Markov property does not specify what the expected future must be, and the martingale property says nothing about the history-dependence.


  1. U.H. Thygesen, Lecture notes on diffusions and stochastic differential equations, 2021, Polyteknisk Kompendie.

© Marcus R.A. Newman, a.k.a. "Prefetch". Available under CC BY-SA 4.0.