Categories: Mathematics, Stochastic analysis.

Markov process

Given a stochastic process $\{X_t : t \ge 0\}$ on a filtered probability space $(\Omega, \mathcal{F}, \{\mathcal{F}_t\}, P)$, it is said to be a Markov process if it satisfies the following requirements:

1. $X_t$ is $\mathcal{F}_t$-adapted, meaning that the current and all past values of $X_t$ can be reconstructed from the filtration $\mathcal{F}_t$.
2. For some function $h(x)$, the conditional expectation $\mathbf{E}[h(X_t) | \mathcal{F}_s] = \mathbf{E}[h(X_t) | X_s]$, i.e. at time $s \le t$, the expectation of $h(X_t)$ depends only on the current $X_s$. Note that $h$ must be bounded and Borel-measurable, meaning $\sigma(h(X_t)) \subseteq \mathcal{F}_t$.

This last condition is called the Markov property, and demands that the future of $X_t$ does not depend on the past, but only on the present $X_s$.

If both $t$ and $X_t$ are taken to be discrete, then $X_t$ is known as a Markov chain. This brings us to the concept of the transition probability $P(X_t \in A | X_s = x)$, which describes the probability that $X_t$ will be in a given set $A$, if we know that currently $X_s = x$.

If $t$ and $X_t$ are continuous, we can often (but not always) express $P$ using a transition density $p(s, x; t, y)$, which gives the probability density that the initial condition $X_s = x$ will evolve into the terminal condition $X_t = y$. Then the transition probability $P$ can be calculated like so, where $B$ is a given Borel set (see $\sigma$-algebra):

A prime examples of a continuous Markov process is the Wiener process. Note that this is also a martingale: often, a Markov process happens to be a martingale, or vice versa. However, those concepts are not to be confused: the Markov property does not specify what the expected future must be, and the martingale property says nothing about the history-dependence.

References

1. U.H. Thygesen, Lecture notes on diffusions and stochastic differential equations, 2021, Polyteknisk Kompendie.