Categories: Mathematics, Stochastic analysis.

# Markov process

Given a stochastic process $$\{X_t : t \ge 0\}$$ on a filtered probability space $$(\Omega, \mathcal{F}, \{\mathcal{F}_t\}, P)$$, it is said to be a Markov process if it satisfies the following requirements:

1. $$X_t$$ is $$\mathcal{F}_t$$-adapted, meaning that the current and all past values of $$X_t$$ can be reconstructed from the filtration $$\mathcal{F}_t$$.
2. For some function $$h(x)$$, the conditional expectation $$\mathbf{E}[h(X_t) | \mathcal{F}_s] = \mathbf{E}[h(X_t) | X_s]$$, i.e. at time $$s \le t$$, the expectation of $$h(X_t)$$ depends only on the current $$X_s$$. Note that $$h$$ must be bounded and Borel-measurable, meaning $$\sigma(h(X_t)) \subseteq \mathcal{F}_t$$.

This last condition is called the Markov property, and demands that the future of $$X_t$$ does not depend on the past, but only on the present $$X_s$$.

If both $$t$$ and $$X_t$$ are taken to be discrete, then $$X_t$$ is known as a Markov chain. This brings us to the concept of the transition probability $$P(X_t \in A | X_s = x)$$, which describes the probability that $$X_t$$ will be in a given set $$A$$, if we know that currently $$X_s = x$$.

If $$t$$ and $$X_t$$ are continuous, we can often (but not always) express $$P$$ using a transition density $$p(s, x; t, y)$$, which gives the probability density that the initial condition $$X_s = x$$ will evolve into the terminal condition $$X_t = y$$. Then the transition probability $$P$$ can be calculated like so, where $$B$$ is a given Borel set (see $$\sigma$$-algebra):

\begin{aligned} P(X_t \in B | X_s = x) = \int_B p(s, x; t, y) \dd{y} \end{aligned}

A prime examples of a continuous Markov process is the Wiener process. Note that this is also a martingale: often, a Markov process happens to be a martingale, or vice versa. However, those concepts are not to be confused: the Markov property does not specify what the expected future must be, and the martingale property says nothing about the history-dependence.

1. U.H. Thygesen, Lecture notes on diffusions and stochastic differential equations, 2021, Polyteknisk Kompendie.

© Marcus R.A. Newman, a.k.a. "Prefetch". Available under CC BY-SA 4.0.