From c0d352dd0f66b47ee91fb96eaf320f895fa78790 Mon Sep 17 00:00:00 2001 From: Prefetch Date: Sun, 14 Nov 2021 17:54:04 +0100 Subject: Expand knowledge base --- content/know/concept/markov-process/index.pdc | 66 +++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 content/know/concept/markov-process/index.pdc (limited to 'content/know/concept/markov-process/index.pdc') diff --git a/content/know/concept/markov-process/index.pdc b/content/know/concept/markov-process/index.pdc new file mode 100644 index 0000000..536aa00 --- /dev/null +++ b/content/know/concept/markov-process/index.pdc @@ -0,0 +1,66 @@ +--- +title: "Markov process" +firstLetter: "M" +publishDate: 2021-11-14 +categories: +- Mathematics + +date: 2021-11-13T21:05:21+01:00 +draft: false +markup: pandoc +--- + +# Markov process + +Given a [stochastic process](/know/concept/stochastic-process/) +$\{X_t : t \ge 0\}$ on a filtered probability space +$(\Omega, \mathcal{F}, \{\mathcal{F}_t\}, P)$, +it is said to be a **Markov process** +if it satisfies the following requirements: + +1. $X_t$ is $\mathcal{F}_t$-adapted, + meaning that the current and all past values of $X_t$ + can be reconstructed from the filtration $\mathcal{F}_t$. +2. For some function $h(x)$, + the [conditional expectation](/know/concept/conditional-expectation/) + $\mathbf{E}[h(X_t) | \mathcal{F}_s] = \mathbf{E}[h(X_t) | X_s]$, + i.e. at time $s \le t$, the expectation of $h(X_t)$ depends only on the current $X_s$. + Note that $h$ must be bounded and *Borel-measurable*, + meaning $\sigma(h(X_t)) \subseteq \mathcal{F}_t$. + +This last condition is called the **Markov property**, +and demands that the future of $X_t$ does not depend on the past, +but only on the present $X_s$. + +If both $t$ and $X_t$ are taken to be discrete, +then $X_t$ is known as a **Markov chain**. +This brings us to the concept of the **transition probability** +$P(X_t \in A | X_s = x)$, which describes the probability that +$X_t$ will be in a given set $A$, if we know that currently $X_s = x$. + +If $t$ and $X_t$ are continuous, we can often (but not always) express $P$ +using a **transition density** $p(s, x; t, y)$, +which gives the probability density that the initial condition $X_s = x$ +will evolve into the terminal condition $X_t = y$. +Then the transition probability $P$ can be calculated like so, +where $B$ is a given Borel set (see [$\sigma$-algebra](/know/concept/sigma-algebra/)): + +$$\begin{aligned} + P(X_t \in B | X_s = x) + = \int_B p(s, x; t, y) \dd{y} +\end{aligned}$$ + +A prime examples of a continuous Markov process is +the [Wiener process](/know/concept/wiener-process/). +Note that this is also a [martingale](/know/concept/martingale/): +often, a Markov process happens to be a martingale, or vice versa. +However, those concepts are not to be confused: +the Markov property does not specify *what* the expected future must be, +and the martingale property says nothing about the history-dependence. + + + +## References +1. U.H. Thygesen, + *Lecture notes on diffusions and stochastic differential equations*, + 2021, Polyteknisk Kompendie. -- cgit v1.2.3