Categories: Mathematics, Stochastic analysis.

Dynkin’s formula

Given an Itō diffusion $$X_t$$ with a time-independent drift $$f$$ and intensity $$g$$ such that the diffusion uniquely exists on the $$t$$-axis. We define the infinitesimal generator $$\hat{A}$$ as an operator with the following action on a given function $$h(x)$$, where $$\mathbf{E}$$ is a conditional expectation:

\begin{aligned} \boxed{ \hat{A}\{h(X_0)\} \equiv \lim_{t \to 0^+} \bigg[ \frac{1}{t} \mathbf{E}\Big[ h(X_t) - h(X_0) \Big| X_0 \Big] \bigg] } \end{aligned}

Which only makes sense for $$h$$ where this limit exists. The assumption that $$X_t$$ does not have any explicit time-dependence means that $$X_0$$ need not be the true initial condition; it can also be the state $$X_s$$ at any $$s$$ infinitesimally smaller than $$t$$.

Conveniently, for a sufficiently well-behaved $$h$$, the generator $$\hat{A}$$ is identical to the Kolmogorov operator $$\hat{L}$$ found in the backward Kolmogorov equation:

\begin{aligned} \boxed{ \hat{A}\{h(x)\} = \hat{L}\{h(x)\} } \end{aligned}

The general definition of resembles that of a classical derivative, and indeed, the generator $$\hat{A}$$ can be thought of as a differential operator. In that case, we would like an analogue of the classical fundamental theorem of calculus to relate it to integration.

Such an analogue is provided by Dynkin’s formula: for a stopping time $$\tau$$ with a finite expected value $$\mathbf{E}[\tau|X_0] < \infty$$, it states that:

\begin{aligned} \boxed{ \mathbf{E}\big[ h(X_\tau) | X_0 \big] = h(X_0) + \mathbf{E}\bigg[ \int_0^\tau \hat{L}\{h(X_t)\} \dd{t} \bigg| X_0 \bigg] } \end{aligned}

A common application of Dynkin’s formula is predicting when the stopping time $$\tau$$ occurs, and in what state $$X_\tau$$ this happens. Consider an example: for a region $$\Omega$$ of state space with $$X_0 \in \Omega$$, we define the exit time $$\tau \equiv \inf\{ t : X_t \notin \Omega \}$$, provided that $$\mathbf{E}[\tau | X_0] < \infty$$.

To get information about when and where $$X_t$$ exits $$\Omega$$, we define the general reward $$\Gamma$$ as follows, consisting of a running reward $$R$$ for $$X_t$$ inside $$\Omega$$, and a terminal reward $$T$$ on the boundary $$\partial \Omega$$ where we stop at $$X_\tau$$:

\begin{aligned} \Gamma = \int_0^\tau R(X_t) \dd{t} + \: T(X_\tau) \end{aligned}

For example, for $$R = 1$$ and $$T = 0$$, this becomes $$\Gamma = \tau$$, and if $$R = 0$$, then $$T(X_\tau)$$ can tell us the exit point. Let us now define $$h(X_0) = \mathbf{E}[\Gamma | X_0]$$, and apply Dynkin’s formula:

\begin{aligned} \mathbf{E}\big[ h(X_\tau) | X_0 \big] &= \mathbf{E}\big[ \Gamma \big| X_0 \big] + \mathbf{E}\bigg[ \int_0^\tau \hat{L}\{h(X_t)\} \dd{t} \bigg| X_0 \bigg] \\ &= \mathbf{E}\big[ T(X_\tau) | X_0 \big] + \mathbf{E}\bigg[ \int_0^\tau \hat{L}\{h(X_t)\} + R(X_t) \dd{t} \bigg| X_0 \bigg] \end{aligned}

The two leftmost terms depend on the exit point $$X_\tau$$, but not directly on $$X_t$$ for $$t < \tau$$, while the rightmost depends on the whole trajectory $$X_t$$. Therefore, the above formula is fulfilled if $$h(x)$$ satisfies the following equation and boundary conditions:

\begin{aligned} \boxed{ \begin{cases} \hat{L}\{h(x)\} + R(x) = 0 & \mathrm{for}\; x \in \Omega \\ h(x) = T(x) & \mathrm{for}\; x \notin \Omega \end{cases} } \end{aligned}

In other words, we have just turned a difficult question about a stochastic trajectory $$X_t$$ into a classical differential boundary value problem for $$h(x)$$.

1. U.H. Thygesen, Lecture notes on diffusions and stochastic differential equations, 2021, Polyteknisk Kompendie.

© Marcus R.A. Newman, a.k.a. "Prefetch". Available under CC BY-SA 4.0.