path: root/source/know/concept/detailed-balance/index.md

---
title: "Detailed balance"
date: 2021-11-27
categories:
- Physics
- Mathematics
- Stochastic analysis
layout: "concept"
---

Consider a system that can be regarded as a
[Markov process](/know/concept/markov-process/),
which means that its components (e.g. particles) are transitioning
between a known set of states,
with no history-dependence and no appreciable influence from interactions.

At equilibrium, the principle of **detailed balance** then says that
for all states, the rate of leaving that state is exactly equal to
the rate of entering it, for every possible transition.
In effect, such a system looks "frozen" to an outside observer,
since all net transition rates are zero.

We will focus on the case where both time and the state space are continuous.
Given some initial conditions,
assume that a component's trajectory can be described
as an [Itō diffusion](/know/concept/ito-calculus/) $X_t$
with a time-independent drift $f$ and intensity $g$,
and with a probability density $\phi(t, x)$ governed by the
[forward Kolmogorov equation](/know/concept/kolmogorov-equations/)
(in 3D):

$$\begin{aligned}
    \pdv{\phi}{t}
    = - \nabla \cdot \big( \vb{u} \phi - D \nabla \phi \big)
\end{aligned}$$

We start by demanding **stationarity**,
which is a weaker condition than detailed balance.
We want the probability $P$ of being in an arbitrary state volume $V$
to be constant in time:

$$\begin{aligned}
    0
    = \pdv{}{t}P(X_t \in V)
    = \pdv{}{t}\int_V \phi \dd{V}
    = \int_V \pdv{\phi}{t} \dd{V}
\end{aligned}$$

We substitute the forward Kolmogorov equation,
and apply the divergence theorem:

$$\begin{aligned}
    0
    = - \int_V \nabla \cdot \big( \vb{u} \phi - D \nabla \phi \big) \dd{V}
    = - \oint_{\partial V} \big( \vb{u} \phi - D \nabla \phi \big) \cdot \dd{\vb{S}}
\end{aligned}$$

In other words, the "flow" of probability *into* the volume $V$
is equal to the flow *out of* $V$.
If such a probability density exists,
it is called a **stationary distribution** $\phi(t, x) = \pi(x)$.
Because $V$ was arbitrary, $\pi$ can be found by solving:

$$\begin{aligned}
    0
    = - \nabla \cdot \big( \vb{u} \pi - D \nabla \pi \big)
\end{aligned}$$

Therefore, stationarity means that the state transition rates are constant.
To get detailed balance, however, we demand that
the transition rates are zero everywhere:
the probability flux through an arbitrary surface $S$ must vanish
(compare to closed surface integral above):

$$\begin{aligned}
    0
    = - \int_{S} \big( \vb{u} \phi - D \nabla \phi \big) \cdot \dd{\vb{S}}
\end{aligned}$$

And since $S$ is arbitrary, this is only satisfied if the flux is trivially zero
(the above justification can easily be repeated in 1D, 2D, 4D, etc.):

$$\begin{aligned}
    \boxed{
        0 = \vb{u} \phi - D \nabla \phi
    }
\end{aligned}$$

This is a stronger condition that stationarity,
but fortunately often satisfied in practice.

The fact that a system in detailed balance appears "frozen"
implies it is **time-reversible**,
meaning its statistics are the same for both directions of time.
Formally, given two arbitrary functions $h(x)$ and $k(x)$,
we have the property:

$$\begin{aligned}
    \boxed{
        \mathbf{E}\big[ h(X_0) \: k(X_t) \big]
        = \mathbf{E}\big[ h(X_t) \: k(X_0) \big]
    }
\end{aligned}$$

<div class="accordion">
<input type="checkbox" id="proof-reversibility"/>
<label for="proof-reversibility">Proof</label>
<div class="hidden" markdown="1">
<label for="proof-reversibility">Proof.</label>
Consider the following weighted inner product,
whose weight function is a stationary distribution $\pi$
satisfying detailed balance,
where $\hat{L}$ is the Kolmogorov operator:

$$\begin{aligned}
    \inprod{\hat{L} h}{k}_\pi
    \equiv \int_{-\infty}^\infty \hat{L}\{h(x)\} \: \pi(x) \: k(x) \dd{x}
    = \int_{-\infty}^\infty h(x) \: \hat{L}{}^\dagger\{\pi(x) k(x)\} \dd{x}
\end{aligned}$$

Where we have used the definition of an adjoint operator.
We would like to rewrite this:

$$\begin{aligned}
    \hat{L}{}^\dagger \{\pi k\}
    = -\nabla \cdot \big( \vb{u} \pi k - D \nabla(\pi k) \big)
    = -\nabla \cdot (\vb{u} \pi k - D k \nabla \pi - D \pi \nabla k)
\end{aligned}$$

Since $\pi$ is stationary by definition,
we know that $\nabla \cdot (\vb{u} \pi - D \nabla \pi) = 0$,
meaning:

$$\begin{aligned}
    \hat{L}{}^\dagger \{\pi k\}
    = \nabla \cdot (D \pi \nabla k)
    = \nabla \pi \cdot (D \nabla k) + \pi \nabla \cdot (D \nabla k)
\end{aligned}$$

Detailed balance demands that $\vb{u} \pi = D \nabla \pi$,
leading to the following:

$$\begin{aligned}
    \hat{L}{}^\dagger \{\pi k\}
    &= D \nabla \pi \cdot \nabla k + \pi \nabla \cdot (D \nabla k)
    = \pi \vb{u} \cdot \nabla k + \pi \nabla \cdot (D \nabla k)
    \\
    &= \pi \big( \vb{u} \cdot \nabla k + \nabla \cdot (D \nabla k) \big)
    = \pi \hat{L}\{k\}
\end{aligned}$$

Where we recognized the definition of $\hat{L}$
from the backward Kolmogorov equation.
Now that we have established that $\hat{L}{}^\dagger\{\pi k\} = \pi \hat{L}\{k\}$,
we return to the inner product:

$$\begin{aligned}
    \inprod{\hat{L} h}{k}_\pi
    = \int_{-\infty}^\infty h(x) \: \pi(x) \: \hat{L}\{k(x)\} \dd{x}
    = \inprod{h}{\hat{L} k}_\pi
\end{aligned}$$

Consequently, the following weighted inner products must also be equivalent:

$$\begin{aligned}
    \Inprod{\exp(t \hat{L}) h}{k}_\pi
    = \Inprod{h}{\exp(t \hat{L}) k}_\pi
\end{aligned}$$

Now, consider the time evolution of the
[conditional expectation](/know/concept/conditional-expectation/)
$\mathbf{E}\big[ k(X_t) | X_0 \big]$:

$$\begin{aligned}
    \pdv{}{t}\mathbf{E}\big[ k(X_t) | X_0 \big]
    &= \pdv{}{t}\int_{-\infty}^\infty k(x) \: \phi(t, x) \dd{x}
    = \int_{-\infty}^\infty k \pdv{\phi}{t} \dd{x}
    \\
    &= \int_{-\infty}^\infty k \: \hat{L}{}^\dagger\{\phi\} \dd{x}
    = \int_{-\infty}^\infty \hat{L}\{k\} \: \phi \dd{x}
    = \mathbf{E}\big[ \hat{L}\{k(X_t)\} | X_0 \big]
\end{aligned}$$

Where we used the forward Kolmogorov equation
and the definition of an adjoint operator.
Therefore, since the expectation $\mathbf{E}$
does not explicitly depend on $t$ (only implicitly via $X_t$),
we can naively move the differentiation inside
(only valid within $\mathbf{E}$):

$$\begin{aligned}
    \pdv{}{t}\mathbf{E}\big[ k(X_t) | X_0 \big]
    = \mathbf{E}\bigg[ \pdv{k(X_t)}{t} \bigg| X_0 \bigg]
    = \mathbf{E}\bigg[ \hat{L}\{k(X_0)\} \bigg| X_0 \bigg]
\end{aligned}$$

A differential equation of the form $\ipdv{k}{t} = \hat{L}\{k(t, x)\}$
for a time-independent operator $\hat{L}$
has a general solution $k(t, x) = \exp(t \hat{L})\{k(0,x)\}$,
therefore:

$$\begin{aligned}
    \mathbf{E}\big[ k(X_t) \big| X_0 \big]
    = \mathbf{E}\big[ \exp(t \hat{L})\{k(X_0)\} \big| X_0 \big]
    = \exp(t \hat{L})\{k(X_0)\}
\end{aligned}$$

With this, we can evaluate the two weighted inner products from earlier,
which we know are equal to each other.
Using the *tower property* of the conditional expectation:

$$\begin{aligned}
    \Inprod{h}{\exp(t \hat{L}) k}_\pi
    &= \mathbf{E}\big[ h(X_0) \: \mathbf{E}[k(X_t) | X_0] \big]
    = \mathbf{E}\big[ h(X_0) \: k(X_t) \big]
    \\
    = \Inprod{\exp(t \hat{L}) h}{k}_\pi
    &= \mathbf{E}\big[ \mathbf{E}[h(X_t) | X_0] \: k(X_0) \big]
    = \mathbf{E}\big[ h(X_t) \: k(X_0) \big]
\end{aligned}$$

Where the integral gave the expectation value at $X_0$,
since $\pi$ does not change in time.
</div>
</div>



## References
1.  U.H. Thygesen,
    *Lecture notes on diffusions and stochastic differential equations*,
    2021, Polyteknisk Kompendie.