Categories: Mathematics, Physics, Stochastic analysis.

Detailed balance

Consider a system that can be regarded as a Markov process, which means that its components (e.g. particles) are transitioning between a known set of states, with no history-dependence and no appreciable influence from interactions.

At equilibrium, the principle of detailed balance then says that for all states, the rate of leaving that state is exactly equal to the rate of entering it, for every possible transition. In effect, such a system looks “frozen” to an outside observer, since all net transition rates are zero.

We will focus on the case where both time and the state space are continuous. Given some initial conditions, assume that a component’s trajectory can be described as an Itō diffusion XtX_t with a time-independent drift ff and intensity gg, and with a probability density ϕ(t,x)\phi(t, x) governed by the forward Kolmogorov equation (in 3D):

ϕt=(uϕDϕ)\begin{aligned} \pdv{\phi}{t} = - \nabla \cdot \big( \vb{u} \phi - D \nabla \phi \big) \end{aligned}

We start by demanding stationarity, which is a weaker condition than detailed balance. We want the probability PP of being in an arbitrary state volume VV to be constant in time:

0=tP(XtV)=tVϕdV=VϕtdV\begin{aligned} 0 = \pdv{}{t}P(X_t \in V) = \pdv{}{t}\int_V \phi \dd{V} = \int_V \pdv{\phi}{t} \dd{V} \end{aligned}

We substitute the forward Kolmogorov equation, and apply the divergence theorem:

0=V(uϕDϕ)dV=V(uϕDϕ)dS\begin{aligned} 0 = - \int_V \nabla \cdot \big( \vb{u} \phi - D \nabla \phi \big) \dd{V} = - \oint_{\partial V} \big( \vb{u} \phi - D \nabla \phi \big) \cdot \dd{\vb{S}} \end{aligned}

In other words, the “flow” of probability into the volume VV is equal to the flow out of VV. If such a probability density exists, it is called a stationary distribution ϕ(t,x)=π(x)\phi(t, x) = \pi(x). Because VV was arbitrary, π\pi can be found by solving:

0=(uπDπ)\begin{aligned} 0 = - \nabla \cdot \big( \vb{u} \pi - D \nabla \pi \big) \end{aligned}

Therefore, stationarity means that the state transition rates are constant. To get detailed balance, however, we demand that the transition rates are zero everywhere: the probability flux through an arbitrary surface SS must vanish (compare to closed surface integral above):

0=S(uϕDϕ)dS\begin{aligned} 0 = - \int_{S} \big( \vb{u} \phi - D \nabla \phi \big) \cdot \dd{\vb{S}} \end{aligned}

And since SS is arbitrary, this is only satisfied if the flux is trivially zero (the above justification can easily be repeated in 1D, 2D, 4D, etc.):

0=uϕDϕ\begin{aligned} \boxed{ 0 = \vb{u} \phi - D \nabla \phi } \end{aligned}

This is a stronger condition that stationarity, but fortunately often satisfied in practice.

The fact that a system in detailed balance appears “frozen” implies it is time-reversible, meaning its statistics are the same for both directions of time. Formally, given two arbitrary functions h(x)h(x) and k(x)k(x), we have the property:

E[h(X0)k(Xt)]=E[h(Xt)k(X0)]\begin{aligned} \boxed{ \mathbf{E}\big[ h(X_0) \: k(X_t) \big] = \mathbf{E}\big[ h(X_t) \: k(X_0) \big] } \end{aligned}

Consider the following weighted inner product, whose weight function is a stationary distribution π\pi satisfying detailed balance, where L^\hat{L} is the Kolmogorov operator:

L^hkπL^{h(x)}π(x)k(x)dx=h(x)L^{π(x)k(x)}dx\begin{aligned} \inprod{\hat{L} h}{k}_\pi \equiv \int_{-\infty}^\infty \hat{L}\{h(x)\} \: \pi(x) \: k(x) \dd{x} = \int_{-\infty}^\infty h(x) \: \hat{L}{}^\dagger\{\pi(x) k(x)\} \dd{x} \end{aligned}

Where we have used the definition of an adjoint operator. We would like to rewrite this:

L^{πk}=(uπkD(πk))=(uπkDkπDπk)\begin{aligned} \hat{L}{}^\dagger \{\pi k\} = -\nabla \cdot \big( \vb{u} \pi k - D \nabla(\pi k) \big) = -\nabla \cdot (\vb{u} \pi k - D k \nabla \pi - D \pi \nabla k) \end{aligned}

Since π\pi is stationary by definition, we know that (uπDπ)=0\nabla \cdot (\vb{u} \pi - D \nabla \pi) = 0, meaning:

L^{πk}=(Dπk)=π(Dk)+π(Dk)\begin{aligned} \hat{L}{}^\dagger \{\pi k\} = \nabla \cdot (D \pi \nabla k) = \nabla \pi \cdot (D \nabla k) + \pi \nabla \cdot (D \nabla k) \end{aligned}

Detailed balance demands that uπ=Dπ\vb{u} \pi = D \nabla \pi, leading to the following:

L^{πk}=Dπk+π(Dk)=πuk+π(Dk)=π(uk+(Dk))=πL^{k}\begin{aligned} \hat{L}{}^\dagger \{\pi k\} &= D \nabla \pi \cdot \nabla k + \pi \nabla \cdot (D \nabla k) = \pi \vb{u} \cdot \nabla k + \pi \nabla \cdot (D \nabla k) \\ &= \pi \big( \vb{u} \cdot \nabla k + \nabla \cdot (D \nabla k) \big) = \pi \hat{L}\{k\} \end{aligned}

Where we recognized the definition of L^\hat{L} from the backward Kolmogorov equation. Now that we have established that L^{πk}=πL^{k}\hat{L}{}^\dagger\{\pi k\} = \pi \hat{L}\{k\}, we return to the inner product:

L^hkπ=h(x)π(x)L^{k(x)}dx=hL^kπ\begin{aligned} \inprod{\hat{L} h}{k}_\pi = \int_{-\infty}^\infty h(x) \: \pi(x) \: \hat{L}\{k(x)\} \dd{x} = \inprod{h}{\hat{L} k}_\pi \end{aligned}

Consequently, the following weighted inner products must also be equivalent:

exp(tL^)h|kπ=h|exp(tL^)kπ\begin{aligned} \Inprod{\exp(t \hat{L}) h}{k}_\pi = \Inprod{h}{\exp(t \hat{L}) k}_\pi \end{aligned}

Now, consider the time evolution of the conditional expectation E[k(Xt)X0]\mathbf{E}\big[ k(X_t) | X_0 \big]:

tE[k(Xt)X0]=tk(x)ϕ(t,x)dx=kϕtdx=kL^{ϕ}dx=L^{k}ϕdx=E[L^{k(Xt)}X0]\begin{aligned} \pdv{}{t}\mathbf{E}\big[ k(X_t) | X_0 \big] &= \pdv{}{t}\int_{-\infty}^\infty k(x) \: \phi(t, x) \dd{x} = \int_{-\infty}^\infty k \pdv{\phi}{t} \dd{x} \\ &= \int_{-\infty}^\infty k \: \hat{L}{}^\dagger\{\phi\} \dd{x} = \int_{-\infty}^\infty \hat{L}\{k\} \: \phi \dd{x} = \mathbf{E}\big[ \hat{L}\{k(X_t)\} | X_0 \big] \end{aligned}

Where we used the forward Kolmogorov equation and the definition of an adjoint operator. Therefore, since the expectation E\mathbf{E} does not explicitly depend on tt (only implicitly via XtX_t), we can naively move the differentiation inside (only valid within E\mathbf{E}):

tE[k(Xt)X0]=E[k(Xt)tX0]=E[L^{k(X0)}X0]\begin{aligned} \pdv{}{t}\mathbf{E}\big[ k(X_t) | X_0 \big] = \mathbf{E}\bigg[ \pdv{k(X_t)}{t} \bigg| X_0 \bigg] = \mathbf{E}\bigg[ \hat{L}\{k(X_0)\} \bigg| X_0 \bigg] \end{aligned}

A differential equation of the form k/t=L^{k(t,x)}\ipdv{k}{t} = \hat{L}\{k(t, x)\} for a time-independent operator L^\hat{L} has a general solution k(t,x)=exp(tL^){k(0,x)}k(t, x) = \exp(t \hat{L})\{k(0,x)\}, therefore:

E[k(Xt)X0]=E[exp(tL^){k(X0)}X0]=exp(tL^){k(X0)}\begin{aligned} \mathbf{E}\big[ k(X_t) \big| X_0 \big] = \mathbf{E}\big[ \exp(t \hat{L})\{k(X_0)\} \big| X_0 \big] = \exp(t \hat{L})\{k(X_0)\} \end{aligned}

With this, we can evaluate the two weighted inner products from earlier, which we know are equal to each other. Using the tower property of the conditional expectation:

h|exp(tL^)kπ=E[h(X0)E[k(Xt)X0]]=E[h(X0)k(Xt)]=exp(tL^)h|kπ=E[E[h(Xt)X0]k(X0)]=E[h(Xt)k(X0)]\begin{aligned} \Inprod{h}{\exp(t \hat{L}) k}_\pi &= \mathbf{E}\big[ h(X_0) \: \mathbf{E}[k(X_t) | X_0] \big] = \mathbf{E}\big[ h(X_0) \: k(X_t) \big] \\ = \Inprod{\exp(t \hat{L}) h}{k}_\pi &= \mathbf{E}\big[ \mathbf{E}[h(X_t) | X_0] \: k(X_0) \big] = \mathbf{E}\big[ h(X_t) \: k(X_0) \big] \end{aligned}

Where the integral gave the expectation value at X0X_0, since π\pi does not change in time.


  1. U.H. Thygesen, Lecture notes on diffusions and stochastic differential equations, 2021, Polyteknisk Kompendie.