summaryrefslogtreecommitdiff
path: root/source/know/concept/central-limit-theorem
diff options
context:
space:
mode:
Diffstat (limited to 'source/know/concept/central-limit-theorem')
-rw-r--r--source/know/concept/central-limit-theorem/index.md62
1 files changed, 31 insertions, 31 deletions
diff --git a/source/know/concept/central-limit-theorem/index.md b/source/know/concept/central-limit-theorem/index.md
index 0c08a6f..595cee7 100644
--- a/source/know/concept/central-limit-theorem/index.md
+++ b/source/know/concept/central-limit-theorem/index.md
@@ -10,16 +10,16 @@ layout: "concept"
In statistics, the **central limit theorem** states that
the sum of many independent variables tends towards a normal distribution,
-even if the individual variables $x_n$ follow different distributions.
+even if the individual variables $$x_n$$ follow different distributions.
-For example, by taking $M$ samples of size $N$ from a population,
-and calculating $M$ averages $\mu_m$ (which involves summing over $N$),
-the resulting means $\mu_m$ are normally distributed
-across the $M$ samples if $N$ is sufficiently large.
+For example, by taking $$M$$ samples of size $$N$$ from a population,
+and calculating $$M$$ averages $$\mu_m$$ (which involves summing over $$N$$),
+the resulting means $$\mu_m$$ are normally distributed
+across the $$M$$ samples if $$N$$ is sufficiently large.
-More formally, for $N$ independent variables $x_n$ with probability distributions $p(x_n)$,
+More formally, for $$N$$ independent variables $$x_n$$ with probability distributions $$p(x_n)$$,
the central limit theorem states the following,
-where we define the sum $S$:
+where we define the sum $$S$$:
$$\begin{aligned}
S = \sum_{n = 1}^N x_n
@@ -29,8 +29,8 @@ $$\begin{aligned}
\sigma_S^2 = \sum_{n = 1}^N \sigma_n^2
\end{aligned}$$
-And crucially, it states that the probability distribution $p_N(S)$ of $S$ for $N$ variables
-will become a normal distribution when $N$ goes to infinity:
+And crucially, it states that the probability distribution $$p_N(S)$$ of $$S$$ for $$N$$ variables
+will become a normal distribution when $$N$$ goes to infinity:
$$\begin{aligned}
\boxed{
@@ -41,14 +41,14 @@ $$\begin{aligned}
We prove this below,
but first we need to introduce some tools.
-Given a probability density $p(x)$, its [Fourier transform](/know/concept/fourier-transform/)
-is called the **characteristic function** $\phi(k)$:
+Given a probability density $$p(x)$$, its [Fourier transform](/know/concept/fourier-transform/)
+is called the **characteristic function** $$\phi(k)$$:
$$\begin{aligned}
\phi(k) = \int_{-\infty}^\infty p(x) \exp(i k x) \dd{x}
\end{aligned}$$
-Note that $\phi(k)$ can be interpreted as the average of $\exp(i k x)$.
+Note that $$\phi(k)$$ can be interpreted as the average of $$\exp(i k x)$$.
We take its Taylor expansion in two separate ways,
where an overline denotes the mean:
@@ -67,7 +67,7 @@ $$\begin{aligned}
\phi^{(n)}(0) = i^n \: \overline{x^n}
\end{aligned}$$
-Next, the **cumulants** $C^{(n)}$ are defined from the Taylor expansion of $\ln\!\big(\phi(k)\big)$:
+Next, the **cumulants** $$C^{(n)}$$ are defined from the Taylor expansion of $$\ln\!\big(\phi(k)\big)$$:
$$\begin{aligned}
\ln\!\big( \phi(k) \big)
@@ -76,7 +76,7 @@ $$\begin{aligned}
C^{(n)} = \frac{1}{i^n} \: \dvn{n}{}{k} \Big(\ln\!\big(\phi(k)\big)\Big) \Big|_{k = 0}
\end{aligned}$$
-The first two cumulants $C^{(1)}$ and $C^{(2)}$ are of particular interest,
+The first two cumulants $$C^{(1)}$$ and $$C^{(2)}$$ are of particular interest,
since they turn out to be the mean and the variance respectively,
using our earlier relation:
@@ -92,14 +92,14 @@ $$\begin{aligned}
= - \overline{x}^2 + \overline{x^2} = \sigma^2
\end{aligned}$$
-Let us now define $S$ as the sum of $N$ independent variables $x_n$, in other words:
+Let us now define $$S$$ as the sum of $$N$$ independent variables $$x_n$$, in other words:
$$\begin{aligned}
S = \sum_{n = 1}^N x_n = x_1 + x_2 + ... + x_N
\end{aligned}$$
-The probability density of $S$ is then as follows, where $p(x_n)$ are
-the densities of all the individual variables and $\delta$ is
+The probability density of $$S$$ is then as follows, where $$p(x_n)$$ are
+the densities of all the individual variables and $$\delta$$ is
the [Dirac delta function](/know/concept/dirac-delta-function/):
$$\begin{aligned}
@@ -109,9 +109,9 @@ $$\begin{aligned}
&= \Big( p_1 * \big( p_2 * ( ... * (p_N * \delta))\big)\Big)(S)
\end{aligned}$$
-In other words, the integrals pick out all combinations of $x_n$ which
-add up to the desired $S$-value, and multiply the probabilities
-$p(x_1) p(x_2) \cdots p(x_N)$ of each such case. This is a convolution,
+In other words, the integrals pick out all combinations of $$x_n$$ which
+add up to the desired $$S$$-value, and multiply the probabilities
+$$p(x_1) p(x_2) \cdots p(x_N)$$ of each such case. This is a convolution,
so the [convolution theorem](/know/concept/convolution-theorem/)
states that it is a product in the Fourier domain:
@@ -128,22 +128,22 @@ $$\begin{aligned}
= \sum_{n = 1}^N \sum_{m = 1}^{\infty} \frac{(ik)^m}{m!} C_n^{(m)}
\end{aligned}$$
-Consequently, the cumulants $C^{(m)}$ stack additively for the sum $S$
-of independent variables $x_m$, and therefore
-the means $C^{(1)}$ and variances $C^{(2)}$ do too:
+Consequently, the cumulants $$C^{(m)}$$ stack additively for the sum $$S$$
+of independent variables $$x_m$$, and therefore
+the means $$C^{(1)}$$ and variances $$C^{(2)}$$ do too:
$$\begin{aligned}
C_S^{(m)} = \sum_{n = 1}^N C_n^{(m)} = C_1^{(m)} + C_2^{(m)} + ... + C_N^{(m)}
\end{aligned}$$
-We now introduce the scaled sum $z$ as the new combined variable:
+We now introduce the scaled sum $$z$$ as the new combined variable:
$$\begin{aligned}
z = \frac{S}{\sqrt{N}} = \frac{1}{\sqrt{N}} (x_1 + x_2 + ... + x_N)
\end{aligned}$$
-Its characteristic function $\phi_z(k)$ is then as follows,
-with $\sqrt{N}$ appearing in the arguments of $\phi_n$:
+Its characteristic function $$\phi_z(k)$$ is then as follows,
+with $$\sqrt{N}$$ appearing in the arguments of $$\phi_n$$:
$$\begin{aligned}
\phi_z(k)
@@ -158,9 +158,9 @@ $$\begin{aligned}
&= \prod_{n = 1}^N \phi_n\Big(\frac{k}{\sqrt{N}}\Big)
\end{aligned}$$
-By expanding $\ln\!\big(\phi_z(k)\big)$ in terms of its cumulants $C^{(m)}$
-and introducing $\kappa = k / \sqrt{N}$, we see that the higher-order terms
-become smaller for larger $N$:
+By expanding $$\ln\!\big(\phi_z(k)\big)$$ in terms of its cumulants $$C^{(m)}$$
+and introducing $$\kappa = k / \sqrt{N}$$, we see that the higher-order terms
+become smaller for larger $$N$$:
$$\begin{gathered}
\ln\!\big( \phi_z(k) \big)
@@ -171,7 +171,7 @@ $$\begin{gathered}
= \frac{1}{i^m N^{m/2}} \dvn{m}{}{\kappa} \sum_{n = 1}^N \ln\!\big( \phi_n(\kappa) \big)
\end{gathered}$$
-For sufficiently large $N$, we can therefore approximate it using just the first two terms:
+For sufficiently large $$N$$, we can therefore approximate it using just the first two terms:
$$\begin{aligned}
\ln\!\big( \phi_z(k) \big)
@@ -182,7 +182,7 @@ $$\begin{aligned}
&\approx \exp(i k \overline{z}) \exp(- k^2 \sigma_z^2 / 2)
\end{aligned}$$
-We take its inverse Fourier transform to get the density $p(z)$,
+We take its inverse Fourier transform to get the density $$p(z)$$,
which turns out to be a Gaussian normal distribution,
which is even already normalized: