diff options
Diffstat (limited to 'source/know/concept/central-limit-theorem')
| -rw-r--r-- | source/know/concept/central-limit-theorem/index.md | 62 |
1 files changed, 31 insertions, 31 deletions
diff --git a/source/know/concept/central-limit-theorem/index.md b/source/know/concept/central-limit-theorem/index.md index 0c08a6f..595cee7 100644 --- a/source/know/concept/central-limit-theorem/index.md +++ b/source/know/concept/central-limit-theorem/index.md @@ -10,16 +10,16 @@ layout: "concept" In statistics, the **central limit theorem** states that the sum of many independent variables tends towards a normal distribution, -even if the individual variables $x_n$ follow different distributions. +even if the individual variables $$x_n$$ follow different distributions. -For example, by taking $M$ samples of size $N$ from a population, -and calculating $M$ averages $\mu_m$ (which involves summing over $N$), -the resulting means $\mu_m$ are normally distributed -across the $M$ samples if $N$ is sufficiently large. +For example, by taking $$M$$ samples of size $$N$$ from a population, +and calculating $$M$$ averages $$\mu_m$$ (which involves summing over $$N$$), +the resulting means $$\mu_m$$ are normally distributed +across the $$M$$ samples if $$N$$ is sufficiently large. -More formally, for $N$ independent variables $x_n$ with probability distributions $p(x_n)$, +More formally, for $$N$$ independent variables $$x_n$$ with probability distributions $$p(x_n)$$, the central limit theorem states the following, -where we define the sum $S$: +where we define the sum $$S$$: $$\begin{aligned} S = \sum_{n = 1}^N x_n @@ -29,8 +29,8 @@ $$\begin{aligned} \sigma_S^2 = \sum_{n = 1}^N \sigma_n^2 \end{aligned}$$ -And crucially, it states that the probability distribution $p_N(S)$ of $S$ for $N$ variables -will become a normal distribution when $N$ goes to infinity: +And crucially, it states that the probability distribution $$p_N(S)$$ of $$S$$ for $$N$$ variables +will become a normal distribution when $$N$$ goes to infinity: $$\begin{aligned} \boxed{ @@ -41,14 +41,14 @@ $$\begin{aligned} We prove this below, but first we need to introduce some tools. -Given a probability density $p(x)$, its [Fourier transform](/know/concept/fourier-transform/) -is called the **characteristic function** $\phi(k)$: +Given a probability density $$p(x)$$, its [Fourier transform](/know/concept/fourier-transform/) +is called the **characteristic function** $$\phi(k)$$: $$\begin{aligned} \phi(k) = \int_{-\infty}^\infty p(x) \exp(i k x) \dd{x} \end{aligned}$$ -Note that $\phi(k)$ can be interpreted as the average of $\exp(i k x)$. +Note that $$\phi(k)$$ can be interpreted as the average of $$\exp(i k x)$$. We take its Taylor expansion in two separate ways, where an overline denotes the mean: @@ -67,7 +67,7 @@ $$\begin{aligned} \phi^{(n)}(0) = i^n \: \overline{x^n} \end{aligned}$$ -Next, the **cumulants** $C^{(n)}$ are defined from the Taylor expansion of $\ln\!\big(\phi(k)\big)$: +Next, the **cumulants** $$C^{(n)}$$ are defined from the Taylor expansion of $$\ln\!\big(\phi(k)\big)$$: $$\begin{aligned} \ln\!\big( \phi(k) \big) @@ -76,7 +76,7 @@ $$\begin{aligned} C^{(n)} = \frac{1}{i^n} \: \dvn{n}{}{k} \Big(\ln\!\big(\phi(k)\big)\Big) \Big|_{k = 0} \end{aligned}$$ -The first two cumulants $C^{(1)}$ and $C^{(2)}$ are of particular interest, +The first two cumulants $$C^{(1)}$$ and $$C^{(2)}$$ are of particular interest, since they turn out to be the mean and the variance respectively, using our earlier relation: @@ -92,14 +92,14 @@ $$\begin{aligned} = - \overline{x}^2 + \overline{x^2} = \sigma^2 \end{aligned}$$ -Let us now define $S$ as the sum of $N$ independent variables $x_n$, in other words: +Let us now define $$S$$ as the sum of $$N$$ independent variables $$x_n$$, in other words: $$\begin{aligned} S = \sum_{n = 1}^N x_n = x_1 + x_2 + ... + x_N \end{aligned}$$ -The probability density of $S$ is then as follows, where $p(x_n)$ are -the densities of all the individual variables and $\delta$ is +The probability density of $$S$$ is then as follows, where $$p(x_n)$$ are +the densities of all the individual variables and $$\delta$$ is the [Dirac delta function](/know/concept/dirac-delta-function/): $$\begin{aligned} @@ -109,9 +109,9 @@ $$\begin{aligned} &= \Big( p_1 * \big( p_2 * ( ... * (p_N * \delta))\big)\Big)(S) \end{aligned}$$ -In other words, the integrals pick out all combinations of $x_n$ which -add up to the desired $S$-value, and multiply the probabilities -$p(x_1) p(x_2) \cdots p(x_N)$ of each such case. This is a convolution, +In other words, the integrals pick out all combinations of $$x_n$$ which +add up to the desired $$S$$-value, and multiply the probabilities +$$p(x_1) p(x_2) \cdots p(x_N)$$ of each such case. This is a convolution, so the [convolution theorem](/know/concept/convolution-theorem/) states that it is a product in the Fourier domain: @@ -128,22 +128,22 @@ $$\begin{aligned} = \sum_{n = 1}^N \sum_{m = 1}^{\infty} \frac{(ik)^m}{m!} C_n^{(m)} \end{aligned}$$ -Consequently, the cumulants $C^{(m)}$ stack additively for the sum $S$ -of independent variables $x_m$, and therefore -the means $C^{(1)}$ and variances $C^{(2)}$ do too: +Consequently, the cumulants $$C^{(m)}$$ stack additively for the sum $$S$$ +of independent variables $$x_m$$, and therefore +the means $$C^{(1)}$$ and variances $$C^{(2)}$$ do too: $$\begin{aligned} C_S^{(m)} = \sum_{n = 1}^N C_n^{(m)} = C_1^{(m)} + C_2^{(m)} + ... + C_N^{(m)} \end{aligned}$$ -We now introduce the scaled sum $z$ as the new combined variable: +We now introduce the scaled sum $$z$$ as the new combined variable: $$\begin{aligned} z = \frac{S}{\sqrt{N}} = \frac{1}{\sqrt{N}} (x_1 + x_2 + ... + x_N) \end{aligned}$$ -Its characteristic function $\phi_z(k)$ is then as follows, -with $\sqrt{N}$ appearing in the arguments of $\phi_n$: +Its characteristic function $$\phi_z(k)$$ is then as follows, +with $$\sqrt{N}$$ appearing in the arguments of $$\phi_n$$: $$\begin{aligned} \phi_z(k) @@ -158,9 +158,9 @@ $$\begin{aligned} &= \prod_{n = 1}^N \phi_n\Big(\frac{k}{\sqrt{N}}\Big) \end{aligned}$$ -By expanding $\ln\!\big(\phi_z(k)\big)$ in terms of its cumulants $C^{(m)}$ -and introducing $\kappa = k / \sqrt{N}$, we see that the higher-order terms -become smaller for larger $N$: +By expanding $$\ln\!\big(\phi_z(k)\big)$$ in terms of its cumulants $$C^{(m)}$$ +and introducing $$\kappa = k / \sqrt{N}$$, we see that the higher-order terms +become smaller for larger $$N$$: $$\begin{gathered} \ln\!\big( \phi_z(k) \big) @@ -171,7 +171,7 @@ $$\begin{gathered} = \frac{1}{i^m N^{m/2}} \dvn{m}{}{\kappa} \sum_{n = 1}^N \ln\!\big( \phi_n(\kappa) \big) \end{gathered}$$ -For sufficiently large $N$, we can therefore approximate it using just the first two terms: +For sufficiently large $$N$$, we can therefore approximate it using just the first two terms: $$\begin{aligned} \ln\!\big( \phi_z(k) \big) @@ -182,7 +182,7 @@ $$\begin{aligned} &\approx \exp(i k \overline{z}) \exp(- k^2 \sigma_z^2 / 2) \end{aligned}$$ -We take its inverse Fourier transform to get the density $p(z)$, +We take its inverse Fourier transform to get the density $$p(z)$$, which turns out to be a Gaussian normal distribution, which is even already normalized: |