From 16555851b6514a736c5c9d8e73de7da7fc9b6288 Mon Sep 17 00:00:00 2001 From: Prefetch Date: Thu, 20 Oct 2022 18:25:31 +0200 Subject: Migrate from 'jekyll-katex' to 'kramdown-math-sskatex' --- source/know/concept/renyi-entropy/index.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) (limited to 'source/know/concept/renyi-entropy') diff --git a/source/know/concept/renyi-entropy/index.md b/source/know/concept/renyi-entropy/index.md index d53ccc0..55f233c 100644 --- a/source/know/concept/renyi-entropy/index.md +++ b/source/know/concept/renyi-entropy/index.md @@ -9,7 +9,7 @@ layout: "concept" In information theory, the **Rényi entropy** is a measure (or family of measures) of the "suprise" or "information" -contained in a random variable $X$. +contained in a random variable $$X$$. It is defined as follows: $$\begin{aligned} @@ -19,12 +19,12 @@ $$\begin{aligned} } \end{aligned}$$ -Where $\alpha \ge 0$ is a free parameter. +Where $$\alpha \ge 0$$ is a free parameter. The logarithm is usually base-2, but variations exist. -The case $\alpha = 0$ is known as the **Hartley entropy** or **max-entropy**, -and quantifies the "surprise" of an event from $X$, -if $X$ is uniformly distributed: +The case $$\alpha = 0$$ is known as the **Hartley entropy** or **max-entropy**, +and quantifies the "surprise" of an event from $$X$$, +if $$X$$ is uniformly distributed: $$\begin{aligned} \boxed{ @@ -33,9 +33,9 @@ $$\begin{aligned} } \end{aligned}$$ -Where $N$ is the cardinality of $X$; the number of different possible events. -The most famous case, however, is $\alpha = 1$. -Since $H_\alpha$ is problematic for $\alpha \to 1$, we must take the limit: +Where $$N$$ is the cardinality of $$X$$; the number of different possible events. +The most famous case, however, is $$\alpha = 1$$. +Since $$H_\alpha$$ is problematic for $$\alpha \to 1$$, we must take the limit: $$\begin{aligned} H_1(X) @@ -44,7 +44,7 @@ $$\begin{aligned} \end{aligned}$$ We then apply L'Hôpital's rule to evaluate this limit, -and use the fact that all $p_i$ sum to $1$: +and use the fact that all $$p_i$$ sum to $$1$$: $$\begin{aligned} H_1(X) @@ -64,9 +64,9 @@ $$\begin{aligned} } \end{aligned}$$ -Next, for $\alpha = 2$, we get the **collision entropy**, which describes +Next, for $$\alpha = 2$$, we get the **collision entropy**, which describes the surprise of two independent and identically distributed variables -$X$ and $Y$ yielding the same event: +$$X$$ and $$Y$$ yielding the same event: $$\begin{aligned} \boxed{ @@ -76,9 +76,9 @@ $$\begin{aligned} } \end{aligned}$$ -Finally, in the limit $\alpha \to \infty$, +Finally, in the limit $$\alpha \to \infty$$, the largest probability dominates the sum, -leading to the definition of the **min-entropy** $H_\infty$, +leading to the definition of the **min-entropy** $$H_\infty$$, describing the surprise of the most likely event: $$\begin{aligned} -- cgit v1.3