summaryrefslogtreecommitdiff
path: root/source/know/concept/renyi-entropy/index.md
diff options
context:
space:
mode:
Diffstat (limited to 'source/know/concept/renyi-entropy/index.md')
-rw-r--r--source/know/concept/renyi-entropy/index.md26
1 files changed, 13 insertions, 13 deletions
diff --git a/source/know/concept/renyi-entropy/index.md b/source/know/concept/renyi-entropy/index.md
index d53ccc0..55f233c 100644
--- a/source/know/concept/renyi-entropy/index.md
+++ b/source/know/concept/renyi-entropy/index.md
@@ -9,7 +9,7 @@ layout: "concept"
In information theory, the **Rényi entropy** is a measure
(or family of measures) of the "suprise" or "information"
-contained in a random variable $X$.
+contained in a random variable $$X$$.
It is defined as follows:
$$\begin{aligned}
@@ -19,12 +19,12 @@ $$\begin{aligned}
}
\end{aligned}$$
-Where $\alpha \ge 0$ is a free parameter.
+Where $$\alpha \ge 0$$ is a free parameter.
The logarithm is usually base-2, but variations exist.
-The case $\alpha = 0$ is known as the **Hartley entropy** or **max-entropy**,
-and quantifies the "surprise" of an event from $X$,
-if $X$ is uniformly distributed:
+The case $$\alpha = 0$$ is known as the **Hartley entropy** or **max-entropy**,
+and quantifies the "surprise" of an event from $$X$$,
+if $$X$$ is uniformly distributed:
$$\begin{aligned}
\boxed{
@@ -33,9 +33,9 @@ $$\begin{aligned}
}
\end{aligned}$$
-Where $N$ is the cardinality of $X$; the number of different possible events.
-The most famous case, however, is $\alpha = 1$.
-Since $H_\alpha$ is problematic for $\alpha \to 1$, we must take the limit:
+Where $$N$$ is the cardinality of $$X$$; the number of different possible events.
+The most famous case, however, is $$\alpha = 1$$.
+Since $$H_\alpha$$ is problematic for $$\alpha \to 1$$, we must take the limit:
$$\begin{aligned}
H_1(X)
@@ -44,7 +44,7 @@ $$\begin{aligned}
\end{aligned}$$
We then apply L'Hôpital's rule to evaluate this limit,
-and use the fact that all $p_i$ sum to $1$:
+and use the fact that all $$p_i$$ sum to $$1$$:
$$\begin{aligned}
H_1(X)
@@ -64,9 +64,9 @@ $$\begin{aligned}
}
\end{aligned}$$
-Next, for $\alpha = 2$, we get the **collision entropy**, which describes
+Next, for $$\alpha = 2$$, we get the **collision entropy**, which describes
the surprise of two independent and identically distributed variables
-$X$ and $Y$ yielding the same event:
+$$X$$ and $$Y$$ yielding the same event:
$$\begin{aligned}
\boxed{
@@ -76,9 +76,9 @@ $$\begin{aligned}
}
\end{aligned}$$
-Finally, in the limit $\alpha \to \infty$,
+Finally, in the limit $$\alpha \to \infty$$,
the largest probability dominates the sum,
-leading to the definition of the **min-entropy** $H_\infty$,
+leading to the definition of the **min-entropy** $$H_\infty$$,
describing the surprise of the most likely event:
$$\begin{aligned}