summaryrefslogtreecommitdiff
path: root/content/know/concept/random-variable/index.pdc
diff options
context:
space:
mode:
authorPrefetch2021-10-23 19:16:13 +0200
committerPrefetch2021-10-23 19:16:13 +0200
commit98236a8eb89c09174971fcb28360cf1ea2b9a8e4 (patch)
tree7970285896576223a2cf81f74626cb6cfd2028e9 /content/know/concept/random-variable/index.pdc
parent78b0c780b1d2c3d31cc092d52e4dfc38928e7c6b (diff)
Expand knowledge base
Diffstat (limited to 'content/know/concept/random-variable/index.pdc')
-rw-r--r--content/know/concept/random-variable/index.pdc171
1 files changed, 171 insertions, 0 deletions
diff --git a/content/know/concept/random-variable/index.pdc b/content/know/concept/random-variable/index.pdc
new file mode 100644
index 0000000..fe50b60
--- /dev/null
+++ b/content/know/concept/random-variable/index.pdc
@@ -0,0 +1,171 @@
+---
+title: "Random variable"
+firstLetter: "R"
+publishDate: 2021-10-22
+categories:
+- Mathematics
+- Statistics
+
+date: 2021-10-21T20:40:42+02:00
+draft: false
+markup: pandoc
+---
+
+# Random variable
+
+**Random variables** are the bread and butter
+of probability theory and statistics,
+and are simply variables whose value depends
+on the outcome of a random experiment.
+Here, we will describe the formal mathematical definition
+of a random variable.
+
+
+## Probability space
+
+A **probability space** or **probability triple** $(\Omega, \mathcal{F}, P)$
+is the formal mathematical model of a given **stochastic experiment**,
+i.e. a process with a random outcome.
+
+The **sample space** $\Omega$ is the set
+of all possible outcomes $\omega$ of the experimement.
+Those $\omega$ are selected randomly according to certain criteria.
+A subset $A \subset \Omega$ is called an **event**,
+and can be regarded as a true statement about all $\omega$ in that $A$.
+
+The **event space** $\mathcal{F}$ is a set of events $A$
+that are interesting to us,
+i.e. we have subjectively chosen $\mathcal{F}$
+based on the problem at hand.
+Since events $A$ represent statements about outcomes $\omega$,
+and we would like to use logic on those statemenets,
+we demand that $\mathcal{F}$ is a [$\sigma$-algebra](/know/concept/sigma-algebra/).
+
+Finally, the **probability measure** or **probability function** $P$
+is a function that maps $A$ events to probabilities $P(A)$.
+Formally, $P : \mathcal{F} \to \mathbb{R}$ is defined to satisfy:
+
+1. If $A \in \mathcal{F}$, then $P(A) \in [0, 1]$.
+2. If $A, B \in \mathcal{F}$ do not overlap $A \cap B = \varnothing$,
+ then $P(A \cup B) = P(A) + P(B)$.
+3. The total probability $P(\Omega) = 1$.
+
+The reason we only assign probability to events $A$
+rather than individual outcomes $\omega$ is that
+if $\Omega$ is continuous, all $\omega$ have zero probability,
+while intervals $A$ can have nonzero probability.
+
+
+## Random variable
+
+Once we have a probability space $(\Omega, \mathcal{F}, P)$,
+we can define a **random variable** $X$
+as a function that maps outcomes $\omega$
+to another set, usually the real numbers.
+
+To be a valid real-valued random variable,
+a function $X : \Omega \to \mathbb{R}^n$ must satisfy the following condition,
+in which case $X$ is said to be **measurable**
+from $(\Omega, \mathcal{F})$ to $(\mathbb{R}^n, \mathcal{B}(\mathbb{R}^n))$:
+
+$$\begin{aligned}
+ \{ \omega \in \Omega : X(\omega) \in B \} \in \mathcal{F}
+ \quad \mathrm{for\:any\:} B \in \mathcal{B}(\mathbb{R}^n)
+\end{aligned}$$
+
+In other words, for a given Borel set (see $\sigma$-algebra) $B \in \mathcal{B}(\mathbb{R}^n)$,
+the set of all outcomes $\omega \in \Omega$ that satisfy $X(\omega) \in B$
+must form a valid event; this set must be in $\mathcal{F}$.
+The point is that we need to be able to assign probabilities
+to statements of the form $X \in [a, b]$ for all $a < b$,
+which is only possible if that statement corresponds to an event in $\mathcal{F}$,
+since $P$'s domain is $\mathcal{F}$.
+
+Given such an $X$, and a set $B \subseteq \mathbb{R}$,
+the **preimage** or **inverse image** $X^{-1}$ is defined as:
+
+$$\begin{aligned}
+ X^{-1}(B)
+ = \{ \omega \in \Omega : X(\omega) \in B \}
+\end{aligned}$$
+
+As suggested by the notation,
+$X^{-1}$ can be regarded as the inverse of $X$:
+it maps $B$ to the event for which $X \in B$.
+With this, our earlier requirement that $X$ be measurable
+can be written as: $X^{-1}(B) \in \mathcal{F}$ for any $B \in \mathcal{B}(\mathbb{R}^n)$.
+This is also often stated as *"$X$ is $\mathcal{F}$-measurable"*.
+
+Now, we are ready to define some familiar concepts from probability theory.
+The **cumulative distribution function** $F_X(x)$ is
+the probability of the event where the realized value of $X$
+is smaller than some given $x \in \mathbb{R}$:
+
+$$\begin{aligned}
+ F_X(x)
+ = P(X \le x)
+ = P(\{ \omega \in \Omega : X(\omega) \le x \})
+ = P(X^{-1}(]\!-\!\infty, x]))
+\end{aligned}$$
+
+If $F_X(x)$ is differentiable,
+then the **probability density function** $f_X(x)$ is defined as:
+
+$$\begin{aligned}
+ f_X(x)
+ = \dv{F_X}{x}
+\end{aligned}$$
+
+
+## Expectation value
+
+The **expectation value** $\mathbf{E}(X)$ of a random variable $X$
+can be defined in the familiar way, as the sum/integral
+of every possible value of $X$ mutliplied by the corresponding probability (density).
+For continuous and discrete sample spaces $\Omega$, respectively:
+
+$$\begin{aligned}
+ \mathbf{E}(X)
+ = \int_{-\infty}^\infty x \: f_X(x) \dd{x}
+ \qquad \mathrm{or} \qquad
+ \mathbf{E}(X)
+ = \sum_{i = 1}^N x_i \: P(X \!=\! x_i)
+\end{aligned}$$
+
+However, $f_X(x)$ is not guaranteed to exist,
+and the distinction between continuous and discrete is cumbersome.
+A more general definition of $\mathbf{E}(X)$
+is the following Lebesgue-Stieltjes integral,
+since $F_X(x)$ always exists:
+
+$$\begin{aligned}
+ \mathbf{E}(X)
+ = \int_{-\infty}^\infty x \dd{F_X(x)}
+\end{aligned}$$
+
+This is valid for any sample space $\Omega$.
+Or, equivalently, a Lebesgue integral can be used:
+
+$$\begin{aligned}
+ \mathbf{E}(X)
+ = \int_\Omega X(\omega) \dd{P(\omega)}
+\end{aligned}$$
+
+An expectation value defined in this way has many useful properties,
+most notably linearity.
+
+We can also define the familiar **variance** $\mathbf{V}(X)$
+of a random variable $X$ as follows:
+
+$$\begin{aligned}
+ \mathbf{V}(X)
+ = \mathbf{E}\big( (X - \mathbf{E}(X))^2 \big)
+ = \mathbf{E}(X^2) - \big(\mathbf{E}(X)\big)^2
+\end{aligned}$$
+
+
+
+## References
+1. U.F. Thygesen,
+ *Lecture notes on diffusions and stochastic differential equations*,
+ 2021, Polyteknisk Kompendie.