ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Variance

Also Standard Deviation · Varianz · Standardabweichung · Streuung

The variance σ²_X = E{(X − μ_X)²} measures how much a random variable varies around its mean; its square root σ_X is the standard deviation.

Know these firstExpected Value

Formula#

\sigma_X^2 = \mathbb{E}\{X^2\} - (\mathbb{E}\{X\})^2 = \mathbb{E}\{X^2\} - \mu_X^2, \qquad \text{i.e. } \mathbb{E}\{X^2\} = \sigma_X^2 + \mu_X^2.

Proof

Expand the square and use linearity, with $\mathbb{E}\{X\}$ a constant:

\begin{aligned} \sigma_X^2 &= \mathbb{E}\{(X - \mathbb{E}\{X\})^2\} = \mathbb{E}\{X^2 - 2X\mathbb{E}\{X\} + (\mathbb{E}\{X\})^2\} \\ &= \mathbb{E}\{X^2\} - 2\mathbb{E}\{X\}\mathbb{E}\{X\} + (\mathbb{E}\{X\})^2 = \mathbb{E}\{X^2\} - (\mathbb{E}\{X\})^2. \end{aligned}

Properties#

$\sigma_X^2 \ge 0$ , with $\sigma_X^2 = 0$ iff $X$ takes the value $\mu_X$ with probability one.
Affine maps: $\operatorname{Var}(a + bX) = b^2\sigma_X^2$ . Shifting does not change the spread; scaling by $b$ scales $\sigma_X$ by $|b|$ .
Units: $\sigma_X$ has the unit of $X$ , $\sigma_X^2$ its square.
Generalizations: the variance is the Covariance of a variable with itself, $\sigma(X, X) = \sigma_X^2$ , and the diagonal of the Covariance Matrix.
Estimation from data: the Sample Variance $S^2$ estimates $\sigma^2$ without bias. The variance of the Sample Mean of $N$ independent measurements is $\sigma^2/N$ .
Noise of a measurement: in MSI, the noise on the current and the voltage has variances $\sigma_i^2$ and $\sigma_u^2$ , and $\frac1N\sum_k n_{i,k}^2 \to \sigma_i^2$ (Law of Large Numbers).

Example#

Sources#

MSI script pp. 14–15 (§2.2.1, definition and variance example), MSI Slides 02 > 2.2.1 Mean and Variance (not published)

Connections

Prerequisites know first1

Expected Value

Mentioned in inline links9

VL02 – Probability and Statistics in a NutshellMSI
The expectation operator $\mathbb{E}\{f(X)\} = \int f(x)\,p_X(x)\,\mathrm{d}x$ defines the mean (Expected Value) and the Variance $\sigma_X^2 = \mathbb{E}\{(X-\mu_X)^2\}$ . The worked example shows $\sigma_X^2 = \mathbb{E}\{X^2\} - \mu_X^2$ … §
Variance: spread around the mean, $\sigma^2 = \mathbb{E}\{X^2\} - \mu^2$ §
Correlation
Correlation is covariance with the scale removed. Multiplying $Z$ by $b > 0$ (e.g. switching from meters to centimeters) multiplies every offset product $(y - \mu_Y)(z - \mu_Z)$ , and with it $\sigma(Y, Z)$ , by $b$ (top row). The standard d… §
Covariance Matrix
Its entries are $(\Sigma_X)_{jk} = \sigma(X_j, X_k)$ (Covariance); the diagonal holds the variances $\sigma_{X_j}^2$ . §
Covariance
(expand the product and use linearity, as for the Variance). §
Expected Value
Not for nonlinear functions: in general $\mathbb{E}\{f(X)\} \ne f(\mathbb{E}\{X\})$ . Example: $\mathbb{E}\{X^2\} = (\mathbb{E}\{X\})^2 + \sigma_X^2$ (Variance). §
Higher moments: Variance, Covariance and the Covariance Matrix are all expectations of products of centred variables $X - \mu_X$ . §
Law of Large Numbers
Take $y_k = n_{i,k}^2$ . Then $\mathbb{E}\{y_k\} = \mathbb{E}\{n_{i,k}^2\} = \sigma_i^2 + 0^2 = \sigma_i^2$ (Variance), so $\frac1N\sum_k n_{i,k}^2 \to \sigma_i^2$ . This term is why the denominator of the Least Squares Estimator converges t… §
Normal Distribution
Parameters are the moments: the mean is $\mu_X = \mu$ and the variance is $\sigma_X^2 = \sigma^2$ . So the distribution is completely determined by its mean and variance. §
Uniform Distribution
Variance: $\sigma_X^2 = \int_a^b \frac{1}{b-a}(x - \mu_X)^2\,\mathrm{d}x$ . Substitute $y = x - \mu_X$ and write $c = b - a$ , so $y$ runs over $[-c/2, c/2]$ : §
Moments of Random Variables
Instead of a whole density, one usually works with a few numbers: the mean (where the variable lies), the variance (how far it spreads), the covariance and correlation (how two variables move together), and for random vectors the covarianc… §
Variance — the spread of one variable §

Search in these mentions →