ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Sample Variance

Also Empirical Variance · Bessel's Correction · Mean Squared Deviation · Stichprobenvarianz · Empirische Varianz

The sample variance S² = (1/(N − 1)) Σ (y_k − M(y))² estimates the variance of N independent measurements without bias; dividing by N instead gives a biased but asymptotically unbiased estimator.

Know these firstSample Mean Variance Asymptotically Unbiased Estimator

Properties#

Unbiased: $\mathbb{E}\{S^2\} = \sigma_y^2$ (Unbiased Estimator).
The $1/N$ version is biased: the mean squared deviation $\hat\theta(y) = \frac1N\sum_k(y_k - M(y))^2 = \frac{N-1}{N}S^2$ has $\mathbb{E}\{\hat\theta\} = \frac{N-1}{N}\sigma_y^2$ . It is only asymptotically unbiased (derivation there). Since the bias is so easy to correct, the $1/N$ version is rarely used in practice.
Why $N - 1$ : the deviations are taken from $M(y)$ , which is computed from the same data and lies closer to the data than $\mu_y$ does. This makes the squared deviations systematically too small by the factor $\frac{N-1}{N}$ .

ProofUnbiasedness of

S^2

$S^2 = \frac{N}{N-1}\hat\theta(y)$ , so $\mathbb{E}\{S^2\} = \frac{N}{N-1}\cdot\frac{N-1}{N}\sigma_y^2 = \sigma_y^2$ .

Example#

Sources#

MSI script pp. 18–19 (§2.4), MSI Slides 02 > 2.4 Statistical Estimators (not published)

Connections

Mentioned in inline links5

Asymptotically Unbiased Estimator
Biased for every finite $N$ (too small by $\sigma_y^2/N$ ), but $\frac{N-1}{N} \to 1$ , so asymptotically unbiased. The bias is removed by the Sample Variance. §
Sample Mean
$M(y)$ depends on every $y_k$ , so it is not independent of the data. This is why the mean squared deviation $\frac1N\sum_k(y_k - M(y))^2$ underestimates the variance (Sample Variance). §
Unbiased Estimator
This is the Sample Mean. By contrast, averaging the squared deviations, $\frac1N\sum_k(y_k - M(y))^2$ , is a biased estimator of the variance; dividing by $N - 1$ instead gives the unbiased Sample Variance. §
Variance
Estimation from data: the Sample Variance $S^2$ estimates $\sigma^2$ without bias. The variance of the Sample Mean of $N$ independent measurements is $\sigma^2/N$ . §
Point Estimation
An estimator turns random measurements into an estimate of a fixed but unknown parameter, so the estimate is a random variable. Two questions judge it: is it right on average (unbiased, or at least asymptotically unbiased), and how much do… §
Sample Variance — the unbiased estimator of a variance, dividing by $N - 1$ §

Search in these mentions →