ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Unbiased Estimator

Also Unbiasedness · Biased Estimator · Unbiased · Biased · Erwartungstreuer Schätzer · Erwartungstreue · Unverzerrter Schätzer

An estimator θ̂ is unbiased iff its expected value equals the true parameter, E{θ̂(y)} = θ_true; otherwise it is biased.

Properties#

Unbiasedness must hold for the given number $N$ of measurements, and for every value $\theta_\text{true}$ could take. It says nothing about the spread: an unbiased estimator can still have a large covariance.
Unbiased ⇒ asymptotically unbiased (the expectation is $\theta_\text{true}$ for every $N$ , hence also in the limit). The converse fails: see Asymptotically Unbiased Estimator.
Comparison: among unbiased estimators, the one with the smaller covariance (in the sense $\operatorname{cov}(\hat\theta^A) \succcurlyeq \operatorname{cov}(\hat\theta^B)$ ) is better.
Linearity helps: an estimator that is linear in the data inherits unbiasedness from the data via $\mathbb{E}\{\sum_k a_k y_k\} = \sum_k a_k\mathbb{E}\{y_k\}$ . Nonlinear estimators (ratios, squares) are often biased, since $\mathbb{E}\{f(y)\} \ne f(\mathbb{E}\{y\})$ (Expected Value).

MSI script pp. 17–18 (§2.4, Definition 1 and the sample mean example), MSI Slides 02 > 2.4 Statistical Estimators (not published)

Connections

Asymptotically Unbiased Estimator
Weaker than unbiased: every unbiased estimator is asymptotically unbiased. A biased estimator is asymptotically unbiased if its bias tends to zero as $N \to \infty$ . §
Estimator
no bias: Unbiased Estimator (for every $N$ ) or Asymptotically Unbiased Estimator (for $N \to \infty$ ); §
Sample Mean
Unbiased: $\mathbb{E}\{M(y)\} = \frac1N\sum_k\mathbb{E}\{y_k\} = \mu_y$ (Unbiased Estimator). §
Sample Variance
Unbiased: $\mathbb{E}\{S^2\} = \sigma_y^2$ (Unbiased Estimator). §
Point Estimation
An estimator turns random measurements into an estimate of a fixed but unknown parameter, so the estimate is a random variable. Two questions judge it: is it right on average (unbiased, or at least asymptotically unbiased), and how much do… §
Unbiased Estimator — right on average for every $N$ (Definition 1) §