ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Asymptotically Unbiased Estimator

Also Asymptotic Unbiasedness · Asymptotically Unbiased · Asymptotisch erwartungstreuer Schätzer · Asymptotische Erwartungstreue

An estimator θ̂(y₁, …, y_N) is asymptotically unbiased iff its expected value tends to the true parameter as the number of measurements grows, lim_{N→∞} E{θ̂(y₁, …, y_N)} = θ_true.

Know these firstUnbiased Estimator

Properties#

Weaker than unbiased: every unbiased estimator is asymptotically unbiased. A biased estimator is asymptotically unbiased if its bias tends to zero as $N \to \infty$ .
Towards consistency: asymptotically unbiased and covariance $\to 0$ as $N \to \infty$ ⇒ consistent.
Easiest property to analyse: for estimators built from averages, the limits can be evaluated with the Law of Large Numbers. This is how MSI decides between the three resistance estimators.

Example#

ExampleVariance by the mean squared deviations (MSI script)

Estimate $\theta \equiv \sigma_y^2$ from $N$ independent measurements via

\hat\theta(y) = \frac1N\sum_{k=1}^N (y_k - M(y))^2, \qquad M(y) = \frac1N\sum_{k=1}^N y_k.

Because $M(y)$ depends on the same data, $\mathbb{E}\{\hat\theta\} = \mathbb{E}\{y_k^2\} - 2\mathbb{E}\{y_k M(y)\} + \mathbb{E}\{M(y)^2\}$ . With $\mathbb{E}\{y_k^2\} = \sigma_y^2 + \mu_y^2$ and $\mathbb{E}\{y_k M(y)\} = \mathbb{E}\{M(y)^2\} = \frac{\sigma_y^2}{N} + \mu_y^2$ (Sample Mean > Properties):

\mathbb{E}\{\hat\theta\} = \mu_y^2 + \sigma_y^2 - 2\left(\frac{\sigma_y^2}{N} + \mu_y^2\right) + \frac{\sigma_y^2}{N} + \mu_y^2 = \frac{N-1}{N}\sigma_y^2.

Biased for every finite $N$ (too small by $\sigma_y^2/N$ ), but $\frac{N-1}{N} \to 1$ , so asymptotically unbiased. The bias is removed by the Sample Variance.

In the resistance example, the Error-in-Variables Estimator is asymptotically unbiased, while the Least Squares Estimator and the Simple Approach Estimator are not.

Sources#

MSI script pp. 18–19 (§2.4, Definition 2 and example), MSI Slides 02 > 2.4 Statistical Estimators (not published)
MSI script pp. 19–21 (§2.5), MSI Slides 02 > 2.5 Analysis of the Resistance Estimation Example (not published)

Connections

Mentioned in inline links7

Least Squares EstimatorMSI
The numerator converges to $u_\text{true} i_\text{true}$ (the cross terms vanish), but the denominator picks up the variance $\sigma_i^2$ of the current noise (Law of Large Numbers). The estimator is not asymptotically unbiased and always… §
Resistance Estimation ExampleMSI
Estimate the resistance $R$ from $N$ noisy measurements $(u_k, i_k)$ . Three natural estimators, averaging the ratios, dividing the averages and least squares, all converge as $N$ grows, but to three different values (script Fig. 1.4). The… §
Consistent Estimator
it is asymptotically unbiased, and §
Estimator
no bias: Unbiased Estimator (for every $N$ ) or Asymptotically Unbiased Estimator (for $N \to \infty$ ); §
Sample Variance
The $1/N$ version is biased: the mean squared deviation $\hat\theta(y) = \frac1N\sum_k(y_k - M(y))^2 = \frac{N-1}{N}S^2$ has $\mathbb{E}\{\hat\theta\} = \frac{N-1}{N}\sigma_y^2$ . It is only asymptotically unbiased (derivation there). Since… §
Unbiased Estimator
Unbiased ⇒ asymptotically unbiased (the expectation is $\theta_\text{true}$ for every $N$ , hence also in the limit). The converse fails: see Asymptotically Unbiased Estimator. §
Point Estimation
An estimator turns random measurements into an estimate of a fixed but unknown parameter, so the estimate is a random variable. Two questions judge it: is it right on average (unbiased, or at least asymptotically unbiased), and how much do… §
Asymptotically Unbiased Estimator — right on average as $N \to \infty$ (Definition 2) §

Search in these mentions →