ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Estimator

Also Estimate · Statistical Estimator · Point Estimator · Bias · Schätzer · Schätzfunktion · Schätzwert

An estimator is a function θ̂(y) of random measurements y = (y₁, …, y_N) that estimates an unknown, non-random parameter θ_true; the estimate θ̂(y) is itself a random variable.

Know these firstRandom Variable

Properties#

Notation (MSI): a hat marks an estimate ( $\hat R$ , $\hat\theta$ ), the subscript $\text{true}$ the true value, and $\hat R^{[N]}$ the estimate after $N$ measurements.
Quality criteria:
- no bias: Unbiased Estimator (for every $N$ ) or Asymptotically Unbiased Estimator (for $N \to \infty$ );
- small spread: the covariance $\operatorname{cov}(\hat\theta(y))$ . Of two unbiased estimators $\hat\theta^A$ , $\hat\theta^B$ , $\hat\theta^B$ performs better if $\operatorname{cov}(\hat\theta^A) \succcurlyeq \operatorname{cov}(\hat\theta^B)$ ;
- convergence in probability to the true value: Consistent Estimator.
Typical behaviour: the covariance of an estimator usually shrinks as the number $N$ of measurements grows, often to zero for $N \to \infty$ . Some biased estimators have a bias that tends to zero as $N \to \infty$ .
Looking at data is not enough: different plausible estimators can converge to different values. Which one (if any) is right follows only from their statistical properties under a noise model (MSI resistance example).

Example#

ExampleThree estimators for one resistance

From noisy voltage and current measurements $(u_k, i_k)$ , MSI builds three estimators of $R$ : the Simple Approach Estimator $\frac1N\sum_k \frac{u_k}{i_k}$ , the Error-in-Variables Estimator $\frac{\sum_k u_k}{\sum_k i_k}$ and the Least Squares Estimator $\frac{\sum_k u_k i_k}{\sum_k i_k^2}$ . All three are functions of the same random data, all three converge, but to three different limits. Only the error-in-variables estimator is asymptotically unbiased (Resistance Estimation Example).

A simpler one: the Sample Mean $\hat\theta(y) = \frac1N\sum_k y_k$ estimates the mean $\mu_y$ .

Sources#

MSI script pp. 17–18 (§2.4), MSI Slides 02 > 2.4 Statistical Estimators (not published)
MSI script pp. 10–12 (§1.2, notation and the three resistance estimators), MSI_VL01 > Indices, time series and estimates, MSI_VL01 > Takeaway

Connections

Prerequisites know first1

Random Variable

Mentioned in inline links8

VL01 – IntroductionMSI
The concept notes for this lecture now exist. In ## Concepts, replace the plain-text names with links: “Estimator” → Estimator (foundation), “Resistance estimation example” → Resistance Estimation Example, “Error-in-Variables estimator” →… §
Asymptotically Unbiased Estimator
With $y = (y_1, \dots, y_N)$ , an estimator $\hat\theta(y)$ is called asymptotically unbiased iff §
Consistent Estimator
With $y = (y_1, \dots, y_N)$ , an estimator $\hat\theta(y)$ is called consistent if, for any $\epsilon > 0$ , §
Covariance Matrix
Covariance of an estimator: for an unbiased estimator $\hat\theta(y)$ , $\operatorname{cov}(\hat\theta(y))$ measures its performance; smaller is better. Two unbiased estimators are compared in the matrix sense:… §
Random Variable
Functions of random variables are random variables: $f(X)$ is itself a random variable, with Expected Value $\mathbb{E}\{f(X)\}$ . In particular every estimate $\hat\theta(y)$ computed from random measurements $y$ is a random variable (Esti… §
Sample Mean
Let $y_1, \dots, y_N$ be independent measurements of a scalar random variable with mean $\mu_y$ and variance $\sigma_y^2$ , $y = (y_1, \dots, y_N)$ . The sample mean is the estimator §
Unbiased Estimator
An estimator $\hat\theta$ is called unbiased iff §
Point Estimation
An estimator turns random measurements into an estimate of a fixed but unknown parameter, so the estimate is a random variable. Two questions judge it: is it right on average (unbiased, or at least asymptotically unbiased), and how much do… §
Estimator — a function of the data; the estimate is random, the parameter is not §

Search in these mentions →