Sample Mean

Properties#

• Unbiased: $\mathbb{E}\{M(y)\} = \frac1N\sum_k\mathbb{E}\{y_k\} = \mu_y$ (Unbiased Estimator).
• Covariance with one measurement: $\operatorname{cov}(M(y), y_k) = \frac{\sigma_y^2}{N}$, so $\mathbb{E}\{y_k M(y)\} = \frac{\sigma_y^2}{N} + \mu_y^2$ (derivation in Covariance > Example).
• Second moment and variance: $$\mathbb{E}\{M(y)^2\} = \frac{1}{N^2}\big(N\mathbb{E}\{y_k^2\} + N(N-1)\mathbb{E}\{y_k\}\mathbb{E}\{y_j\}\big) = \frac{1}{N^2}\big(N(\sigma_y^2 + \mu_y^2) + N(N-1)\mu_y^2\big) = \frac{\sigma_y^2}{N} + \mu_y^2,$$ using that the $N$ squares have $\mathbb{E}\{y_k^2\} = \sigma_y^2 + \mu_y^2$ and the $N(N-1)$ cross terms $y_k y_j$, $j \ne k$, are products of independent variables. Hence $\operatorname{Var}(M(y)) = \mathbb{E}\{M(y)^2\} - \mu_y^2 = \frac{\sigma_y^2}{N}$: the standard deviation shrinks like $1/\sqrt N$.
• Consistent: unbiased with variance $\to 0$ (Consistent Estimator). For $N \to \infty$, $M(y) \to \mu_y$ (Law of Large Numbers).
• $M(y)$ depends on every $y_k$, so it is not independent of the data. This is why the mean squared deviation $\frac1N\sum_k(y_k - M(y))^2$ underestimates the variance (Sample Variance).

Example#

Sources#

• MSI script pp. 17–18 (§2.4, unbiasedness, covariance with a single measurement, $\mathbb{E}\{M(y)^2\}$), MSI Slides 02 > 2.4 Statistical Estimators (not published)