Resistance Estimation Example

Concepts#

1. 1 Simple Approach Estimator$\frac1N\sum_k u_k/i_k$: average of the ratios, biased upwards (and may not converge) start here
2. 2 Error-in-Variables Estimator$\sum_k u_k / \sum_k i_k$: ratio of the averages, asymptotically unbiased
3. 3 Least Squares Estimator$\sum_k u_k i_k / \sum_k i_k^2$: minimizes squared voltage residuals, biased downwards

How it fits together#

The noise model (script §2.5): $u_k = u_\text{true} + n_{u,k}$, $i_k = i_\text{true} + n_{i,k}$, with $n_{u,k}$, $n_{i,k}$ independent of each other, i.i.d., zero mean, variances $\sigma_u^2$, $\sigma_i^2$, and $R_\text{true} = u_\text{true}/i_\text{true}$. All three analyses use the same three limits from the Law of Large Numbers: averages of the noise $\to 0$, averages of products of the two noises $\to 0$, averages of squared current noise $\to \sigma_i^2$.

Result. With the relative current noise $\kappa = \sigma_i^2 / i_\text{true}^2$:

This matches Fig. 1.4 (script §2.5.4): EV converges to the true value, LS to a smaller one, and SA, where it converges, to a larger one.

Why they differ: where the nonlinearity sits relative to the averaging. The noise is zero mean, so it cancels from any average that is linear in it. A $\sigma_i^2$ survives wherever the current noise enters squared before averaging, because $\mathbb{E}\{n^2\} = \sigma^2 \ne 0$ (Moments of Random Variables).

• EV averages first and applies the only nonlinear operation (the division) afterwards: no squared noise term, no bias.
• LS averages $i_k^2 = i_\text{true}^2 + 2i_\text{true}n_{i,k} + n_{i,k}^2$ in its denominator, which picks up $\sigma_i^2$ and makes the denominator too large.
• SA divides first and averages $1/i_k$; the Taylor series of $1/i_k$ contains $+n_{i,k}^2/i_\text{true}^2$, which makes the average too large.

Only the current noise matters. $\sigma_u$ appears in none of the limits. The voltage noise enters all three estimators linearly (and independently of the current noise), so it averages out. What biases LS is noise on the regression variable $i_k$, a general property of least squares.

Special case: noise-free current. If $\sigma_i = 0$, then $\kappa = 0$ and all three estimators agree in the limit; in fact $i_k = i_\text{true}$ for all $k$, and all three formulas reduce to $\frac{1}{i_\text{true}}\cdot\frac1N\sum_k u_k$. This is the setting in which least squares is the natural choice (output noise only), the case studied in script chapter 4. It answers the open question in MSI_VL01 > Open Questions.

Analysis effort. EV and LS only need limits of averages. SA needs a Taylor expansion valid only for small noise, and with noise that can make $i_k = 0$ (e.g. Gaussian), the average need not converge at all. The script therefore treats SA last and only under “optimistic assumptions”; even then it is biased, so it is biased in general.

Example#

Lectures#

• MSI_VL01 — §1.2: the three estimators and Fig. 1.4, the question which one is right
• Script §2.5 (MSI Slides 02 > 2.5 Analysis of the Resistance Estimation Example (not published)) resolves it; no write-up yet

Related Topics#

• Point Estimation — the definitions (unbiased, asymptotically unbiased, consistent) used to judge the three estimators
• Moments of Random Variables — $\mathbb{E}\{n^2\} = \sigma^2$ is where every bias in this example comes from