ConceptProbability and Statistics used inMSIAI-written · not yet proofread

Expected Value

Also Expectation · Expectation Value · Expectation Operator · Mean · Mean Value · Erwartungswert

The expected value of a function f(X) of a random variable X is the PDF-weighted integral E{f(X)} = ∫ f(x) p_X(x) dx; for f(x) = x it is the mean μ_X.

Know these firstProbability Density Function

Properties#

Linearity: since the integral is linear, $\mathbb{E}\{a + bX\} = a + b\,\mathbb{E}\{X\}$ for fixed $a, b \in \mathbb{R}$ ; in the vector case $\mathbb{E}\{QX\} = Q\,\mathbb{E}\{X\}$ for a fixed matrix $Q$ . Constants come out: $\mathbb{E}\{c\} = c$ .
Not for nonlinear functions: in general $\mathbb{E}\{f(X)\} \ne f(\mathbb{E}\{X\})$ . Example: $\mathbb{E}\{X^2\} = (\mathbb{E}\{X\})^2 + \sigma_X^2$ (Variance).
Product of independent variables: $\mathbb{E}\{XY\} = \mathbb{E}\{X\}\,\mathbb{E}\{Y\}$ if $X$ , $Y$ are independent.
Higher moments: Variance, Covariance and the Covariance Matrix are all expectations of products of centred variables $X - \mu_X$ .
Averages approach the expected value: $\frac1N\sum_{k=1}^N y_k \to \mathbb{E}\{y\}$ for many measurements (Law of Large Numbers).

Example#

Sources#

MSI script p. 14 (§2.2.1), MSI Slides 02 > 2.2.1 Mean and Variance (not published)
MSI script p. 16 (§2.3, vector case), MSI Slides 02 > 2.3 Multidimensional Random Variables (not published)

Connections

Mentioned in inline links10

Simple Approach EstimatorMSI
Root cause: $1/i$ is nonlinear, and $\mathbb{E}\{u/i\} \ne \mathbb{E}\{u\}/\mathbb{E}\{i\}$ (Expected Value). Averaging after the nonlinear division keeps a noise contribution; the Error-in-Variables Estimator divides only after averaging. §
VL02 – Probability and Statistics in a NutshellMSI
The expectation operator $\mathbb{E}\{f(X)\} = \int f(x)\,p_X(x)\,\mathrm{d}x$ defines the mean (Expected Value) and the Variance $\sigma_X^2 = \mathbb{E}\{(X-\mu_X)^2\}$ . The worked example shows $\sigma_X^2 = \mathbb{E}\{X^2\} - \mu_X^2$ … §
Expected Value: the expectation operator, linear for affine functions §
Multivariate Normal Distribution
Moments: $\mathbb{E}\{X\} = \mu$ and $\operatorname{cov}(X) = \Sigma$ (by integral computations), so the mean and the covariance matrix determine the distribution. §
Normal Distribution
Parameters are the moments: the mean is $\mu_X = \mu$ and the variance is $\sigma_X^2 = \sigma^2$ . So the distribution is completely determined by its mean and variance. §
Probability Density Function
Expectations are integrals against the PDF: $\mathbb{E}\{f(X)\} = \int f(x)\,p_X(x)\,\mathrm{d}x$ (Expected Value). §
Random Variable
Functions of random variables are random variables: $f(X)$ is itself a random variable, with Expected Value $\mathbb{E}\{f(X)\}$ . In particular every estimate $\hat\theta(y)$ computed from random measurements $y$ is a random variable (Esti… §
Unbiased Estimator
Linearity helps: an estimator that is linear in the data inherits unbiasedness from the data via $\mathbb{E}\{\sum_k a_k y_k\} = \sum_k a_k\mathbb{E}\{y_k\}$ . Nonlinear estimators (ratios, squares) are often biased, since… §
Uniform Distribution
Mean: $\displaystyle\int_a^b \frac{1}{b-a}\, x\,\mathrm{d}x = \frac{b^2 - a^2}{2(b-a)} = \frac{a+b}{2}$ . §
Variance
Expand the square and use linearity, with $\mathbb{E}\{X\}$ a constant: §
Moments of Random Variables
Instead of a whole density, one usually works with a few numbers: the mean (where the variable lies), the variance (how far it spreads), the covariance and correlation (how two variables move together), and for random vectors the covarianc… §
Expected Value — the linear operator $\mathbb{E}\{\cdot\}$ behind all moments §

Search in these mentions →