Correlation

Properties#

• Unitless and bounded: $\rho(Y, Z) \in [-1, 1]$.
• Extreme values: $|\rho| = 1$ iff $Z$ is an affine function of $Y$ (with probability one); the sign is the sign of the slope.
• Independence ⇒ uncorrelated, not conversely: independent variables have $\sigma(Y, Z) = 0$, hence $\rho = 0$. Uncorrelated variables can still be dependent (example below). Correlation only measures linear dependence.

Example#

Intuition#

Correlation is covariance with the scale removed. Multiplying $Z$ by $b > 0$ (e.g. switching from meters to centimeters) multiplies every offset product $(y - \mu_Y)(z - \mu_Z)$, and with it $\sigma(Y, Z)$, by $b$ (top row). The standard deviation $\sigma_Z$ grows by the same factor $b$, so dividing by $\sigma_Y\sigma_Z$ cancels the scaling exactly. Equivalently, $\rho(Y, Z) = \mathbb{E}\big\{\frac{Y - \mu_Y}{\sigma_Y} \cdot \frac{Z - \mu_Z}{\sigma_Z}\big\}$ is the covariance of the standardized variables: rescale each axis to spread 1, then weight the quadrants as for the covariance (bottom row, three times the same cloud).

$\sigma_Y\sigma_Z$ is also the ceiling of the covariance: $|\sigma(Y, Z)| \le \sigma_Y\sigma_Z$, with equality only if the points lie exactly on a line ($Z = cY$ gives $\sigma(Y, Z) = c\,\sigma_Y^2 = \sigma_Y\sigma_Z$). So $\rho$ is the share of the ceiling that the covariance reaches (bars below the panels). Adding independent noise $N$ to $Z = 2Y + N$ leaves the covariance at $2\sigma_Y^2$, but widens $\sigma_Z = \sqrt{4\sigma_Y^2 + \sigma_N^2}$, so $\rho$ falls. On a line $\rho = \pm 1$ whatever the slope: the slope is in the covariance, not in $\rho$.

$\rho$ measures how well the best straight line fits the cloud, not how strongly $Z$ depends on $Y$. In all three panels $Z$ is fully determined by $Y \sim \mathcal{N}(0, 1)$ (Normal Distribution), yet $\rho$ falls from the line ($\rho = 1$) over the monotone curve $Z = Y^3$ ($\rho = 3/\sqrt{15} \approx 0.77$, from $\mathbb{E}\{Y^4\} = 3$, $\mathbb{E}\{Y^6\} = 15$) to the symmetric parabola $Z = Y^2$ ($\rho = 0$, uncorrelated but dependent).

Sources#

• MSI script p. 16 (§2.2.4), MSI Slides 02 > 2.2.4 Covariance and Correlation (not published)