Eigenvalue

MML defines eigenvalues as real numbers but uses complex ones from Thm. 4.16 on. The same definition with $\lambda \in \mathbb{C}$, $x \in \mathbb{C}^n \setminus \{0\}$ gives the complex eigenvalues. A real matrix can have them (see Intuition).

Properties#

ProofWhy

$Ax = \lambda x \iff Ax - \lambda I_n x = 0 \iff (A - \lambda I_n)x = 0$, so (1) ⇔ (2). A nonzero solution exists iff $\ker(A - \lambda I_n) \neq \{0\}$, iff $\operatorname{rank}(A - \lambda I_n) < n$ by rank–nullity (Matrix Rank), iff $\det(A - \lambda I_n) = 0$ (Determinant > Properties).

Characteristic polynomial#

For $A \in \mathbb{R}^{n \times n}$ the characteristic polynomial is (MML Def. 4.5)

$$p_A(\lambda) = \det(A - \lambda I) = c_0 + c_1\lambda + \dots + c_{n-1}\lambda^{n-1} + (-1)^n\lambda^n, \qquad c_0 = \det(A),\quad c_{n-1} = (-1)^{n-1}\operatorname{tr}(A),$$

where $\operatorname{tr}(A) = \sum_{i=1}^n a_{ii}$ is the trace, the sum of the diagonal entries.

• The algebraic multiplicity of an eigenvalue $\lambda_i$ is how often $\lambda_i$ appears as a root of $p_A$ (MML Def. 4.9). The geometric multiplicity is $\dim E_{\lambda_i}$, the number of linearly independent eigenvectors (Eigenvector > Properties). Always $1 \le$ geometric $\le$ algebraic.
• $p_A$ has degree $n$, so $A$ has at most $n$ distinct eigenvalues, and exactly $n$ complex ones counted with algebraic multiplicity.
• Computing eigenvalues as roots of $p_A$ only works by hand for small $n$. For $n \ge 5$ there is no general closed formula for the roots, and numerical software uses iterative methods instead.

Determinant, trace and invariance#

ProofProof (beyond the lecture)

Over $\mathbb{C}$ the polynomial factors as $p_A(\lambda) = (-1)^n \prod_{i=1}^n (\lambda - \lambda_i) = \prod_{i=1}^n (\lambda_i - \lambda)$. Setting $\lambda = 0$: $\det(A) = p_A(0) = \prod_i \lambda_i$. The coefficient of $\lambda^{n-1}$ in $\prod_i (\lambda_i - \lambda)$ is $(-1)^{n-1}\sum_i \lambda_i$. Comparing it with $c_{n-1} = (-1)^{n-1}\operatorname{tr}(A)$ gives $\operatorname{tr}(A) = \sum_i \lambda_i$.

• Singular ⇔ eigenvalue 0: $\det(A) = \prod_i \lambda_i = 0$ iff some $\lambda_i = 0$. So $A$ is invertible iff $0$ is not an eigenvalue, and then $A^{-1}$ has the eigenvalues $1/\lambda_i$ (from $Ax = \lambda x \Rightarrow x = \lambda A^{-1}x$).
• Basis-independent: $A$ and $A^\top$ have the same eigenvalues, and so do similar matrices $S^{-1}AS$ (same $p_A$). So the eigenvalues, like $\det$ and $\operatorname{tr}$, belong to the linear map, not to its matrix (MML p. 106).
• Triangular matrices (beyond the lecture): $p_T(\lambda) = \prod_i (T_{ii} - \lambda)$, so the eigenvalues are the diagonal entries.
• Symmetric matrices have only real eigenvalues (spectral theorem, Eigenvector > Properties). A symmetric matrix is positive semidefinite iff all its eigenvalues are $\ge 0$, and positive definite iff all are $> 0$.

Example#

Intuition#

On the line through an eigenvector $x$, the matrix acts like a single number: $Ax = \lambda x$. The eigenvalue says what happens on that line: $|\lambda| > 1$ stretches it, $|\lambda| < 1$ shrinks it, $\lambda < 0$ flips it to the other side of the origin, and $\lambda = 0$ squashes it to $0$, so $x \in \ker(A)$. The four matrices here keep lengths across $x$, so the image of the unit circle (red) shows only the effect of $\lambda$.

Why the eigenvalues are roots of $\det(A - \lambda I)$: $A - \lambda I$ maps $x$ to $Ax - \lambda x$, and its determinant is the area of the image of the unit square. For a $\lambda$ that is not an eigenvalue ($\lambda = 3.5$) the square becomes a real parallelogram (signed area $p_A(3.5) = -2.25$). At $\lambda = 5$ the eigenvector $x$ satisfies $(A - 5I)x = Ax - 5x = 0$. So a whole direction is sent to $0$, the plane collapses onto a line, and the area $p_A(5)$ is $0$. The curve on the left also shows $p_A(0) = \det(A)$.

Not every matrix has real eigenvalues. Each point $x$ on the unit circle moves to $Ax$, and eigen directions are those where the arrow points straight along the radius (or is $0$). A rotation moves every point sideways, so no direction is kept and the eigenvalues $0.87 \pm 0.50\,i$ are complex. The shear has the double root $\lambda = 1$, but only the horizontal direction stays put, so a repeated eigenvalue can come with fewer eigenvectors than its multiplicity.

For a symmetric $S$ with orthonormal eigenvectors $x_1, x_2$, the unit square spanned by $x_1, x_2$ is mapped to the rectangle spanned by $\lambda_1 x_1, \lambda_2 x_2$. Its sides are $\lambda_1 = 4$ and $\lambda_2 = 2$, so its area is $\lambda_1\lambda_2 = 8$, and that is $\det(S)$ by the area picture of the determinant. This is $\det(A) = \prod_i \lambda_i$. The trace counts the side lengths instead: $\lambda_1 + \lambda_2 = 6 = \operatorname{tr}(S)$ (MML Fig. 4.6).

Sources#

• MML pp. 104–105 (Def. 4.5: characteristic polynomial, $c_0$, $c_{n-1}$), MML p. 105 (Def. 4.6, equivalent conditions)
• MML p. 106 (Thm. 4.8, Def. 4.9 algebraic multiplicity, Def. 4.10 spectrum, invariance), MML pp. 107–108 (Examples 4.5, 4.6)
• MML pp. 108–110 (graphical intuition, Fig. 4.4: scaling, shear, rotation, collapse)
• MML pp. 112–113 (Thm. 4.16, 4.17: $\det$ = product, $\operatorname{tr}$ = sum, Fig. 4.6)
• Use in MSI: eigenvalue test for positive (semi)definiteness, MSI script p. 10, MSI_VL01 > Sets, vectors, matrices