Determinants and Eigenvalues

Concepts#

1. 1 Determinantsigned volume factor $\det(A)$, test for invertibility start here
2. 2 Eigenvaluethe scaling factors $\lambda$, roots of $p_A(\lambda) = \det(A - \lambda I)$
3. 3 Eigenvectorthe directions $x \neq 0$ with $Ax = \lambda x$, eigenspaces, eigendecomposition

How it fits together#

The chain. Each concept is built on the previous one:

In practice: compute $p_A(\lambda)$ with the determinant, find its roots (the eigenvalues), then solve the homogeneous system $(A - \lambda I)x = 0$ for each root (the eigenspace). That is MML Example 4.5.

One picture behind all three. A linear map in $\mathbb{R}^2$ turns the unit square into a parallelogram:

• $|\det(A)|$ is its area, and $\operatorname{sign}\det(A)$ says whether it is mirrored.
• Eigenvectors are the directions that stay on their own line, and eigenvalues the stretch factors along them.
• If the eigenvectors form a basis, the parallelogram spanned by them is mapped to one with sides scaled by $\lambda_i$. So $\det(A) = \prod_i \lambda_i$.

Singularity tests. For $A \in \mathbb{R}^{n \times n}$ these are all the same statement (Invertible Matrix > Properties, Rank and Invertibility):

Invariants of a linear map. $\det$, $\operatorname{tr}$ and the eigenvalues don’t depend on the basis: similar matrices $S^{-1}AS$ share them. The eigenvectors do change with the basis, as vectors $x \mapsto S^{-1}x$, but they represent the same directions of the map.

How many eigenvectors? The algebraic multiplicity (from $p_A$) is an upper bound for the geometric multiplicity ($\dim E_\lambda$).

Symmetric matrices in MSI. The symmetric case is the one that matters for least squares and covariances. A symmetric matrix is positive (semi)definite iff all its eigenvalues are $> 0$ ($\ge 0$). By $\det = \prod\lambda_i$, a positive definite matrix has $\det > 0$ and is invertible. The converse fails: $\operatorname{diag}(-1, -1)$ has $\det = 1$.

Key Formulas#

• $\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$ (Determinant)
• $\det(AB) = \det(A)\det(B)$, $\det(A^\top) = \det(A)$, $\det(A^{-1}) = 1/\det(A)$, $\det(\lambda A) = \lambda^n\det(A)$ (Determinant)
• $Ax = \lambda x$, $x \neq 0$ (Eigenvalue, Eigenvector)
• $p_A(\lambda) = \det(A - \lambda I)$, eigenvalues = roots of $p_A$ (Eigenvalue > Characteristic polynomial)
• $\det(A) = \prod_i \lambda_i$, $\operatorname{tr}(A) = \sum_i \lambda_i$ (Eigenvalue > Determinant, trace and invariance)
• $E_\lambda = \ker(A - \lambda I)$ (Eigenvector)
• $A = PDP^{-1}$ iff the eigenvectors form a basis; $A = PDP^\top$ for symmetric $A$ (Eigenvector > Properties)

Lectures#

• No lecture yet. Source: MML ch. 4.1–4.2, pp. 99–114, eigendecomposition ch. 4.4, pp. 115–117.

Related Topics#

• Rank and Invertibility: rank as the cause of invertibility; $\det$ and eigenvalues give equivalent tests for square matrices