Multivariable Differentiation

Concepts#

1. 1 Jacobian Matrixthe matrix of all first partial derivatives, the derivative of a vector-valued function start here

How it fits together#

• From partial derivatives to one matrix: each partial derivative $\frac{\partial f_i}{\partial x_j}$ describes one component in one direction. Collecting them row by row (per output) gives the Jacobian, and the first-order Taylor expansion $f(\bar x + h) \approx f(\bar x) + J(\bar x)h$ turns the nonlinear $f$ into an affine model around $\bar x$.
• Special shapes: for scalar $f$ ($m = 1$) the Jacobian is a single row and its transpose is the gradient. Second derivatives of a scalar $f$ give the Hessian $\nabla^2 f(x) \in \mathbb{R}^{n \times n}$, which MSI only defines for scalar $f$ (MSI_VL01 > Derivatives).
• Link to linear algebra: local properties of $f$ are matrix properties of $J(\bar x)$. If $m = n$ and $J(\bar x)$ is invertible, $f$ is locally invertible and Newton steps $x_{k+1} = x_k - J(x_k)^{-1} f(x_k)$ are well defined. If $J$ has full column rank, the linearized least squares problem in Gauss–Newton has a unique solution (see Rank and Invertibility).

Exercises & Exam Questions#

Lectures#

• MSI_VL01 — §1.1 notation: Jacobian convention and $\nabla f = \frac{\partial f}{\partial x}^\top$ for vector-valued $f$