Jacobian Matrix

Properties#

• Best linear approximation: $f(x + h) = f(x) + J(x)\,h + o(\|h\|)$. The linear map $h \mapsto J(x)h$ is the derivative of $f$ at $x$.
• Linear and affine maps: for $f(x) = Ax + b$ the Jacobian is $J(x) = A$ for every $x$.
• Chain rule: for $g : \mathbb{R}^m \to \mathbb{R}^p$, $\frac{\partial (g \circ f)}{\partial x}(x) = \frac{\partial g}{\partial y}(f(x)) \, \frac{\partial f}{\partial x}(x)$, a $p \times m$ times $m \times n$ product.
• Gradient: for scalar $f$ ($m = 1$) the Jacobian is the row vector $\frac{\partial f}{\partial x}(x) \in \mathbb{R}^{1 \times n}$, and the gradient is its transpose, $\nabla f(x) = \frac{\partial f}{\partial x}(x)^\top \in \mathbb{R}^n$.
• Square case ($m = n$): if $J(\bar x)$ is an invertible matrix and $f$ is continuously differentiable, then $f$ is locally invertible around $\bar x$ (inverse function theorem). Newton’s method for $f(x) = 0$ needs $J(x_k)$ invertible in every step.

Example#

Sources#

• MSI script p. 9 (notation: Jacobian $\frac{\partial f}{\partial x}(x) \in \mathbb{R}^{m \times n}$, generalized gradient), MSI_VL01 > Derivatives
• MSI script p. 51 (Gauss–Newton needs $J(\theta^{[k]})$ of full rank)
• Exercise: MSI Sheet 00, task 2 (not published) (is $J(\bar x)$ invertible?)