Jacobians

The Jacobian Matrix

Consider a function that maps reals to reals, . The linear approximation of this function is given by

This is pretty simple to do, and follows from taylor’s expansion upto the first order.

Let’s try expanding this concept to vector spaces. For a function , it’s linear approximation is given by

(bold type indicates vectors). This also follows from the taylor theorem for multivariable functions.

What do we do when we have a function mapping vector spaces to vector spaces? Consider the function . What would be the approximation term here?

Let’s try to solve this by decomposing the function into , . For each , we get

If we collect all these approximations into a vector by representing the term as a matrix product, we get our approximation for :

I’ve written the functions out in matrix form for clarity. denotes the transpose of the gradient vector. In simpler terms, we can rewrite this as

Here, is called the Jacobian matrix, and when expanded, it looks something like this:

The Jacobian Matrix thus, is an analog of the gradient vector for functions that map vector spaces to vector spaces. Everything that we can do using gradients can be done in a more general form using the Jacobian Matrix. Consider the condition for differentiability of a multivariate scalar function : for to be differentiable at , we have the condition

For a function , the condition would be:

here, is the euclidean norm of the vector.

Relating Jacobian Matrices and Transformation Matrices

If you notice, the Jacobian Matrix need not be square; for the special case , that is for , the Jacobian matrix is square and acts as a linear transformation between the two n-dimensional vector spaces, as shown below:

Therefore, this acts like a linear transformation between the infinitesimal elements in the space of and in the space of . One more proof of scaling is involved before the use of jacobian matrices in integrals becomes clear.

Scaling factor and the Jacobian Determinant

Recall that for any linear transformation, the determinant of the transformation gives us the scaling factor, that is the ratio of the change in ‘volume’ occupied by the vector. This is also known as a dilation transformation (because just the size is involved, without worrying about orientation).

I’ll provide a proof for the statement highlighted above in , but extending it to is easy enough (an exercise for the reader, as they say in math textbooks :)). Consider the unit cube centered at origin, ie having it’s 3 vectors as and . On applying a transformation to this cube, we get the vertices as the transformation matrix itself. Here’s an image that speaks a thousand words (taken from Hop’s Blog):

In algebra terms, we get

The volume of a parallelepiped is given by the determinants of the 3 edge vectors, and hence the volume of the transformed cube is

Which is the determinant of the linear transform that we started with.

This is the main principle that allows us to use the Jacobian in multiple integrals while changing variables: it scales up or down the size of the area or volume element we are using proportionately to the change of variables.

References:

1. What is the Jacobian Matrix, a good MSE thread on the Jacobian Matrix
2. Jacobian Matrix and Determinant Wikipedia page, of course
3. A Quora question on Jacobian matrices with another very nice answer