Table of Contents
The Gradient
Take another look at the formula for the differential of a function of several variables: $$ df = \Partial{f}{x}\,dx + \Partial{f}{y}\,dy + \Partial{f}{z}\,dz $$ Each term is a product of two factors, labeled by $x$, $y$, and $z$. This looks like a dot product! Separating out the pieces, we have $$ df = \left( \Partial{f}{x}\,\ii + \Partial{f}{y}\,\jj + \Partial{f}{z}\,\kk \right) \cdot (dx\,\ii + dy\,\jj + dz\,\kk) $$ The last factor is just $d\rr$, and you may recognize the first factor as the gradient of $f$, that is $$ \grad{f} = \Partial{f}{x}\,\ii + \Partial{f}{y}\,\jj + \Partial{f}{z}\,\kk $$ Putting this all together, we have the master formula 1) \begin{equation} df = \grad{f} \cdot d\rr \label{Master} \end{equation}
Recall that $df$ represents the infinitesimal change in $f$ when moving to a “nearby” point. What information do you need in order to know how $f$ changes? You must know where you started, which way you went, and something about how $f$ behaves. The master formula organizes this information into two geometrically different pieces, namely the gradient, containing generic information about how $f$ changes at the point in question, and the vector differential $d\rr$, containing information about the particular change in position being made.
Like all derivative operators, the gradient is linear (the gradient of a sum is the sum of the gradients), and also satisfies a product rule $$\grad(fg) = (\grad{f})\,g + f\,(\grad{g})$$
Differentials such as $df$ are rarely themselves the answer to any physical question. So what good is the master formula? The short answer is that you can use it to answer any question about how $f$ changes. Here are some examples.
- Suppose you are an ant walking in a puddle on a flat table. The depth of the puddle is given by $h(x,y)$. You are given $x$ and $y$ as functions of time $t$. How fast is the depth of water through which you are walking changing per unit time?
This problem is asking for the derivative of $h$ with respect to $t$. So divide the master formula by $dt$ to get $$ {dh\over dt} = \grad{h} \cdot {d\rr\over dt} $$ where $\rr$ describes the particular path you are taking. The factor $d\rr\over dt$ is simply your velocity! This dot product is easy to evaluate, and yields the answer to the question.
(There are of course many ways to solve this problem; which method you choose may depend on how your path is described. It is often easiest to simply insert the given expressions for $x$ and $y$ in terms of $t$ directly into $h$, then differentiate the resulting function of a single variable, thus calculating the left-hand side directly.)
- You are another ant on the same surface, moving on a path with $y=3x$. How fast is the depth changing compared with $x$?
This problem is asking for the derivative of $h$ with respect to $x$ as you move along the path; note that this is the total derivative $dh\over dx$, not the partial derivative $\Partial{h}{x}$, which would only be appropriate if $y$ were constant along the path. So divide the master formula by $dx$ to get $$ {dh\over dx} = \grad{h} \cdot {d\rr\over dx} $$ then use what you know ($y=3x$) to relate the changes in $x$ and $y$ ($dy=3\,dx$), so that $$ d\rr = dx\,\ii+dy\,\jj = dx\,\ii+3\,dx\,\jj = (\ii+3\,\jj) \,dx $$ to obtain $$ {d\rr\over dx} % = {dx\over dx}\ii + {dy\over dx}\jj = \ii + 3\,\jj $$ Evaluating the dot product yields the answer to the question.
- You are still moving on the same surface, but now the question is how fast is the depth changing per unit distance along the path?
This problem is asking for the derivative of $h$ with respect to arclength $ds$. We can divide the master formula by $ds$, which leads to $$ {dh\over ds} = \grad{h} \cdot {d\rr\over ds} $$ Unfortunately, it is often difficult to determine $s$; it is not always possible to express $h$ as a function of $s$. On the other hand, all we need to know is that $$ ds = |d\rr| $$ so that dividing $d\rr$ by $ds$ is just dividing by its length; the result must be a unit vector! Which unit vector? The one tangent to your path, namely the unit tangent vector $\TT$, so \begin{equation} {dh\over ds} = \grad{h} \cdot \TT \label{Directional} \end{equation} Evaluating the dot product answers the question, without ever worrying about arclength.
Directional Derivatives
We have just seen that the derivative of $f$ along a curve splits into two parts: a derivative of $f$ (namely $\grad{f}$), and a derivative of the curve ($d\rr/du$). But the latter depends only on the tangent direction of the curve at the given point, not on the detailed shape of the curve. This leads us to the concept of the directional derivative of $f$ at a particular point $\rr=\rr_0=\rr(u_0)$ along the vector $\vv$, which is traditionally defined as follows: 2) $$D_{\vv} f = \lim_{\epsilon\to0} {f(\rr_0+\epsilon\vv)-f(\rr_0) \over \epsilon}$$ According to the above discussion, this is just $df/du$ along the tangent line, which is $$D_{\vv} f = \grad{f} \cdot \vv$$ In Example 3 above, the left-hand side of (\ref{Directional}) is just the directional derivative of $h$ in the direction $\TT$, and could have been denoted by $\DD{\TT}h$.
What does the gradient mean geometrically? Let $\Hat w$ be a unit vector, and consider $$D_{\Hat w} f = \grad{f} \cdot \Hat w = |\grad{f}| \> |\Hat w| \cos\theta = |\grad{f}| \cos\theta$$ which is clearly maximized by $\theta=0$. Thus, the direction of $\grad{f}$ is just the direction in which $f$ increases the fastest, and the magnitude of $\grad{f}$ is the rate of increase of $f$ in that direction.
You can also visualize the gradient using the level surfaces on which $f(x,y,z)={\rm const}$. (In 2-d there is the analogous concept of level curves, on which $f(x,y)={\rm const}$.) Let $\rr=\rr(u)$ be any curve contained in a level surface of $f$, so that $$f\Big(\rr(u)\Big) = {\rm const}$$ But then $$\grad{f} \cdot \vv = {df\over du} = 0$$ Therefore, $\grad{f}$ is perpendicular to $\vv$, and hence to every curve in the level surface. In other words, $\grad{f}$ is perpendicular to the level surfaces of $f$: $$\grad{f} \perp \{f(x,y,z)={\rm const}\}$$
Other Coordinates
The master formula can be used to derive formulas for the gradient in other coordinate systems. We illustrate the method for polar coordinates.
In polar coordinates, we have $$ df = \Partial{f}{r}\,dr + \Partial{f}{\phi}\,d\phi $$ and of course $$ d\rr = dr\,\rhat + r\,d\phi\,\phat $$ which is (3) of Section 2. Comparing these expressions with (\ref{Master}), we see immediately that we must have \begin{equation} \grad f = \Partial{f}{r}\,\rhat + {{1}\over{r}}\Partial{f}{\phi}\,\phat \label{gradpolar} \end{equation} Note the factor of ${{1}\over{r}}$, which is needed to compensate for the factor of $r$ in (3) of Section 2. Such factors are typical for the component expressions of vector derivatives in curvilinear coordinates.
Why would one want to compute the gradient in polar coordinates? Consider the computation of $\grad\,\left({\ln\sqrt{x^2+y^2}}\right)$, which can done by brute force in rectangular coordinates; the calculation is straightforward but messy, even if you first use the properties of logarithms to remove the square root. Alternatively, using (\ref{gradpolar}), it follows immediately that $$ \grad\,\left({\ln\sqrt{x^2+y^2}}\right) = \grad\,({\ln r}) = {1\over r}\,\rhat $$
GOALS
- Understand gradients and directional derivatives.
- Know the master formula.
