Table of Contents

Lagrange Multipliers Revisited

As described in § Lagrange Multipliers, a standard technique for solving constrained optimization problems is to use Lagrange Multipliers. However, there are also several other techniques for solving such problems. We describe several such methods below.

Consider as an example the question of finding the maximum and minimum values of $xy$ on the circle of radius $3$. In this case, the function we wish to extremize is $f=xy$, and the constraint function is $g=x^2+y^2$, with constraint $g=9$.

Method 1: Substitution

Solve the constraint for one variable, then substitute the result.

For our example, we solve $g=9$ for $y$, obtaining \begin{equation} y = \pm \sqrt{9-x^2} \end{equation} and insert the result into $f$, obtaining \begin{equation} f = \pm x\,\sqrt{9-x^2} \end{equation} It is now straightforward, if messy, to differentiate $f$ with respect to $x$ to find the critical points at $x=\pm\frac{3}{\sqrt2}$. Don't forget to check the endpoints at $x=\pm3$!

Method 2: Parametrization

Parametrize the constraint, then substitute the result.

Our constraint $g=9$ is a circle, on which $x=3\,\cos\phi$, $y=3\,\sin\phi$. Substituting this parametric description of the circle into $f$ yields \begin{equation} f=9\sin\phi\cos\phi \end{equation} which can easily be differentiated with respect to $\phi$ to find the critical points at $\phi=\frac{\pi}{4},\frac{3\pi}{4},\frac{5\pi}{4},\frac{7\pi}{4}$. In this particular example, it is not necessary to check the “endpoints” at $\phi=0,2\pi$. (Why not?)

Method 3: Lagrange Multipliers

Use Lagrange multipliers: $\grad{f}=\lambda\grad{g}$.

In our example, we have \begin{eqnarray} \grad{f} &=& y\,\xhat + x\,\yhat \\ \grad{g} &=& 2x\,\xhat + 2y\,\yhat \end{eqnarray} so setting $\grad{f}=\lambda\grad{g}$ leads to \begin{equation} \frac{y}{2x} = \lambda = \frac{x}{2y} \end{equation} Eliminating $\lambda$ then yields $y^2=x^2$, which can be substituted into the constraint $g=9$ to determine the critical points.

The geometry behind this approach is described in § Lagrange Multipliers,

Method 4: Cross Product

Use the cross product: $\grad{f}\times\grad{g}=\zero$.

We have already computed $\grad{f}$ and $\grad{g}$; their cross product is \begin{equation} \grad{f}\times\grad{g} = 2(y^2-x^2)\,\zhat \end{equation} which immediately yields $y^2=x^2$, as before.

Method 5: Differentials

Use differentials: $df=0=dg$.

In our example, we have \begin{eqnarray} df &=& y\,dx + x\,dy \\ dg &=& 2x\,dx + 2y\,dy \end{eqnarray} Solving $dg=0$ for $dy$ yields \begin{equation} dy=-\frac{x}{y}dx \label{dylag} \end{equation} and substituting this result into $df=0$ yields \begin{equation} \left( y-\frac{x^2}{y} \right) dx = 0 \end{equation} from which it again follows that $y^2=x^2$.

An equivalent description of Method 5, which brings out the geometry behind this method, is to find the points where $\grad{f}$ is perpendicular to the constraint. So we want to find the points where \begin{equation} \grad{f}\cdot d\rr = 0 \label{drlag} \end{equation} with $d\rr$ pointing along the constraint. To determine $d\rr$, we start as usual with \begin{equation} d\rr = dx\,\xhat + dy\,\yhat \end{equation} in rectangular coordinates in two dimensions, then use what we know ($g=9$) to relate $dy$ to $dx$. Thus, we solve $dg=0$ for $dy$, obtaining (\ref{dylag}), and substitute the result into $(\ref{drlag})$. But the Master Formula tells us that (\ref{drlag}) is just the statement that $df=0$ (along the curve)!

Whether one describes this method in terms of $d\rr$, which is more geometric, or simply sets $df=0$, which is more algebraic, is therefore a matter of choice; these two computations are in fact the same.

Discussion

Which of these methods is used is largely a matter of personal preference. Methods 1 and 2 are the most straightforward, but also usually involve the messiest computations. Method 3 emphasizes the geometry, but the extra step of eliminating $\lambda$ seems unnecessary — unless one is interested in the value of $\lambda$ itself, as is sometimes the case in economics. Method 4 is a simplification of Method 3 for the case of one constraint, but, unlike the other methods, doesn't easily generalize to situations involving multiple constraints. Method 5 is quite robust, but requires familiarity either with $d\rr$, or at least with manipulations involving differentials.