Optimization in Several Variables
Cheat sheet.

The key rules, formulas and reminders from all 4 topics, gathered into reference cards.

Key formulas, conditions and traps · Read down each column.

Critical Points and the Hessian

At a stationary point: $D=f_{xx}f_{yy}-f_{xy}^2$ . $D<0$ : saddle. $D>0$ : use the sign of $f_{xx}$ . $D=0$ : inconclusive.

This is a local test requiring second-order regularity. Semidefinite Hessians do not settle the classification.

For continuous functions on compact regions, compare interior critical points, boundary critical points and boundary endpoints/corners.

A boundary extremum need not have zero gradient. Check whether the domain actually includes limiting boundary points.

\nabla f=\lambda\nabla g,\quad g=c,\quad \nabla g\ne0.

Compare candidates and check singular constraint points. Multiple constraints require independent gradients for the usual theorem.

P_2=f(a)+\nabla f(a)^Th+\tfrac12h^TH(a)h,\quad |f-L|\le\tfrac M2\|h\|^2.

Include both mixed terms. A rigorous error bound requires a derivative bound along the entire input segment.

2 reference blocks

At a stationary point: $D=f_{xx}f_{yy}-f_{xy}^2$ . $D<0$ : saddle. $D>0$ : use the sign of $f_{xx}$ . $D=0$ : inconclusive.

This is a local test requiring second-order regularity. Semidefinite Hessians do not settle the classification.

2 reference blocks

For continuous functions on compact regions, compare interior critical points, boundary critical points and boundary endpoints/corners.

A boundary extremum need not have zero gradient. Check whether the domain actually includes limiting boundary points.

2 reference blocks

\nabla f=\lambda\nabla g,\quad g=c,\quad \nabla g\ne0.

Compare candidates and check singular constraint points. Multiple constraints require independent gradients for the usual theorem.

2 reference blocks

P_2=f(a)+\nabla f(a)^Th+\tfrac12h^TH(a)h,\quad |f-L|\le\tfrac M2\|h\|^2.

Include both mixed terms. A rigorous error bound requires a derivative bound along the entire input segment.