Multivariable Taylor Approximation and Error

Add curvature to the tangent model

For a twice continuously differentiable scalar function near $a$ , with displacement vector $h$ , the second-order Taylor expansion is

f(a+h)=f(a)+\nabla f(a)^Th+\tfrac12h^TH(a)h+o(\|h\|^2).

In two variables the quadratic term is $\tfrac12(f_{xx}h_x^2+2f_{xy}h_xh_y+f_{yy}h_y^2)$ . The factor two on the mixed term comes from both off-diagonal Hessian entries.

To bound the error of the first-order approximation, suppose the Hessian operator norm is at most $M$ along the segment from $a$ to $a+h$ . Then $|f(a+h)-f(a)-\nabla f(a)^Th|\le M\|h\|^2/2$ . The bound must hold along the segment, not only at the base point.

Visual guide

VISUAL GUIDEThe quadratic correction measures curvature

Exact quadratic value
Linear prediction

Along displacement h = s(0.1, −0.2) from (1, 1), f = x² + xy + y² becomes 3 − 0.3s + 0.03s². The linear estimate omits the upward quadratic correction. At s = 1 the missing term is 0.03.

Worked example: an exact quadratic

For $f=x^2+xy+y^2$ at $a=(1,1)$ , $f(a)=3$ , $\nabla f(a)=(3,3)$ and $H=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ . At displacement $h=(0.1,-0.2)$ , the linear model gives $3+0.3-0.6=2.7$ .

The quadratic correction is $h_x^2+h_xh_y+h_y^2=0.03$ , giving the exact answer $2.73$ . There is no higher remainder because the original function is quadratic. Since the Hessian eigenvalues are $1,3$ , the first-order error bound is $3(0.05)/2=0.075$ , safely above the actual error $0.03$ .

Worked example: uncertainty propagation

For area $A=xy$ near $x=2,y=3$ , with both input errors at most $0.01$ , this linear bound is $0.05$ . The exact correction $\Delta x\Delta y$ has magnitude at most $0.0001$ , so a rigorous total bound is $0.0501$ . Units are area units; independent statistical errors would call for a different probabilistic calculation.

Practice

Write the quadratic Taylor polynomial for $e^{x+y}$ at zero.
If $M=4$ and $\|h\|=0.1$ , bound the linearization error.
Why does a zero gradient not make the quadratic correction vanish?

Show worked solutions

$1+x+y+(x^2+2xy+y^2)/2$ .
At most $4(0.1)^2/2=0.02$ .
The gradient controls the first-order term; the Hessian can still be nonzero, as at the origin of $x^2+y^2$ .

Further study

MIT OpenCourseWare: Multivariable Calculus provides a full university course with additional lectures and exercises.