[DEFINITIONS] Multivariable approximation

Total degree

The total degree of the monomial $x^ny^m$ is $n+m$ . The total degree of a polynomial is the largest total degree among its monomial terms.

For example, in

p(x,y)=1+x-y+x^2-xy+x^3y^2,

the term $x^3y^2$ has total degree $5$ , so $\deg p=5$ . This is the degree used when deciding which terms belong in a multivariable Taylor polynomial.

Hessian matrix

For $f:\mathbb{R}^2\to\mathbb{R}$ , the Hessian at $\mathbf{x}_0=(x_0,y_0)$ is

H_f(\mathbf{x}_0) = \begin{pmatrix} f_{xx}(\mathbf{x}_0)&f_{xy}(\mathbf{x}_0)\\ f_{yx}(\mathbf{x}_0)&f_{yy}(\mathbf{x}_0) \end{pmatrix}.

When the mixed partials are continuous, $f_{xy}=f_{yx}$ , so the Hessian is symmetric. It records the second-order changes omitted by the linear approximation and its determinant classifies stationary points through the quadratic part of the local Taylor model.

Taylor polynomial through degree three

Let $\mathbf{a}=(x_0,y_0)$ , $h_x=x-x_0$ and $h_y=y-y_0$ . If $f$ has continuous partial derivatives through order three near $\mathbf{a}$ , its Taylor polynomial through total degree three is

\begin{aligned} T_3(x,y) ={}&f(\mathbf{a}) +f_x(\mathbf{a})h_x +f_y(\mathbf{a})h_y\\ &+\frac{1}{2!} \left( f_{xx}(\mathbf{a})h_x^2 +2f_{xy}(\mathbf{a})h_xh_y +f_{yy}(\mathbf{a})h_y^2 \right)\\ &+\frac{1}{3!} \left( f_{xxx}(\mathbf{a})h_x^3 +3f_{xxy}(\mathbf{a})h_x^2h_y +3f_{xyy}(\mathbf{a})h_xh_y^2 +f_{yyy}(\mathbf{a})h_y^3 \right). \end{aligned}

The degree-two part can be written with the Hessian as

T_2(\mathbf{x}) = f(\mathbf{a}) +\nabla f(\mathbf{a})\cdot(\mathbf{x}-\mathbf{a}) +\frac12 (\mathbf{x}-\mathbf{a})^{\mathsf T} H_f(\mathbf{a}) (\mathbf{x}-\mathbf{a}).

The quadratic and cubic parts refine the tangent-plane model by the terms of total degree two and three.

To see where the mixed coefficients come from, apply the one-variable Taylor formula to $g(t)=f(\mathbf{a}+t\mathbf h)$ . Then $g''(0)=f_{xx}h_x^2+2f_{xy}h_xh_y+f_{yy}h_y^2$ ; the two mixed terms arise by differentiating once in each coordinate. The cubic coefficients count the three orders in which two $x$ -derivatives and one $y$ -derivative can occur, or vice versa.

Under the stated $C^3$ assumption, the omitted remainder satisfies

f(\mathbf a+\mathbf h)=T_3(\mathbf a+\mathbf h)+o(\|\mathbf h\|^3) \qquad(\mathbf h\to\mathbf0).

If $f$ is $C^4$ near $\mathbf a$ , the remainder improves to $O(\|\mathbf h\|^4)$ for sufficiently small $\mathbf h$ .

Definitions from Multivariable approximation

Total degree

Hessian matrix

Taylor polynomial through degree three

On this page