1.矩阵的导数

\qquad如果矩阵 A(t)=[aij(t)]m×n\boldsymbol A(t)=[a_{ij}(t)]_{m\times n}A(t)=[aij(t)]m×n 的每一个元素 aij(t)a_{ij}(t)aij(t) 都是变量 ttt 的可微函数，则称矩阵 A(t)\boldsymbol A(t)A(t) 是可微的，其导数定义为：

dA(t)dt=[daij(t)dt]m×n=[da11(t)dtda12(t)dt⋯da1n(t)dtda21(t)dtda22(t)dt⋯da2n(t)dt⋮⋮⋯⋮dam1(t)dtdam2(t)dt⋯damn(t)dt]\qquad\qquad \dfrac{\mathrm{d}\boldsymbol A(t)}{\mathrm{d}t}=\left[\dfrac{\mathrm{d}a_{ij}(t)}{\mathrm{d}t}\right]_{m\times n}=\left[\begin{matrix} \dfrac{\mathrm{d}a_{11}(t)}{\mathrm{d}t} & \dfrac{\mathrm{d}a_{12}(t)}{\mathrm{d}t} & \cdots & \dfrac{\mathrm{d}a_{1n}(t)}{\mathrm{d}t} \\ \\ \dfrac{\mathrm{d}a_{21}(t)}{\mathrm{d}t} & \dfrac{\mathrm{d}a_{22}(t)}{\mathrm{d}t} & \cdots & \dfrac{\mathrm{d}a_{2n}(t)}{\mathrm{d}t} \\ \\ \vdots & \vdots & \cdots & \vdots \\ \\ \dfrac{\mathrm{d}a_{m1}(t)}{\mathrm{d}t} & \dfrac{\mathrm{d}a_{m2}(t)}{\mathrm{d}t} & \cdots & \dfrac{\mathrm{d}a_{mn}(t)}{\mathrm{d}t} \\ \end{matrix}\right]dtdA(t)=[dtdaij(t)]m×n=⎣⎡dtda11(t)dtda21(t)⋮dtdam1(t)dtda12(t)dtda22(t)⋮dtdam2(t)⋯⋯⋯⋯dtda1n(t)dtda2n(t)⋮dtdamn(t)⎦⎤

\qquad

当 m=1m=1m=1 时，矩阵 A(t)=[a1(t),a2(t),⋯,an(t)]\boldsymbol A(t)=[a_1(t),a_2(t),\cdots,a_n(t)]A(t)=[a1(t),a2(t),⋯,an(t)] 为（行）向量值函数

dA(t)dt=[daj(t)dt]1×n=[da1(t)dtda2(t)dt⋯dan(t)dt]1×n\qquad\qquad \dfrac{\mathrm{d}\boldsymbol A(t)}{\mathrm{d}t}=\left[\dfrac{\mathrm{d}a_{j}(t)}{\mathrm{d}t}\right]_{1\times n}=\left[\begin{matrix} \dfrac{\mathrm{d}a_{1}(t)}{\mathrm{d}t} & \dfrac{\mathrm{d}a_{2}(t)}{\mathrm{d}t} & \cdots & \dfrac{\mathrm{d}a_{n}(t)}{\mathrm{d}t} \\ \end{matrix}\right]_{1\times n}dtdA(t)=[dtdaj(t)]1×n=[dtda1(t)dtda2(t)⋯dtdan(t)]1×n

\qquad
当 n=1n=1n=1 时，矩阵 A(t)=[a1(t),a2(t),⋯,am(t)]T\boldsymbol A(t)=[a_1(t),a_2(t),\cdots,a_m(t)]^TA(t)=[a1(t),a2(t),⋯,am(t)]T 为（列）向量值函数
dA(t)dt=[dai(t)dt]m×1=[da1(t)dtda2(t)dt⋮dam(t)dt]m×1\qquad\qquad \dfrac{\mathrm{d}\boldsymbol A(t)}{\mathrm{d}t}=\left[\dfrac{\mathrm{d}a_{i}(t)}{\mathrm{d}t}\right]_{m\times 1}=\left[\begin{matrix} \dfrac{\mathrm{d}a_{1}(t)}{\mathrm{d}t} \\ \\ \dfrac{\mathrm{d}a_{2}(t)}{\mathrm{d}t} \\ \\ \vdots\\ \\ \dfrac{\mathrm{d}a_{m}(t)}{\mathrm{d}t}\\ \end{matrix}\right]_{m\times 1}dtdA(t)=[dtdai(t)]m×1=⎣⎡dtda1(t)dtda2(t)⋮dtdam(t)⎦⎤m×1

\qquad

2.多元函数对矩阵的导数

\qquad设矩阵 X=[xij]m×n\bold X=[x_{ij}]_{m\times n}X=[xij]m×n，考虑该矩阵的 mnmnmn 元函数 f(X)=f(x11,x12,⋯,xm1,xm2,⋯,xmn)f(\bold X)=f(x_{11},x_{12},\cdots,x_{m1},x_{m2},\cdots,x_{mn})f(X)=f(x11,x12,⋯,xm1,xm2,⋯,xmn)，那么 f(X)f(\bold X)f(X) 对矩阵 X\bold XX 的导数定义为：

df(X)dX=[∂f∂xij]m×n=[∂f∂x11∂f∂x12⋯∂f∂x1n∂f∂x21∂f∂x22⋯∂f∂x2n⋮⋮⋯⋮∂f∂xm1∂f∂xm2⋯∂f∂xmn]\qquad\qquad \dfrac{\mathrm{d}f(\bold X)}{\mathrm{d}\bold X}=\left[\dfrac{\partial f}{\partial x_{ij}}\right]_{m\times n}=\left[\begin{matrix} \dfrac{\partial f}{\partial x_{11}} & \dfrac{\partial f}{\partial x_{12}} & \cdots & \dfrac{\partial f}{\partial x_{1n}} \\ \\ \dfrac{\partial f}{\partial x_{21}} & \dfrac{\partial f}{\partial x_{22}} & \cdots & \dfrac{\partial f}{\partial x_{2n}} \\ \\ \vdots & \vdots & \cdots & \vdots \\ \\ \dfrac{\partial f}{\partial x_{m1}} & \dfrac{\partial f}{\partial x_{m2}} & \cdots & \dfrac{\partial f}{\partial x_{mn}} \\ \end{matrix}\right]dXdf(X)=[∂xij∂f]m×n=⎣⎡∂x11∂f∂x21∂f⋮∂xm1∂f∂x12∂f∂x22∂f⋮∂xm2∂f⋯⋯⋯⋯∂x1n∂f∂x2n∂f⋮∂xmn∂f⎦⎤

\qquad

3.多元函数对(列)向量的导数

\qquad设 nnn 维（列）向量 x=[x1,x2,⋯,xn]T\boldsymbol x=[x_1,x_2,\cdots,x_n]^Tx=[x1,x2,⋯,xn]T，考虑该向量的 nnn 元函数 f(x)=f(x1,x2,⋯,xn)f(\boldsymbol x)=f(x_{1},x_{2},\cdots,x_{n})f(x)=f(x1,x2,⋯,xn)，那么：

df(x)dx=[∂f∂x1,∂f∂x2,⋯,∂f∂xn]T=[∂f∂x1∂f∂x2⋮∂f∂xn]\qquad\qquad \dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}=\left[\dfrac{\partial f}{\partial x_1},\dfrac{\partial f}{\partial x_2},\cdots,\dfrac{\partial f}{\partial x_n}\right]^T=\left[\begin{matrix}\dfrac{\partial f}{\partial x_1}\\ \\ \dfrac{\partial f}{\partial x_2}\\ \\ \vdots\\ \\ \dfrac{\partial f}{\partial x_n}\end{matrix}\right]dxdf(x)=[∂x1∂f,∂x2∂f,⋯,∂xn∂f]T=⎣⎡∂x1∂f∂x2∂f⋮∂xn∂f⎦⎤，即：f(x)f(\boldsymbol x)f(x) 的梯度 ∇f(x)=df(x)dx\nabla f(\boldsymbol x)=\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}∇f(x)=dxdf(x)

df(x)dxT=[∂f∂x1,∂f∂x2,⋯,∂f∂xn]\qquad\qquad \dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x^T}=\left[\dfrac{\partial f}{\partial x_1},\dfrac{\partial f}{\partial x_2},\cdots,\dfrac{\partial f}{\partial x_n}\right]dxTdf(x)=[∂x1∂f,∂x2∂f,⋯,∂xn∂f]，即：f(x)f(\boldsymbol x)f(x) 的梯度的转置 ∇Tf(x)=df(x)dxT\nabla^T f(\boldsymbol x)=\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x^T}∇Tf(x)=dxTdf(x)
\qquad

\qquad因此∇f(x)=df(x)dx=[df(x)dxT]T\qquad\nabla f(\boldsymbol x)=\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}=\left[\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x^T}\right]^T∇f(x)=dxdf(x)=[dxTdf(x)]T
\qquad

常用公式

(1)\qquad(1)(1) 海塞 (Hessian)\text{(Hessian)}(Hessian) 矩阵：

\qquad　∇T{∇f(x)}=ddxT(df(x)dx)\nabla^T \{\nabla f(\boldsymbol x)\}=\dfrac{\mathrm{d}}{\mathrm{d}\boldsymbol x^T}\left(\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}\right)∇T{∇f(x)}=dxTd(dxdf(x))　或　∇{∇Tf(x)}=ddx(df(x)dxT)\nabla \{\nabla^T f(\boldsymbol x)\}=\dfrac{\mathrm{d}}{\mathrm{d}\boldsymbol x}\left(\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x^T}\right)∇{∇Tf(x)}=dxd(dxTdf(x))

\qquad
ddxT(dfdx)=[∂2f∂x12∂2f∂x1∂x2⋯∂2f∂x1∂xn∂2f∂x2∂x1∂2f∂x22⋯∂2f∂x2∂xn⋮⋮⋱⋮∂2f∂xn∂x1∂2f∂xn∂x2⋯∂2f∂xn2]\qquad\qquad\qquad \dfrac{\mathrm{d}}{\mathrm{d}\boldsymbol x^T}\left(\dfrac{\mathrm{d}f}{\mathrm{d}\boldsymbol x}\right)=\left[\begin{matrix} \dfrac{\partial^2 f}{\partial x_1^2} & \dfrac{\partial^2 f}{\partial x_1\partial x_2} & \cdots & \dfrac{\partial^2 f}{\partial x_1\partial x_n} \\ \\ \dfrac{\partial^2 f}{\partial x_2\partial x_1} & \dfrac{\partial^2 f}{\partial x_2^2} & \cdots & \dfrac{\partial^2 f}{\partial x_2\partial x_n} \\ \\ \vdots & \vdots & \ddots & \vdots \\ \\ \dfrac{\partial^2 f}{\partial x_n\partial x_1} & \dfrac{\partial^2 f}{\partial x_n\partial x_2} & \cdots & \dfrac{\partial^2 f}{\partial x_n^2} \\ \end{matrix}\right]dxTd(dxdf)=⎣⎡∂x12∂2f∂x2∂x1∂2f⋮∂xn∂x1∂2f∂x1∂x2∂2f∂x22∂2f⋮∂xn∂x2∂2f⋯⋯⋱⋯∂x1∂xn∂2f∂x2∂xn∂2f⋮∂xn2∂2f⎦⎤
\qquad

(2)\qquad(2)(2) 二次函数 f(x)=xTAxf(\boldsymbol x)=\boldsymbol x^T \boldsymbol A \boldsymbol xf(x)=xTAx 的导数为 df(x)dx=(A+AT)x\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}=(\boldsymbol A+\boldsymbol A^T )\boldsymbol xdxdf(x)=(A+AT)x

\quad　　　若 A=[aij]n×n\boldsymbol A=[a_{ij}]_{n\times n}A=[aij]n×n 为对称矩阵，那么 df(x)dx=2Ax\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}=2\boldsymbol A \boldsymbol xdxdf(x)=2Ax

\qquad　证明：
f(x)=xTAx=∑i=1n∑j=1naijxixj=x1∑j=1na1jxj+x2∑j=1na2jxj+⋯+xk∑j=1nakjxj+⋯+xn∑j=1nanjxj\qquad\qquad\qquad \begin{aligned}f(\boldsymbol x)&=\boldsymbol x^T \boldsymbol A \boldsymbol x=\displaystyle\sum_{i=1}^{n}\displaystyle\sum_{j=1}^{n}a_{ij}x_ix_j \\ &=x_1\displaystyle\sum_{j=1}^{n}a_{1j}x_j +x_2\displaystyle\sum_{j=1}^{n}a_{2j}x_j+\cdots +x_k\displaystyle\sum_{j=1}^{n}a_{kj}x_j+\cdots+x_n\displaystyle\sum_{j=1}^{n}a_{nj}x_j \\ \end{aligned}f(x)=xTAx=i=1∑nj=1∑naijxixj=x1j=1∑na1jxj+x2j=1∑na2jxj+⋯+xkj=1∑nakjxj+⋯+xnj=1∑nanjxj

∂f∂xk=x1a1k+x2a2k+⋯+(∑j=1nakjxj+xkakk)+⋯+xnank=(x1a1k+x2a2k+⋯+xkakk+⋯+xnank)+∑j=1nakjxj=∑i=1naikxi+∑j=1nakjxj\qquad\qquad\qquad \begin{aligned}\dfrac{\partial f}{\partial x_k}&=x_1a_{1k}+x_2a_{2k}+\cdots+\left(\displaystyle\sum_{j=1}^{n}a_{kj}x_j+x_ka_{kk}\right)+\cdots+x_na_{nk}\\ &=(x_1a_{1k}+x_2a_{2k}+\cdots+x_ka_{kk}+\cdots+x_na_{nk}) +\displaystyle\sum_{j=1}^{n}a_{kj}x_j \\ &=\displaystyle\sum_{i=1}^{n}a_{ik}x_i +\displaystyle\sum_{j=1}^{n}a_{kj}x_j \end{aligned}∂xk∂f=x1a1k+x2a2k+⋯+(j=1∑nakjxj+xkakk)+⋯+xnank=(x1a1k+x2a2k+⋯+xkakk+⋯+xnank)+j=1∑nakjxj=i=1∑naikxi+j=1∑nakjxj

df(x)dx=[∂f∂x1⋮∂f∂xk⋮∂f∂xn]=[∑i=1nai1xi+∑j=1na1jxj⋮∑i=1naikxi+∑j=1nakjxj⋮∑i=1nainxi+∑j=1nanjxj]=[∑i=1nai1xi⋮∑i=1naikxi⋮∑i=1nainxi]+[∑j=1na1jxj⋮∑j=1nakjxj⋮∑j=1nanjxj]=Ax+ATx=(A+AT)x\qquad\qquad\qquad\begin{aligned} \dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}&=\left[\begin{matrix}\dfrac{\partial f}{\partial x_1}\\ \\ \vdots\\ \\ \dfrac{\partial f}{\partial x_k}\\ \\ \vdots\\ \\ \dfrac{\partial f}{\partial x_n}\end{matrix}\right]=\left[\begin{matrix}\displaystyle\sum_{i=1}^{n}a_{i1}x_i +\displaystyle\sum_{j=1}^{n}a_{1j}x_j\\ \\ \vdots\\ \\ \displaystyle\sum_{i=1}^{n}a_{ik}x_i +\displaystyle\sum_{j=1}^{n}a_{kj}x_j\\ \\ \vdots\\ \\ \displaystyle\sum_{i=1}^{n}a_{in}x_i +\displaystyle\sum_{j=1}^{n}a_{nj}x_j \end{matrix}\right]=\left[\begin{matrix}\displaystyle\sum_{i=1}^{n}a_{i1}x_i \\ \\ \vdots\\ \\ \displaystyle\sum_{i=1}^{n}a_{ik}x_i \\ \\ \vdots\\ \\ \displaystyle\sum_{i=1}^{n}a_{in}x_i \end{matrix}\right]+\left[\begin{matrix}\displaystyle\sum_{j=1}^{n}a_{1j}x_j\\ \\ \vdots\\ \\ \displaystyle\sum_{j=1}^{n}a_{kj}x_j\\ \\ \vdots\\ \\ \displaystyle\sum_{j=1}^{n}a_{nj}x_j \end{matrix}\right] \\ &=\boldsymbol A\boldsymbol x+\boldsymbol A^T\boldsymbol x \\ &=(\boldsymbol A +\boldsymbol A^T)\boldsymbol x \\ \end{aligned}dxdf(x)=⎣⎡∂x1∂f⋮∂xk∂f⋮∂xn∂f⎦⎤=⎣⎡i=1∑nai1xi+j=1∑na1jxj⋮i=1∑naikxi+j=1∑nakjxj⋮i=1∑nainxi+j=1∑nanjxj⎦⎤=⎣⎡i=1∑nai1xi⋮i=1∑naikxi⋮i=1∑nainxi⎦⎤+⎣⎡j=1∑na1jxj⋮j=1∑nakjxj⋮j=1∑nanjxj⎦⎤=Ax+ATx=(A+AT)x
\qquad

(3)\qquad(3)(3) 线性函数 f(x)=bTxf(\boldsymbol x)=\boldsymbol b^T \boldsymbol xf(x)=bTx 的导数为 df(x)dx=b\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}=\boldsymbol bdxdf(x)=b，或者 df(x)dxT=bT\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x^T}=\boldsymbol b^TdxTdf(x)=bT

\quad　　　若假设 b\boldsymbol bb 为变量，由于 bTx=xTb\boldsymbol b^T \boldsymbol x= \boldsymbol x^T \boldsymbol bbTx=xTb，因此 df(b)db=x\dfrac{\mathrm{d}f(\boldsymbol b)}{\mathrm{d}\boldsymbol b}=\boldsymbol xdbdf(b)=x

\qquad　证明：　f(x)=bTx=∑i=1nbixif(\boldsymbol x) =\boldsymbol b^T \boldsymbol x=\displaystyle\sum_{i=1}^{n}b_ix_if(x)=bTx=i=1∑nbixi

df(x)dx=[∂f∂x1⋮∂f∂xk⋮∂f∂xn]=[b1⋮bk⋮bn]=b\qquad\qquad\qquad \dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x}=\left[\begin{matrix}\dfrac{\partial f}{\partial x_1}\\ \\ \vdots\\ \\ \dfrac{\partial f}{\partial x_k}\\ \\ \vdots\\ \\ \dfrac{\partial f}{\partial x_n}\end{matrix}\right]= \left[\begin{matrix} b_1\\ \\ \vdots\\ \\ b_k\\ \\ \vdots\\ \\ b_n\end{matrix}\right]=\boldsymbol bdxdf(x)=⎣⎡∂x1∂f⋮∂xk∂f⋮∂xn∂f⎦⎤=⎣⎡b1⋮bk⋮bn⎦⎤=b
\qquad

\qquad

4.一元函数关于向量的复合求导

\qquad设向量值函数 x(t)=[x1(t),x2(t),⋯,xn(t)]T\boldsymbol x(t)=[x_1(t),x_2(t),\cdots,x_n(t)]^Tx(t)=[x1(t),x2(t),⋯,xn(t)]T，考虑该向量函数的一元函数 f(x(t))=f(x1(t),x2(t),⋯,xn(t))f(\boldsymbol x(t))=f(x_1(t),x_2(t),\cdots,x_n(t))f(x(t))=f(x1(t),x2(t),⋯,xn(t))，那么：

dfdt=[dfdx]Tdxdt=dfdxTdxdt\qquad\qquad\dfrac{\mathrm{d}f}{\mathrm{d}t}=\left[\dfrac{\mathrm{d}f}{\mathrm{d}\boldsymbol x}\right]^T\dfrac{\mathrm{d}\boldsymbol x}{\mathrm{d}t}=\dfrac{\mathrm{d}f}{\mathrm{d}\boldsymbol x^T}\dfrac{\mathrm{d}\boldsymbol x}{\mathrm{d}t}dtdf=[dxdf]Tdtdx=dxTdfdtdx

\qquad又由于 ∇Tf(x)=df(x)dxT\nabla^T f(\boldsymbol x)=\dfrac{\mathrm{d}f(\boldsymbol x)}{\mathrm{d}\boldsymbol x^T}∇Tf(x)=dxTdf(x)，因此 dfdt=dfdxTdxdt=∇Tf(x)dxdt\dfrac{\mathrm{d}f}{\mathrm{d}t}=\dfrac{\mathrm{d}f}{\mathrm{d}\boldsymbol x^T}\dfrac{\mathrm{d}\boldsymbol x}{\mathrm{d}t}=\nabla^T f(\boldsymbol x)\dfrac{\mathrm{d}\boldsymbol x}{\mathrm{d}t}dtdf=dxTdfdtdx=∇Tf(x)dtdx

\qquad证明：

dfdt=∂f∂x1dx1dt+∂f∂x2dx2dt+⋯+∂f∂xndxndt=[∂f∂x1,∂f∂x2,⋯,∂f∂xn][dx1dtdx2dt⋮dxndt]=[dfdx]Tdxdt=dfdxTdxdt\qquad\qquad \begin{aligned}\dfrac{\mathrm{d}f}{\mathrm{d}t}&=\dfrac{\partial f}{\partial x_1}\dfrac{\mathrm{d}x_1}{\mathrm{d}t}+\dfrac{\partial f}{\partial x_2}\dfrac{\mathrm{d}x_2}{\mathrm{d}t}+\cdots+\dfrac{\partial f}{\partial x_n}\dfrac{\mathrm{d}x_n}{\mathrm{d}t}\\ &=\left[\dfrac{\partial f}{\partial x_1},\dfrac{\partial f}{\partial x_2},\cdots,\dfrac{\partial f}{\partial x_n}\right] \left[\begin{matrix}\dfrac{\mathrm{d} x_1}{\mathrm{d} t}\\ \\ \dfrac{\mathrm{d} x_2}{\mathrm{d} t}\\ \\ \vdots\\ \\ \dfrac{\mathrm{d} x_n}{\mathrm{d} t}\end{matrix}\right]=\left[\dfrac{\mathrm{d}f}{\mathrm{d}\boldsymbol x}\right]^T\dfrac{\mathrm{d}\boldsymbol x}{\mathrm{d}t}=\dfrac{\mathrm{d}f}{\mathrm{d}\boldsymbol x^T}\dfrac{\mathrm{d}\boldsymbol x}{\mathrm{d}t}\\ \end{aligned}dtdf=∂x1∂fdtdx1+∂x2∂fdtdx2+⋯+∂xn∂fdtdxn=[∂x1∂f,∂x2∂f,⋯,∂xn∂f]⎣⎡dtdx1dtdx2⋮dtdxn⎦⎤=[dxdf]Tdtdx=dxTdfdtdx
\qquad

5. 泰勒级数

\qquad首先考虑二维的情况，即 x=[x1,x2]T\boldsymbol x=[x_1,x_2]^Tx=[x1,x2]T，那么

f(x1+δ1,x2+δ2)=f(x1,x2)+∂f∂x1δ1+∂f∂x2δ2+12(∂2f∂x12δ12+∂2f∂x1∂x2δ1δ2+∂2f∂x22δ22)+o(∥δ∥2)\qquad\qquad\begin{aligned}f(x_1+\delta_1,x_2+\delta_2)&=f(x_1,x_2)+\dfrac{\partial f}{\partial x_1}\delta_1+\dfrac{\partial f}{\partial x_2}\delta_2\\ &\quad+\dfrac{1}{2}\left( \dfrac{\partial^2 f}{\partial x_1^2}\delta_1^2+\dfrac{\partial^2 f}{\partial x_1\partial x_2}\delta_1\delta_2+\dfrac{\partial^2 f}{\partial x_2^2}\delta_2^2 \right) \\ &\quad+o\left(\Vert\boldsymbol\delta\Vert^2\right) \end{aligned}f(x1+δ1,x2+δ2)=f(x1,x2)+∂x1∂fδ1+∂x2∂fδ2+21(∂x12∂2fδ12+∂x1∂x2∂2fδ1δ2+∂x22∂2fδ22)+o(∥δ∥2)

\qquad扩展到 nnn 维的情况，即 x=[x1,x2,⋯,xn]T\boldsymbol x=[x_1,x_2,\cdots,x_n]^Tx=[x1,x2,⋯,xn]T，那么

f(x1+δ1,x2+δ2,⋯,xn+δn)=f(x1,x2,⋯,xn)+∑i=1n∂f∂xiδi+12∑i=1n∑j=1n∂2f∂xi∂xjδiδj+o(∥δ∥2)\qquad\qquad \begin{aligned}f(x_1+\delta_1,x_2+\delta_2,\cdots,x_n+\delta_n)&=f(x_1,x_2,\cdots,x_n)+\displaystyle\sum_{i=1}^n\dfrac{\partial f}{\partial x_i}\delta_i \\ &\quad+\dfrac{1}{2}\displaystyle\sum_{i=1}^n\displaystyle\sum_{j=1}^n\dfrac{\partial^2 f}{\partial x_i\partial x_j}\delta_i\delta_j\\ &\quad+o\left(\Vert\boldsymbol\delta\Vert^2\right) \end{aligned}f(x1+δ1,x2+δ2,⋯,xn+δn)=f(x1,x2,⋯,xn)+i=1∑n∂xi∂fδi+21i=1∑nj=1∑n∂xi∂xj∂2fδiδj+o(∥δ∥2)

\qquad
\qquad写成矩阵的形式：

f(x+δ)=f(x)+∇f(x)Tδ+12δT∇2f(x)δ+o(∥δ∥2)\qquad\qquad f(\boldsymbol x+\boldsymbol\delta)=f(\boldsymbol x)+\nabla f(\boldsymbol x)^T\boldsymbol\delta+\dfrac{1}{2}\boldsymbol\delta^T\nabla^2 f(\boldsymbol x)\boldsymbol\delta+o\left(\Vert\boldsymbol\delta\Vert^2\right)f(x+δ)=f(x)+∇f(x)Tδ+21δT∇2f(x)δ+o(∥δ∥2)，其中 δ=[δ1,δ2,⋯,δn]T\boldsymbol\delta=[\delta_1,\delta_2,\cdots,\delta_n]^Tδ=[δ1,δ2,⋯,δn]T

\qquad
\qquad或者，写成向量值函数 f(x)f(\boldsymbol x)f(x) 在点 xˉ\bar{\boldsymbol x}xˉ 的展开形式：

f(x)=f(xˉ)+∇f(xˉ)T(x−xˉ)+12(x−xˉ)T∇2f(xˉ)(x−xˉ)+o(∥x−xˉ∥2)\qquad\qquad f(\boldsymbol x)=f(\bar{\boldsymbol x})+\nabla f(\bar{\boldsymbol x})^T(\boldsymbol x-\bar{\boldsymbol x})+\dfrac{1}{2}(\boldsymbol x-\bar{\boldsymbol x})^T\nabla^2 f(\bar{\boldsymbol x})(\boldsymbol x-\bar{\boldsymbol x})+o\left(\Vert\boldsymbol x-\bar{\boldsymbol x}\Vert^2\right)f(x)=f(xˉ)+∇f(xˉ)T(x−xˉ)+21(x−xˉ)T∇2f(xˉ)(x−xˉ)+o(∥x−xˉ∥2)

\qquad【注】此处采用 ∇f(x)\nabla f(\boldsymbol x)∇f(x) 表示梯度，采用 ∇2f(x)\nabla^2 f(\boldsymbol x)∇2f(x) 表示 hessian\text{hessian}hessian 矩阵（而非 PDE\text{PDE}PDE 中的拉普拉斯算符）。

矩阵微分常用公式整理相关推荐

excel常用公式整理
数据分析中,常常会用到Excel来分析处理数据,使用函数可以辅助我们更好的分析处理数据以下是整理的常用公式: 将时间戳转换为日期(其中的i2是要被转换的单元格): =TEXT((LEFT(i2,10 ...
高数部分常用公式整理（扫描版——个人整理）
概率密度变换公式雅可比矩阵_机器人雅可比矩阵的理解和常用公式
雅可比矩阵就是一种特殊的矩阵导数考虑一个简单的问题: .其中x是自变量,y是因变量.对两边求导可得: .这里的表示可以使式子成立的一个和a有关的一个值. 将上面的问题拓展到多维情况,可以得到: , ...
[机器学习-数学] 矩阵求导(分母布局与分子布局)，以及常用的矩阵求导公式
一, 矩阵求导 1,矩阵求导的本质矩阵A对矩阵B求导: 矩阵A中的每一个元素分别对矩阵B中的每个元素进行求导. A1×1A_{1\times1}A1×1, B1×1B_{1\times1}B1×1 ...
【转载】矩阵求导、几种重要的矩阵及常用的矩阵求导公式
一.矩阵求导一般来讲,我们约定x=(x1,x2,-xN)Tx=(x1,x2,-xN)T,这是分母布局.常见的矩阵求导方式有:向量对向量求导,标量对向量求导,向量对标量求导. 1.向量对向量求导 Nu ...
DL4J的矩阵处理模块ND4J的常用函数整理
ND4J矩阵变换的常用函数整理由本人阅读nd4j源码整理,nd4j是dl4j为了矩阵运算整出来的一套工具.对应python里的numpy,但是并没有numpy那么普及,不过至少使用dl4j搭建神经网 ...
矩阵求导(分母布局与分子布局)，以及常用的矩阵求导公式
神经网络反向传播时的梯度到底怎么求, 注意与下图矩阵求导的本质的区分:神经网络的在反向传播过程中,矩阵(m×n)对矩阵(p×q)求导的结果是只列出对相关元素求偏导的结果,其他不相关的元素根本就没有对应 ...
矩阵求导常用公式（避坑）+矩阵的模和矩阵的绝对值的求导
目录矩阵求导常用公式 1.分母布局与分子布局 2.分母布局与分子布局的矩阵求导公式 (1)向量对向量求导 (2).标量对向量求导 (3).向量对标量求导 3.验证求导结果矩阵的模和矩阵的绝对值的求 ...
矩阵计算 | 常用矩阵求导公式速查
文章目录参考资料常用矩阵求导公式参考资料 Matrix Calculu The Matrix Cookbook 常用矩阵求导公式对于一个矩阵A,向量x\mathrm{x}x,有如下求导公式: ...
[计算数学基础]矩阵微分
矩阵微分函数对于变量的微分在高等数学里面讲的比较多,而矩阵微分在我印象中没有在高等代数中讲解.矩阵微分也是很常用的一个数学工具,我最早是在一门研究生课程"优化设计"中接触到,优化 ...

矩阵微分常用公式整理

矩阵微分常用公式整理

1.矩阵的导数

2.多元函数对矩阵的导数

3.多元函数对(列)向量的导数

常用公式

4.一元函数关于向量的复合求导

5. 泰勒级数

矩阵微分常用公式整理相关推荐

最新文章

热门文章