1 Making the dual

min⁡xcTxs.t.Ax≤b\begin{aligned} \min_x& \quad c^Tx\\ s.t.& \quad Ax\leq b \end{aligned}xmins.t.cTxAx≤b

⇔\Leftrightarrow⇔

min⁡xmax⁡p≤0cTx+pT(b−Ax)\min_x \max_{p\leq 0} c^Tx+p^T(b-Ax)minxmaxp≤0cTx+pT(b−Ax) ≥\geq≥

max⁡p≤0min⁡xcTx+pT(b−Ax)\max_{p\leq 0} \min_x c^Tx+p^T(b-Ax)maxp≤0minxcTx+pT(b−Ax) ⇔\Leftrightarrow⇔ max⁡p≥0min⁡x(cT−pTA)x+pTb\max_{p\geq 0} \min_x (c^T-p^TA)x+p^Tbmaxp≥0minx(cT−pTA)x+pTb

⇔\Leftrightarrow⇔

min⁡xbTps.t.ATp=c\begin{aligned} \min_x& \quad b^Tp\\ s.t.& \quad A^Tp=c \end{aligned}xmins.t.bTpATp=c

The constraints on penalty variables and dual constraints are determined by making the corresponding terms zero.

2 Weak duality

For any function f(x,y)f(x,y)f(x,y):
min⁡xmax⁡yf(x,y)≥max⁡ymin⁡xf(x,y)\min_x \max_y f(x,y)\geq \max_y \min_xf(x,y)xminymaxf(x,y)≥ymaxxminf(x,y)

Assume min⁡xcTx\min_x c^TxminxcTx and max⁡ppTb\max_p p^TbmaxppTb, and xxx and ppp are feasible, then cTx≥pTbc^Tx\geq p^TbcTx≥pTb

3 Strong duality

If both primal and dual are feasible, and
primal optimum x∗x^*x∗
dual optimum p∗p^*p∗
cTx∗=bTp∗c^{T}x^*=b^Tp^{*}cTx∗=bTp∗

4 Applying duality

If either the primal or the dual is feasible and has a bounded optimal solution, then so is the other, and the values are equal. (Strong duality)

If the primal is unbounded the dual is infeasible. (weak duality)

If the dual is unbounded the primal is infeasible. (weak duality)

It is possible for both to be infeasible.

If the dual is feasible, the primal is either feasible, or infeasible, but cannot be unbounded. (weak duality)

5 Complementray slackness

Complementary slackness holds as written, for any formulation of primal and dual.

If a constraint in the primal optimal (resp. dual) is not tight, then the corresponding dual optimal (resp. primal) variable must be equal to zero.
If a variable in the primal optimal (resp. dual) is not equal to zero, then the corresponding constraint in the dual optimal (resp. primal) must be tight.

min⁡xcTxs.t.Ax≥bx≥0\begin{aligned} \min_x& \quad c^Tx\\ s.t.& \quad Ax\geq b\\ &\quad x\geq 0 \end{aligned}xmins.t.cTxAx≥bx≥0 \quad\quad\quad min⁡xbTys.t.ATy≤cy≥0\begin{aligned} \min_x& \quad b^Ty\\ s.t.& \quad A^Ty\leq c\\ &\quad y\geq0 \end{aligned}xmins.t.bTyATy≤cy≥0

Therom

If xxx is primal feasible
yyy is dual feasible
Then x,yx,yx,y are (respectively) optimal iff:
(bi−∑jaijxj)yi=0(b_i-\sum_j a_{ij}x_j)y_i=0(bi−∑jaijxj)yi=0, (b−Ax)∗y(b-Ax)*y(b−Ax)∗y, for all iii
(cj−∑iaijyi)xj=0(c_j-\sum_i a_{ij}y_i)x_j=0(cj−∑iaijyi)xj=0, (c−ATy)∗x(c-A^Ty)*x(c−ATy)∗x for all jjj

Optimization Week 4: Duality相关推荐

浅析SVM中的对偶问题
浅析SVM中的对偶问题关于SVM对偶问题求解的博客有很多,但是关于为什么要进行对偶问题的分析却很零散,这里做一个总结 1. 为什么要研究对偶问题? 广义上讲,将原问题的研究转换为对偶问题的研究主要有 ...
Levenberg-Marquardt(LM算法)的理解
Levenberg-Marquardt LM算法的理解 1. convex optimization 1.1 convex set 1.2 convex function 1.3 optimizat ...
机器学习数学基础——最优化理论
以下文章摘录自 <机器学习观止--核心原理与实践> 京东: https://item.jd.com/13166960.html 当当:http://product.dangdang.com ...
实用线性代数和凸优化 Convex Optimization
If not specified, the following conditions are assumed. X∈Rn∗mA∈Rm∗nX \in R^{n*m} \\ A \in R^{m*n} X ...
SVM——（三）对偶性和KKT条件（Lagrange duality and KKT condition）
之前说到过拉格朗日乘数法以及推导过程,那么今天要说的就是拉格朗日对偶性以及KKT条件 1.Lagrange multipliers 一句话说,拉格朗日乘数法就是用来解决条件极值的一个方法,且约束条件都 ...
支持向量机_6：Duality
在之前关于 support vector 的推导中,我们提到了 dual ,这里再来补充一点相关的知识.这套理论不仅适用于 SVM 的优化问题,而是对于所有带约束的优化问题都适用的,是优化理论中的一个 ...
RN-Integer Programming:Lagrangian Relaxation and Duality Theory
Content 1 Lagrangian Relaxation 2 Continuous Relaxation and Dual Relaxation 3 Dual Search 4 Conclusi ...
Convex Optimization: Primal Problem to Dual problem clearly explained 详细
往期文章链接目录文章目录往期文章链接目录 The Lagrange dual function Lower bound property Derive an analytical expressi ...
TVM 图优化Graph Optimization
TVM 图优化Graph Optimization Codegen
向量算子优化Vector Operation Optimization
向量算子优化Vector Operation Optimization 查看MATLAB命令View MATLAB Command 示例显示Simulink®编码器™ ,将生成向量的块输出,设置为标量 ...

Optimization Week 4: Duality

Week 4:Duality of Linear Programming, LP