【Machine Learning 学习笔记】Stochastic Dual Coordinate Ascent for SVM 代码实现

通过本篇博客记录一下Stochastic Dual Coordinate Ascent for SVM 代码实现，数据集使用sklearn的breast cancer。

Dual representation of the optimal hyperplane

Dual Soft-Margin SVM

Kernel trick

Algorithm

import sys
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.svm import SVC
from sklearn.metrics import f1_score, auc, roc_curve, roc_auc_score# load the data
X, y = load_breast_cancer(return_X_y=True)  # X input, y output
print(X.shape, y.shape)# to convert the {0,1} output into {-1,+1}
y = 2 * y - 1# X is the input matrix
mdata, ndim = X.shape
# normalization by L infinity norm
scaling = 1
if scaling == 1:    # L infinity normX /= np.outer(np.ones(mdata), np.max(np.abs(X), 0))
elif scaling == 2:  # L2 normx_norm = np.sqrt(np.sum(X ** 2, 1))x_norm += (x_norm == 0)X /= np.outer(x_norm, np.ones(X.shape[1]))# number of iteration
niter = 10# penalty constant for the of the Stochastic Dual Coordinate Ascent algorithm
C = 1000# dual stochastic gradient algorithm
# initialize alpha
alpha = np.zeros(mdata)# compute the linear kernel
K = np.dot(X, X.T)for iter in range(niter):for i in range(mdata):# process all sample examples sequentially# sum on i!=j alpha[i] = (1-y[i]*(np.sum(K[i]*y*alpha)-alpha[i]*y[i]*K[i,i]))/K[i, i]# Clip to satisfy the constraintsalpha[i] = min(C/mdata, max(0, alpha[i]))# accuracy of the full data
X_test = X
m_test = X.shape[0]# Linear kernel
K_cross = np.dot(X, X_test.T)
# Stochastic Dual Coordinate Ascent for SVM prediction
ysdca = np.sign(np.sum(np.outer(y*alpha, np.ones(m_test))*K_cross, 0))sdca_f1 = f1_score(y, ysdca)
sdca_auc = roc_auc_score(y, ysdca)
tsdca_roc = roc_curve(y, ysdca)
sdca_auc2 = auc(tsdca_roc[0], tsdca_roc[1])# result
print("Dual stochastic gradient F1: {}".format(sdca_f1))
print("Dual stochastic gradient AUC: {}".format(sdca_auc))
print("Dual stochastic gradient AUC2: {}".format(sdca_auc2))

result

Dual stochastic gradient F1: 0.9633649932157395
Dual stochastic gradient AUC: 0.9382366154008773
Dual stochastic gradient AUC2: 0.9382366154008773

【Machine Learning 学习笔记】Stochastic Dual Coordinate Ascent for SVM 代码实现相关推荐

【Machine Learning 学习笔记】feature engineering中noisy feature的影响
[Machine Learning 学习笔记]feature engineering中noisy feature的影响通过本篇博客记录一下添加噪声对Lasso和SVM的影响,采用的数据集为sklea ...
[Python Machine Learning] 学习笔记之scikit-learn机器学习库
1. scikit-learn介绍 scikit-learn是Python的一个开源机器学习模块,它建立在NumPy,SciPy和matplotlib模块之上.值得一提的是,scikit-learn最 ...
Machine Learning 学习笔记1 - 基本概念以及各分类
What is machine learning? 并没有广泛认可的定义来准确定义机器学习.以下定义均为译文,若以后有时间,将补充原英文...... 定义1.来自Arthur Samuel(上世纪50 ...
IBM Machine Learning学习笔记（一）——Exploratory Data Analysis for Machine Learning
数据的探索性分析 1. 读入数据 (1)csv文件读取 (2)json文件读取 (3)SQL数据库读取 (4)Not-only SQL (NoSQL)读取 (5)从网络中获取 2. 数据清洗 (1)缺 ...
Machine Learning学习笔记（十）K-means聚类算法
K-Means介绍 K-means算法是聚类分析中使用最广泛的算法之一.它把n个对象根据他们的属性分为k个聚类以便使得所获得的聚类满足:同一聚类中的对象相似度较高:而不同聚类中的对象相似度较小.其聚类 ...
IBM Machine Learning学习笔记（二）——Supervised Learning: Regression
文章目录一.Introduction to Supervised Machine Learning 二.Data Splits and Cross Validation 三.Regression w ...
Machine Learning学习笔记（四）EML极限学习机
EML定义极限学习机器( Extreme Learning Machine,ELM) 是神经网络研究中的一种算法,是一种泛化的单隐层前馈神经网络( Single-hidden Layer Feed ...
Georgia Tech - machine learning 学习笔记一
机器学习的分类 1.监督学习(supervised learning) 获取已标记的数据集,通过标记的数据集来收集信息,以便能标记新的数据集,也就是一个函数逼近的过程. 所有的机器学习,当然也包括监督 ...
machine learning学习笔记
1. 做classfication 时候一定要将label 取成是balanced的,也就是一比一的比例,因为如果不是一比一的比例的话,举个极端一点的例子:如果pos:neg label目前是9:1的 ...

【Machine Learning 学习笔记】Stochastic Dual Coordinate Ascent for SVM 代码实现

【Machine Learning 学习笔记】Stochastic Dual Coordinate Ascent for SVM 代码实现

Dual representation of the optimal hyperplane

Dual Soft-Margin SVM

Kernel trick

Algorithm

result

【Machine Learning 学习笔记】Stochastic Dual Coordinate Ascent for SVM 代码实现相关推荐

最新文章

热门文章