Programming Exercise 6:Support Vector Machines

大家好，我是Mac Jiang，今天和大家分享一下coursera网站上Stanford University的Machine Learning公开课（吴恩达老师）课程第六次作业：Programming Exercise 6:Support Vector Machines。写这篇博客的目的是为在课程学习中遇到困难的同学提供一些帮助，同时帮助自己巩固这周的课程内容。欢迎博友转载此文章，但希望您在转载之前与我联系并标明文章的出处，谢谢！

由于Programming Exercise 6的作业内容可以分为两大块，即1Support Vector Machines 2Span Classification.其中，第一块主要是描述SVM算法的具体实现过程，是本周课程内容的基础，第二块垃圾邮件分类是基于第一块代码的基础上的具体实际应用。由于文章篇幅有限，本文只讲述第一部分Support Vector Machines的实现过程，第二部分的内容我将在下一篇博客中给出，地址为http://blog.csdn.net/a1015553840/article/details/50826728。

好的，话不多说，开始讲述第一部分的实现过程。

数据集有：ex6data1.mat

ex6data2.mat

ex6data3.mat

实现过程与函数：ex6.m ---控制实验的进程，不用修改

svmTrain.m ---训练SVM算法，是已经开发完善的包，不用修改，直接调用

svmPredict.m ---对新样本进行预测，已开发的包，不用修改，直接调用

plotData.m ----绘制2D图像，不用修改，直接调用

visualizeBoundaryLinear,visualizeBoundaty.m ----可视化决策线，不用修改，直接调用

linearKernel.m ----线性核函数（无核函数），是以完善的包，不用修改，直接调用

guassianKernel.m ----高斯核函数，需要修改

dataset3Params.m ---这个是计算最优C,sigma的函数，需要修改

所以，这部分实验我们实际上只要修改两个文件就可以了，非常简单。

1.好的，我们先看算法的实现流程，实现流程是在ex6.m这个文件中：

%% Initialization
clear ; close all; clc
%% =============== Part 1: Loading and Visualizing Data ================
%  We start the exercise by first loading and visualizing the dataset.
%  The following code will load the dataset into your environment and plot
%  the data.
%fprintf('Loading and Visualizing Data ...\n')% Load from ex6data1:
% You will have X, y in your environment
load('ex6data1.mat');% Plot training data
plotData(X, y);fprintf('Program paused. Press enter to continue.\n');
pause;%% ==================== Part 2: Training Linear SVM ====================
%  The following code will train a linear SVM on the dataset and plot the
%  decision boundary learned.
%% Load from ex6data1:
% You will have X, y in your environment
load('ex6data1.mat');fprintf('\nTraining Linear SVM ...\n')% You should try to change the C value below and see how the decision
% boundary varies (e.g., try C = 1000)
C = 1;
model = svmTrain(X, y, C, @linearKernel, 1e-3, 20);
visualizeBoundaryLinear(X, y, model);fprintf('Program paused. Press enter to continue.\n');
pause;%% =============== Part 3: Implementing Gaussian Kernel ===============
%  You will now implement the Gaussian kernel to use
%  with the SVM. You should complete the code in gaussianKernel.m
%
fprintf('\nEvaluating the Gaussian Kernel ...\n')x1 = [1 2 1]; x2 = [0 4 -1]; sigma = 2;
sim = gaussianKernel(x1, x2, sigma);fprintf(['Gaussian Kernel between x1 = [1; 2; 1], x2 = [0; 4; -1], sigma = 0.5 :' ...'\n\t%f\n(this value should be about 0.324652)\n'], sim);fprintf('Program paused. Press enter to continue.\n');
pause;%% =============== Part 4: Visualizing Dataset 2 ================
%  The following code will load the next dataset into your environment and
%  plot the data.
%fprintf('Loading and Visualizing Data ...\n')% Load from ex6data2:
% You will have X, y in your environment
load('ex6data2.mat');% Plot training data
plotData(X, y);fprintf('Program paused. Press enter to continue.\n');
pause;%% ========== Part 5: Training SVM with RBF Kernel (Dataset 2) ==========
%  After you have implemented the kernel, we can now use it to train the
%  SVM classifier.
%
fprintf('\nTraining SVM with RBF Kernel (this may take 1 to 2 minutes) ...\n');% Load from ex6data2:
% You will have X, y in your environment
load('ex6data2.mat');% SVM Parameters
C = 1; sigma = 0.1;% We set the tolerance and max_passes lower here so that the code will run
% faster. However, in practice, you will want to run the training to
% convergence.
model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma));
visualizeBoundary(X, y, model);fprintf('Program paused. Press enter to continue.\n');
pause;%% =============== Part 6: Visualizing Dataset 3 ================
%  The following code will load the next dataset into your environment and
%  plot the data.
%fprintf('Loading and Visualizing Data ...\n')% Load from ex6data3:
% You will have X, y in your environment
load('ex6data3.mat');% Plot training data
plotData(X, y);fprintf('Program paused. Press enter to continue.\n');
pause;%% ========== Part 7: Training SVM with RBF Kernel (Dataset 3) ==========%  This is a different dataset that you can use to experiment with. Try
%  different values of C and sigma here.
% % Load from ex6data3:
% You will have X, y in your environment
load('ex6data3.mat');% Try different SVM Parameters here
[C, sigma] = dataset3Params(X, y, Xval, yval);% Train the SVM
model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma));
visualizeBoundary(X, y, model);fprintf('Program paused. Press enter to continue.\n');
pause;

可以看到，实现过程可以分为七个部分：

part1:loading and visualizing data ---导入ex6data1.mat数据集，并做出样本分布图

part2:Traning linear SVM -利用已存在的linearKernal.m和svmTrian.m对数据集1训练线性SVM，做出决策线

part3:Impletementing Gaussian Kernel -调用guassian Kernel计算x1,x2之间的相似程度，此处需完善guassianKernel.m

part4:Visualizing Dataset 2 -导入ex6data2.mat，对立面的数据进行可视化，即做出样本分布图

part5:Training SVM with RBF Kernel(Dataset 2) - 利用guassiankernel.m ,svmTrain.m对数据集2计算SVM，做出决策线

part6:Visual Dataset 3 - 导入ex6data3.mat,对数据进行可视化，及做出样本分布图

part7:Training SVM with RBF kernel(dataset 3) -在dataset3Params.m中计算得到最优参数C，sigma,并利用它训练高斯核函数的SVM，做出决策线，此处要完善dataset3Params.m

注意：part1,part2针对的是ex6dataset1.mat,利用的是线性核函数，目的是让我们了解线性核函数的作用和使用过程

part3,part4,part5针对的是ex6dataset2.mat，利用的是高斯核函数，目的是让我们了解高斯核函数的作用和使用过程

part6,part7针对的是ex6dataset3.mat，利用的是高斯核函数，给出8种C的值，8种sigma的值，共64种组合，通过计算错误率找出最优组合

2.完善guassianKernel.m（此处需要编写代码！！！）

function sim = gaussianKernel(x1, x2, sigma)
%RBFKERNEL returns a radial basis function kernel between x1 and x2
%   sim = gaussianKernel(x1, x2) returns a gaussian kernel between x1 and x2
%   and returns the value in sim% Ensure that x1 and x2 are column vectors
x1 = x1(:); x2 = x2(:);% You need to return the following variables correctly.
sim = 0;
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return the similarity between x1
%               and x2 computed using a Gaussian kernel with bandwidth
%               sigma
%
%
x12 = x1 - x2;                                            %计算x1 - x2的向量，放在x12中
temp = -x12' * x12 / (2 * sigma * sigma);                 %计算指数部分，其中x1 - x2的模的平方可以利用向量x12'*x12的方式计算，计算速度快
sim = exp(temp);                                          %计算Guassian Kernel值
% =============================================================
end

这里注意，计算x(i) - x(j)的模的平方时候可以直接用x12'*x12向量计算（x12表示x(i)-x(j)）,利用向量的方法可以加快计算速度。

3.完善dataset3Params.m(此处需要完善代码！！！)

function [C, sigma] = dataset3Params(X, y, Xval, yval)
% You need to return the following variables correctly.
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return the optimal C and sigma
%               learning parameters found using the cross validation set.
%               You can use svmPredict to predict the labels on the cross
%               validation set. For example,
%                   predictions = svmPredict(model, Xval);
%               will return the predictions on the cross validation set.
%
%  Note: You can compute the prediction error using
%        mean(double(predictions ~= yval))
%
C_array = [0.01;0.03;0.1;0.3;1;3;10;30];     %C数组
sigma_array = [0.01;0.03;0.1;0.3;1;3;10;30]; %sigma数组
error_array = zeros(8,8);                    %创造错误率数组，第（i，j）个元素表示C（i），sigma(j)对应的错误率
error_min = 10000;                           %记录当前最小错误率for i = 1:8,for j = 1:8,model= svmTrain(X, y, C_array(i), @(x1, x2) gaussianKernel(x1, x2, sigma_array(j)));   %以C(i),sigma(j)为参数训练SVM，（X，y）为训练样本predictions = svmPredict(model, Xval);                                     %利用上面训练得到的model对交叉验证样本进行预测error_array(i,j) =  mean(double(predictions ~= yval));                     %记录错误率if(error_array(i,j) < error_min)                                           %如果当前的错误率更小，则记录他，并记录此时的C和sigmaerror_min = error_array(i,j);C = C_array(i);sigma = sigma_array(j);endend
end
fprintf('The training C    The training sigma    error\n');                 %打印出不同C,sigma下对应的误差率，方便我们查找验证算法的正确性
for i = 1:8,for j = 1:8,fprintf('%f   %f   %f\n',C_array(i),sigma_array(j),error_array(i,j));               %打印C(i)，sigma(j)对应的误差率end
end
fprintf('%f and %f perform best, the error is %f',C,sigma,error_min);                      %打印找到的最优C,sigma，以及得到的最小错误率
% =========================================================================
end

实验要求给出8钟C和sigma的值[0.01,0.03,0.1,0.3,1,3,10,30],C与sigma之间有8*8=64种组合，实验要求对这64种组合分别利用训练样本（X,y）进行训练，得到训练model,然后利用交叉验证样本（Xval,yval）计算训练得到的模型对交叉验证样本的预测错误率，选择64种组合中错误率最小的一种。

实验得到的C,sigma,error的训练结果为：

这样我们就完成了第一大块 Support Vector Machines的实验代码编写，第二大Spam Classification的实验代码我将在接下来的博客中给出。可能我的写的代码不是最好的，如果同学们有什么更好的想法，请留言联系，谢谢！

From：http://blog.csdn.net/a1015553840/article/details/50824397

Programming Exercise 6:Support Vector Machines相关推荐

Ex6_机器学习_吴恩达课程作业（Python）：SVM支持向量机（Support Vector Machines）
Ex6_机器学习_吴恩达课程作业(Python):SVM支持向量机(Support Vector Machines) 文章目录 Ex6_机器学习_吴恩达课程作业(Python):SVM支持向量机(Su ...
OpenCV3.3中支持向量机(Support Vector Machines, SVM)实现简介及使用
OpenCV 3.3中给出了支持向量机(Support Vector Machines)的实现,即cv::ml::SVM类,此类的声明在include/opencv2/ml.hpp文件中,实现在mod ...
Machine Learning week 7 quiz: Support Vector Machines
Support Vector Machines 5 试题 1. Suppose you have trained an SVM classifier with a Gaussian kernel, a ...
【原】Coursera—Andrew Ng机器学习—课程笔记 Lecture 12—Support Vector Machines 支持向量机...
Lecture 12 支持向量机 Support Vector Machines 12.1 优化目标 Optimization Objective 支持向量机(Support Vector Machi ...
初译 Support Vector Machines：A Simple Tutorial（一）
<Support Vector Machines:A Simple Tutorial> 作者 Alexey Nefedov (一)Introduction 在本章节中将会介绍一些用于定 ...
机器学习之支持向量机： Support Vector Machines (SVM)
机器学习之支持向量机: Support Vector Machines (SVM) 欢迎访问人工智能研究网课程中心网址是:http://i.youku.com/studyai 本篇博客介绍机器学习 ...
论文阅读和分析：Hybrid Mathematical Symbol Recognition using Support Vector Machines
HMER论文系列 1.论文阅读和分析:When Counting Meets HMER Counting-Aware Network for HMER_KPer_Yang的博客-CSDN博客 2.论文 ...
Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines 论文研读
摘要本文提出了一种用于训练支持向量机的新算法:序列最小优化算法(SMO).训练支持向量机需要解决非常大的二次规划(QP)优化问题.SMO 将这个大的 QP 问题分解为一系列最小的 QP 问题.这些小 ...
支持向量机（Support Vector Machines，SVM）
文章目录 1. 线性可分SVM 与硬间隔最大化 1.1 线性可分SVM 1.2 函数间隔.几何间隔 1.3 间隔最大化 2. 线性SVM 与软间隔最大化 2.1 线性SVM 3. 非线性SVM 与 ...

Programming Exercise 6:Support Vector Machines

Programming Exercise 6:Support Vector Machines相关推荐

最新文章

热门文章