机器学习__

一.带有随机隐藏节点的单隐层前馈神经网络

　1.相关条件：

N个不同样本(xi,tix_i,t_i), xix_i =[xi1,xi2,xi3,........,xin]T[x_{i1},x_{i2},x_{i3},........,x_{in}]^T, tit_i =[ti1,ti2,ti3,........,tim]T[t_{i1},t_{i2},t_{i3},........,t_{im}]^T
第i个隐藏节点和输入节点间的权重向量：wi=[wi1,wi2,........win]Tw_i=[w_{i1},w_{i2},........w_{in}]^T
第i个隐藏节点的阀值：bi=[bi1,bi2,........bin]Tb_i=[b_{i1},b_{i2},........b_{in}]^T
第i个隐藏节点和输出节点间的权重向量：βi=[βi1,βi2,........βin]T\beta_i=[\beta_{i1},\beta_{i2},........\beta_{in}]^T
激活函数：g(x)

　2.方程改写：

SLFNs（单隐层前馈神经网络）： ∑N‘i=1βig(wi∗xj+bi)\sum_{i=1}^{N^`}\beta_i g(w_i *x_j + b_i) = tjt_j, 简写成：Hβ\beta=TT
其中

　3.简易模型：

　4.两个相关定理(可以自行证明)：

定理 1：给定一个具有N个隐藏节点以及在任何区间都无限可导的激活函数的标准SLFN。对N个任意不同样本，，SLFN在随机产生的情况下，形成的隐藏层输出矩阵H是可逆的，且
定理 2. 对于任意小的，及在任何区间都无限可导的激活函数，对N个任意不同样本，，总存在个隐节点的SLFN，使得在随机产生的情况下。

二.SLFNs的最小范数的最小二乘(LS)

由定理1，2可知：只要激活函数无限可导，输入权重和隐藏层阀值可以随机分配, (即：可认为wi,βiw_i,\beta_i已知)，因此训练SLFNs等价于找到Hβ\beta=TT的一个最小二乘解 β∗\beta^*
其中根据相容方程组：β∗=H+T\beta^* = H^+T 　 (H+H^+是H的广义逆)

三.ELM算法

　给定训练样本集合N个不同样本(xi,tix_i,t_i)，激活函数g(x)和隐藏单元个数N∗N^*

任意指定输入权值和阈值wi,bi(i=1,....N∗)w_i,b_i (i = 1,....N^*)；
计算隐藏层输出矩阵H；
计算输出权重β\beta: β∗=H+T\beta^* = H^+ T
其中T=[t1,t2,.......tN]TT = [t_1,t_2,.......t_N]^T

四.代码

function [TrainingTime, TestingTime, TrainingAccuracy, TestingAccuracy] = elm(TrainingData_File, TestingData_File, Elm_Type, NumberofHiddenNeurons, ActivationFunction)% Usage: elm(TrainingData_File, TestingData_File, Elm_Type, NumberofHiddenNeurons, ActivationFunction)
% OR:    [TrainingTime, TestingTime, TrainingAccuracy, TestingAccuracy] = elm(TrainingData_File, TestingData_File, Elm_Type, NumberofHiddenNeurons, ActivationFunction)
%
% Input:
% TrainingData_File     - Filename of training data set
% TestingData_File      - Filename of testing data set
% Elm_Type              - 0 for regression; 1 for (both binary and multi-classes) classification
% NumberofHiddenNeurons - Number of hidden neurons assigned to the ELM
% ActivationFunction    - Type of activation function:
%                           'sig' for Sigmoidal function
%                           'sin' for Sine function
%                           'hardlim' for Hardlim function
%                           'tribas' for Triangular basis function
%                           'radbas' for Radial basis function (for additive type of SLFNs instead of RBF type of SLFNs)
%
% Output:
% TrainingTime          - Time (seconds) spent on training ELM
% TestingTime           - Time (seconds) spent on predicting ALL testing data
% TrainingAccuracy      - Training accuracy:
%                           RMSE for regression or correct classification rate for classification
% TestingAccuracy       - Testing accuracy:
%                           RMSE for regression or correct classification rate for classification
%
% MULTI-CLASSE CLASSIFICATION: NUMBER OF OUTPUT NEURONS WILL BE AUTOMATICALLY SET EQUAL TO NUMBER OF CLASSES
% FOR EXAMPLE, if there are 7 classes in all, there will have 7 output
% neurons; neuron 5 has the highest output means input belongs to 5-th class
%
% Sample1 regression: [TrainingTime, TestingTime, TrainingAccuracy, TestingAccuracy] = elm('sinc_train', 'sinc_test', 0, 20, 'sig')
% Sample2 classification: elm('diabetes_train', 'diabetes_test', 1, 20, 'sig')
%%%%%    Authors:    MR QIN-YU ZHU AND DR GUANG-BIN HUANG%%%%    NANYANG TECHNOLOGICAL UNIVERSITY, SINGAPORE%%%%    EMAIL:      EGBHUANG@NTU.EDU.SG; GBHUANG@IEEE.ORG%%%%    WEBSITE:    http://www.ntu.edu.sg/eee/icis/cv/egbhuang.htm%%%%    DATE:       APRIL 2004%%%%%%%%%%% Macro definition
REGRESSION=0;
CLASSIFIER=1;%%%%%%%%%%% Load training dataset
train_data=load(TrainingData_File);
T=train_data(:,1)';
P=train_data(:,2:size(train_data,2))';
clear train_data;                                   %   Release raw training data array%%%%%%%%%%% Load testing dataset
test_data=load(TestingData_File);
TV.T=test_data(:,1)';
TV.P=test_data(:,2:size(test_data,2))';
clear test_data;                                    %   Release raw testing data arrayNumberofTrainingData=size(P,2);
NumberofTestingData=size(TV.P,2);
NumberofInputNeurons=size(P,1);if Elm_Type~=REGRESSION%%%%%%%%%%%% Preprocessing the data of classificationsorted_target=sort(cat(2,T,TV.T),2);label=zeros(1,1);                               %   Find and save in 'label' class label from training and testing data setslabel(1,1)=sorted_target(1,1);j=1;for i = 2:(NumberofTrainingData+NumberofTestingData)if sorted_target(1,i) ~= label(1,j)j=j+1;label(1,j) = sorted_target(1,i);endendnumber_class=j;NumberofOutputNeurons=number_class;%%%%%%%%%% Processing the targets of trainingtemp_T=zeros(NumberofOutputNeurons, NumberofTrainingData);for i = 1:NumberofTrainingDatafor j = 1:number_classif label(1,j) == T(1,i)break; endendtemp_T(j,i)=1;endT=temp_T*2-1;%%%%%%%%%% Processing the targets of testingtemp_TV_T=zeros(NumberofOutputNeurons, NumberofTestingData);for i = 1:NumberofTestingDatafor j = 1:number_classif label(1,j) == TV.T(1,i)break; endendtemp_TV_T(j,i)=1;endTV.T=temp_TV_T*2-1;end                                                 %   end if of Elm_Type%%%%%%%%%%% Calculate weights & biases
start_time_train=cputime;%%%%%%%%%%% Random generate input weights InputWeight (w_i) and biases BiasofHiddenNeurons (b_i) of hidden neurons
InputWeight=rand(NumberofHiddenNeurons,NumberofInputNeurons)*2-1;
BiasofHiddenNeurons=rand(NumberofHiddenNeurons,1);
tempH=InputWeight*P;
clear P;                                            %   Release input of training data
ind=ones(1,NumberofTrainingData);
BiasMatrix=BiasofHiddenNeurons(:,ind);              %   Extend the bias matrix BiasofHiddenNeurons to match the demention of H
tempH=tempH+BiasMatrix;%%%%%%%%%%% Calculate hidden neuron output matrix H
switch lower(ActivationFunction)case {'sig','sigmoid'}%%%%%%%% Sigmoid H = 1 ./ (1 + exp(-tempH));case {'sin','sine'}%%%%%%%% SineH = sin(tempH);    case {'hardlim'}%%%%%%%% Hard LimitH = double(hardlim(tempH));case {'tribas'}%%%%%%%% Triangular basis functionH = tribas(tempH);case {'radbas'}%%%%%%%% Radial basis functionH = radbas(tempH);%%%%%%%% More activation functions can be added here
end
clear tempH;                                        %   Release the temparary array for calculation of hidden neuron output matrix H%%%%%%%%%%% Calculate output weights OutputWeight (beta_i)
OutputWeight=pinv(H') * T';                        % slower implementation
%OutputWeight=inv(eye(size(H,1))/C+H * H') * H * T';   % faster method 1
%implementation; one can set regularizaiton factor C properly in classification applications
%OutputWeight=(eye(size(H,1))/C+H * H') \ H * T';      % faster method 2
%implementation; one can set regularizaiton factor C properly in classification applications%If you use faster methods or kernel method, PLEASE CITE in your paper properly: %Guang-Bin Huang, Hongming Zhou, Xiaojian Ding, and Rui Zhang, "Extreme Learning Machine for Regression and Multi-Class Classification," submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence, October 2010. end_time_train=cputime;
TrainingTime=end_time_train-start_time_train        %   Calculate CPU time (seconds) spent for training ELM%%%%%%%%%%% Calculate the training accuracy
Y=(H' * OutputWeight)';                             %   Y: the actual output of the training data
if Elm_Type == REGRESSIONTrainingAccuracy=sqrt(mse(T - Y))               %   Calculate training accuracy (RMSE) for regression case
end
clear H;%%%%%%%%%%% Calculate the output of testing input
start_time_test=cputime;
tempH_test=InputWeight*TV.P;
clear TV.P;             %   Release input of testing data
ind=ones(1,NumberofTestingData);
BiasMatrix=BiasofHiddenNeurons(:,ind);              %   Extend the bias matrix BiasofHiddenNeurons to match the demention of H
tempH_test=tempH_test + BiasMatrix;
switch lower(ActivationFunction)case {'sig','sigmoid'}%%%%%%%% Sigmoid H_test = 1 ./ (1 + exp(-tempH_test));case {'sin','sine'}%%%%%%%% SineH_test = sin(tempH_test);        case {'hardlim'}%%%%%%%% Hard LimitH_test = hardlim(tempH_test);        case {'tribas'}%%%%%%%% Triangular basis functionH_test = tribas(tempH_test);        case {'radbas'}%%%%%%%% Radial basis functionH_test = radbas(tempH_test);        %%%%%%%% More activation functions can be added here
end
TY=(H_test' * OutputWeight)';                       %   TY: the actual output of the testing data
end_time_test=cputime;
TestingTime=end_time_test-start_time_test           %   Calculate CPU time (seconds) spent by ELM predicting the whole testing dataif Elm_Type == REGRESSIONTestingAccuracy=sqrt(mse(TV.T - TY))            %   Calculate testing accuracy (RMSE) for regression case
endif Elm_Type == CLASSIFIER
%%%%%%%%%% Calculate training & testing classification accuracyMissClassificationRate_Training=0;MissClassificationRate_Testing=0;for i = 1 : size(T, 2)[x, label_index_expected]=max(T(:,i));[x, label_index_actual]=max(Y(:,i));if label_index_actual~=label_index_expectedMissClassificationRate_Training=MissClassificationRate_Training+1;endendTrainingAccuracy=1-MissClassificationRate_Training/size(T,2)for i = 1 : size(TV.T, 2)[x, label_index_expected]=max(TV.T(:,i));[x, label_index_actual]=max(TY(:,i));if label_index_actual~=label_index_expectedMissClassificationRate_Testing=MissClassificationRate_Testing+1;endendTestingAccuracy=1-MissClassificationRate_Testing/size(TV.T,2)
end

机器学习___ELM相关推荐

机器学习分类指标：精确率、准确率、召回率详解
混淆矩阵在介绍具体的定义之前先了解一些混淆矩阵(confusion matrix): 一种 NxN 表格,用于总结分类模型的预测效果:即标签和模型预测的分类之间的关联.在混淆矩阵中,一个轴表示模型预 ...
【机器学习】RNN循环神经网络
循环神经网络归属: 领域:机器学习方向:自然语言处理贡献:自动文本生成循环神经网络实际应用: 生活中因为原始数据都是序列化的,比如自然语言,语音处理,时间序列问题(股票价格)等问题, 这个时候需 ...
开源自动化机器学习框架
20211101 在 Airbnb 使用机器学习预测房源的价格 https://blog.csdn.net/weixin_33735077/article/details/87976278?spm=1 ...
机器学习常用术语词汇表
EOF是一个计算机术语,为End Of File的缩写 ,在操作系统中表示资料源无更多的资料可读取. 刚接触机器学习框架 TensorFlow 的新手们,这篇由 Google 官方出品的常用术语词汇表 ...
预见未来丨机器学习：未来十年研究热点
<h2 class="subheader">机器学习:未来十年研究热点 </h2><div class="gray-d1-c margin- ...
SMOTE算法代码实现-机器学习
类别不平衡问题类别不平衡问题,顾名思义,即数据集中存在某一类样本,其数量远多于或远少于其他类样本,从而导致一些机器学习模型失效的问题.例如逻辑回归即不适合处理类别不平衡问题,例如逻辑回归在欺诈检测问 ...
LARS 算法简介-机器学习
https://cosx.org/2011/04/modified-lars-and-lasso/ 查看全文 http://www.taodudu.cc/news/show-64111.html 相关 ...
1-1 机器学习和深度学习综述-paddle
课程>我的课程>百度架构师手把手教深度学习>1-1 机器学习和深度学习综述> 1-1 机器学习和深度学习综述 paddle初级课程王然(学生) Notebook 教育初级深 ...
机器学习——标准化/归一化的目的、作用和场景
对每个特征进行归一化处理,使得每个特征的取值缩放到0~1之间.这样做有两个好处: 模型训练更高效. 特征前的权重大小可代表该变量对预测结果的贡献度(因为每个特征值本身的范围相同). (一)归一化的作用 ...

机器学习___ELM

一.带有随机隐藏节点的单隐层前馈神经网络

1.相关条件：

2.方程改写：

3.简易模型：

4.两个相关定理(可以自行证明)：

二.SLFNs的最小范数的最小二乘(LS)

三.ELM算法

四.代码

机器学习___ELM相关推荐

最新文章

热门文章