Matlab深度学习笔记——深度学习工具箱说明

本文是Rasmus Berg Palm发布在Github上的Deep-learning toolbox的说明文件，作者对这个工具箱进行了详细的介绍（原文链接：https://github.com/rasmusbergpalm/DeepLearnToolbox#deeplearntoolbox点击打开链接）。

警告：

这个工具箱已经过时，并且不再被保留。

有很多比这个工具箱更好的工具可用于深度学习，如 Theano, torch 或者 tensorflow。

我建议您使用上面提到的工具之一，而不是使用这个工具箱。

祝好，Rasmus

DeepLearnToolbox——一个用于深度学习的Matlab工具箱

深度学习是机器学习的一个新领域，它的重点是学习深层次的数据模型。它的灵感来自于人脑表面的深度分层体系结构。深度学习理论的一个很好的概述是学习人工智能的深层架构。
想得到更多相关介绍，请看下面的Geoffrey Hinton和Andrew Ng的视频。
The Next Generation of Neural Networks (Hinton, 2007)
Recent Developments in Deep Learning (Hinton, 2010)
Unsupervised Feature Learning and Deep Learning (Ng, 2011)

工具箱中包含的目录

NN/ - 一个前馈BP神经网络的库
CNN/ - 一个卷积神经网络的库
DBN/ - 一个深度置信网络的库

SAE/ - 一个堆栈自动编码器的库

CAE/ - 一个卷积自动编码器的库
util/ - 库中使用的效用函数
data/ - 例子所用到的数据

tests/ - 用来验证工具箱正在工作的测试

设置

1.下载

2.addpath(genpath('DeepLearnToolbox'));

例：深度置信网络Deep Belief Network

function test_example_DBN
load mnist_uint8;train_x = double(train_x) / 255;
test_x  = double(test_x)  / 255;
train_y = double(train_y);
test_y  = double(test_y);%%  ex1 train a 100 hidden unit RBM and visualize its weights
rand('state',0)
dbn.sizes = [100];
opts.numepochs =   1;
opts.batchsize = 100;
opts.momentum  =   0;
opts.alpha     =   1;
dbn = dbnsetup(dbn, train_x, opts);
dbn = dbntrain(dbn, train_x, opts);
figure; visualize(dbn.rbm{1}.W');   %  Visualize the RBM weights%%  ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NN
rand('state',0)
%train dbn
dbn.sizes = [100 100];
opts.numepochs =   1;
opts.batchsize = 100;
opts.momentum  =   0;
opts.alpha     =   1;
dbn = dbnsetup(dbn, train_x, opts);
dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nn
nn = dbnunfoldtonn(dbn, 10);
nn.activation_function = 'sigm';%train nn
opts.numepochs =  1;
opts.batchsize = 100;
nn = nntrain(nn, train_x, train_y, opts);
[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');

例：堆叠式自动编码器Stacked Auto-Encoders

function test_example_SAE
load mnist_uint8;train_x = double(train_x)/255;
test_x  = double(test_x)/255;
train_y = double(train_y);
test_y  = double(test_y);%%  ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN
%  Setup and train a stacked denoising autoencoder (SDAE)
rand('state',0)
sae = saesetup([784 100]);
sae.ae{1}.activation_function       = 'sigm';
sae.ae{1}.learningRate              = 1;
sae.ae{1}.inputZeroMaskedFraction   = 0.5;
opts.numepochs =   1;
opts.batchsize = 100;
sae = saetrain(sae, train_x, opts);
visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNN
nn = nnsetup([784 100 10]);
nn.activation_function              = 'sigm';
nn.learningRate                     = 1;
nn.W{1} = sae.ae{1}.W{1};% Train the FFNN
opts.numepochs =   1;
opts.batchsize = 100;
nn = nntrain(nn, train_x, train_y, opts);
[er, bad] = nntest(nn, test_x, test_y);
assert(er < 0.16, 'Too big error');
Example: Convolutional Neural Netsfunction test_example_CNN
load mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;
test_x = double(reshape(test_x',28,28,10000))/255;
train_y = double(train_y');
test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network
%will run 1 epoch in about 200 second and get around 11% error.
%With 100 epochs you'll get around 1.2% error
rand('state',0)
cnn.layers = {struct('type', 'i') %input layerstruct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layerstruct('type', 's', 'scale', 2) %sub sampling layerstruct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layerstruct('type', 's', 'scale', 2) %subsampling layer
};
cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;
opts.batchsize = 50;
opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared error
figure; plot(cnn.rL);assert(er<0.12, 'Too big error');

例：神经网络Neural Networks

function test_example_NN
load mnist_uint8;train_x = double(train_x) / 255;
test_x  = double(test_x)  / 255;
train_y = double(train_y);
test_y  = double(test_y);% normalize
[train_x, mu, sigma] = zscore(train_x);
test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural net
rand('state',0)
nn = nnsetup([784 100 10]);
opts.numepochs =  1;   %  Number of full sweeps through data
opts.batchsize = 100;  %  Take a mean gradient step over this many samples
[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decay
rand('state',0)
nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4;  %  L2 weight decay
opts.numepochs =  1;        %  Number of full sweeps through data
opts.batchsize = 100;       %  Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);
assert(er < 0.1, 'Too big error');%% ex3 neural net with dropout
rand('state',0)
nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5;   %  Dropout fraction
opts.numepochs =  1;        %  Number of full sweeps through data
opts.batchsize = 100;       %  Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);
assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation function
rand('state',0)
nn = nnsetup([784 100 10]);nn.activation_function = 'sigm';    %  Sigmoid activation function
nn.learningRate = 1;                %  Sigm require a lower learning rate
opts.numepochs =  1;                %  Number of full sweeps through data
opts.batchsize = 100;               %  Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);
assert(er < 0.1, 'Too big error');%% ex5 plotting functionality
rand('state',0)
nn = nnsetup([784 20 10]);
opts.numepochs         = 5;            %  Number of full sweeps through data
nn.output              = 'softmax';    %  use softmax output
opts.batchsize         = 1000;         %  Take a mean gradient step over this many samples
opts.plot              = 1;            %  enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);
assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error
% split training data into training and validation data
vx   = train_x(1:10000,:);
tx = train_x(10001:end,:);
vy   = train_y(1:10000,:);
ty = train_y(10001:end,:);rand('state',0)
nn                      = nnsetup([784 20 10]);
nn.output               = 'softmax';                   %  use softmax output
opts.numepochs          = 5;                           %  Number of full sweeps through data
opts.batchsize          = 1000;                        %  Take a mean gradient step over this many samples
opts.plot               = 1;                           %  enable plotting
nn = nntrain(nn, tx, ty, opts, vx, vy);                %  nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);
assert(er < 0.1, 'Too big error');