DL之DNN：利用MultiLayerNetExtend模型【6*100+ReLU+SGD,dropout】对Mnist数据集训练来抑制过拟合

输出结果

设计思路

核心代码

更多输出

输出结果

设计思路

190417更新

核心代码

class RMSprop:def __init__(self, lr=0.01, decay_rate = 0.99):self.lr = lrself.decay_rate = decay_rateself.h = Nonedef update(self, params, grads):if self.h is None:self.h = {}for key, val in params.items():self.h[key] = np.zeros_like(val)for key in params.keys():self.h[key] *= self.decay_rateself.h[key] += (1 - self.decay_rate) * grads[key] * grads[key]params[key] -= self.lr * grads[key] / (np.sqrt(self.h[key]) + 1e-7)class Nesterov:def __init__(self, lr=0.01, momentum=0.9):self.lr = lrself.momentum = momentumself.v = Nonedef update(self, params, grads):if self.v is None:self.v = {}for key, val in params.items():self.v[key] = np.zeros_like(val)for key in params.keys():self.v[key] *= self.momentumself.v[key] -= self.lr * grads[key]params[key] += self.momentum * self.momentum * self.v[key]params[key] -= (1 + self.momentum) * self.lr * grads[key]use_dropout = True
dropout_ratio = 0.2network = MultiLayerNetExtend(input_size=784, hidden_size_list=[100, 100, 100, 100, 100, 100],output_size=10, use_dropout=use_dropout, dropout_ration=dropout_ratio)
trainer = Trainer(network, x_train, t_train, x_test, t_test, epochs=301, mini_batch_size=100,optimizer='sgd', optimizer_param={'lr': 0.01}, verbose=True)
trainer.train()
train_acc_list, test_acc_list = trainer.train_acc_list, trainer.test_acc_list

更多输出

1、DNN[6*100+ReLU,SGD]: accuracy of not dropout on Minist dataset

train loss:2.3364575765992637
=== epoch:1, train acc:0.10333333333333333, test acc:0.1088 ===
train loss:2.414526554119518
train loss:2.341182306768928
train loss:2.3072782723352496
=== epoch:2, train acc:0.09666666666666666, test acc:0.1103 ===
train loss:2.2600377181768887
train loss:2.263350960525319
train loss:2.2708260374887645……=== epoch:298, train acc:1.0, test acc:0.7709 ===
train loss:0.00755416896470134
train loss:0.009934657874546435
train loss:0.008421672959852643
=== epoch:299, train acc:1.0, test acc:0.7712 ===
train loss:0.007142981215285884
train loss:0.008205245499586114
train loss:0.007319626293763803
=== epoch:300, train acc:1.0, test acc:0.7707 ===
train loss:0.00752230499930163
train loss:0.008431046288276818
train loss:0.008067532729014863
=== epoch:301, train acc:1.0, test acc:0.7707 ===
train loss:0.010729407851274233
train loss:0.007776889701033221
=============== Final Test Accuracy ===============
test acc:0.771

2、DNN[6*100+ReLU,SGD]: accuracy of dropout(0.2) on Minist dataset

train loss:2.3064018541384437
=== epoch:1, train acc:0.11, test acc:0.1112 ===
train loss:2.316626942558816
train loss:2.314434337198633
train loss:2.318862771955365
=== epoch:2, train acc:0.11333333333333333, test acc:0.1128 ===
train loss:2.3241989320140717
train loss:2.317694982413387
train loss:2.3079716553885006……=== epoch:298, train acc:0.6266666666666667, test acc:0.5168 ===
train loss:1.2359381134877185
train loss:1.2833380447791383
train loss:1.2728131428100005
=== epoch:299, train acc:0.63, test acc:0.52 ===
train loss:1.1687601000183936
train loss:1.1435412548991142
train loss:1.3854277174616834
=== epoch:300, train acc:0.6333333333333333, test acc:0.5244 ===
train loss:1.3039470016588997
train loss:1.2359979876607923
train loss:1.2871396654831204
=== epoch:301, train acc:0.63, test acc:0.5257 ===
train loss:1.1690084424502523
train loss:1.1820777530873694
=============== Final Test Accuracy ===============
test acc:0.5269

DL之DNN：利用MultiLayerNetExtend模型【6*100+ReLU+SGD,dropout】对Mnist数据集训练来抑制过拟合相关推荐

DL之DNN：利用MultiLayerNet模型【6*100+ReLU+SGD】对Mnist数据集训练来理解过拟合现象
DL之DNN:利用MultiLayerNet模型[6*100+ReLU+SGD]对Mnist数据集训练来理解过拟合现象导读自定义少量的Mnist数据集,利用全连接神经网络MultiLayerNet ...
DL之DNN：自定义MultiLayerNet【6*100+ReLU,SGD】对MNIST数据集训练进而比较【多个超参数组合最优化】性能
DL之DNN:自定义MultiLayerNet[6*100+ReLU,SGD]对MNIST数据集训练进而比较[多个超参数组合最优化]性能目录输出结果设计思路核心代码输出结果 val_acc: ...
DL之DNN：利用MultiLayerNet模型【6*100+ReLU+SGD,weight_decay】对Mnist数据集训练来抑制过拟合
DL之DNN:利用MultiLayerNet模型[6*100+ReLU+SGD,weight_decay]对Mnist数据集训练来抑制过拟合目录输出结果设计思路核心代码更多输出输出结果设 ...
DL之DNN：自定义MultiLayerNet(5*100+ReLU+SGD/Momentum/AdaGrad/Adam四种最优化)对MNIST数据集训练进而比较不同方法的性能
DL之DNN:自定义MultiLayerNet(5*100+ReLU+SGD/Momentum/AdaGrad/Adam四种最优化)对MNIST数据集训练进而比较不同方法的性能目录输出结果设计思 ...
DL之CNN：利用自定义DeepConvNet【7+1】算法对mnist数据集训练实现手写数字识别、模型评估(99.4%)
DL之CNN:利用自定义DeepConvNet[7+1]算法对mnist数据集训练实现手写数字识别.模型评估(99.4%) 目录输出结果设计思路核心代码输出结果设计思路核心代码 netwo ...
DL之CNN：利用自定义DeepConvNet【7+1】算法对mnist数据集训练实现手写数字识别并预测(超过99%)
DL之CNN:利用自定义DeepConvNet[7+1]算法对mnist数据集训练实现手写数字识别并预测(超过99%) 目录输出结果设计思路核心代码输出结果准确度都在99%以上 1.出错记录 ...
DL之DNN优化技术：自定义MultiLayerNetExtend算法(BN层使用/不使用+权重初始值不同)对Mnist数据集训练评估学习过程
DL之DNN优化技术:自定义MultiLayerNetExtend算法(BN层使用/不使用+权重初始值不同)对Mnist数据集训练评估学习过程目录输出结果设计思路核心代码更多输出相关文章: ...
DL之DNN优化技术：自定义MultiLayerNet【5*100+ReLU】对MNIST数据集训练进而比较三种权重初始值（Xavier参数初始化、He参数初始化）性能差异
DL之DNN优化技术:自定义MultiLayerNet[5*100+ReLU]对MNIST数据集训练进而比较三种权重初始值(Xavier参数初始化.He参数初始化)性能差异导读 #思路:观察不同的权 ...
DL之CNN可视化：利用SimpleConvNet算法【3层,im2col优化】基于mnist数据集训练并对卷积层输出进行可视化
DL之CNN可视化:利用SimpleConvNet算法[3层,im2col优化]基于mnist数据集训练并对卷积层输出进行可视化导读利用SimpleConvNet算法基于mnist数据集训练并对卷 ...

DL之DNN：利用MultiLayerNetExtend模型【6*100+ReLU+SGD,dropout】对Mnist数据集训练来抑制过拟合

输出结果

设计思路

核心代码

更多输出

DL之DNN：利用MultiLayerNetExtend模型【6*100+ReLU+SGD,dropout】对Mnist数据集训练来抑制过拟合相关推荐

最新文章

热门文章