第二卷 第十章 GoogLeNet

在本章中，我们将研究GoogLeNet 架构。 首先，与 AlexNet 和 VGGNet 相比，模型架构很小（权重本身为约28MB）。作者能够通过移除完全连接的层并使用全局平均池化来实现网络架构大小的显着下降（同时仍然增加整个网络的深度）。CNN 中的大部分权重都可以在密集的 FC 层中找到，如果这些层可以被移除，内存节省是巨大的。

具体来说， Inception 模块是一个适合卷积神经网络的构建块，使其能够学习具有多种过滤器尺寸的 CONV 层，从而将该模块转变为多级特征提取器。

Inception 等微架构启发了其他重要的变体，包括 ResNet中的 Residual 模块和 SqueezeNet 中的 Fire 模块。我们将在本章后面讨论 Inception 模块（及其变体）。 并了解它的原理，我们将实现一个较小版本的 GoogLeNet，称为“MiniGoogLeNet”——我们将在 CIFAR-10 数据集上训练这个架构，并获得比之前章节任何其他架构更高的准确率。

之后，我们将继续进行更困难的Tiny ImageNet挑战。这项挑战是为参加斯坦福大学 cs231n 卷积神经网络视觉识别课程的学生提供的。这意味着让体验与现代架构上的大规模深度学习相关的挑战，而不是像整个 ImageNet 数据集那样耗时或费力。

通过在 Tiny ImageNet 上从头开始训练 GoogLeNet，我们将演示如何在 Tiny ImageNet 排行榜上获得最高排名。 在我们的下一章中，我们将利用 ResNet 从从头开始训练。

1、Inception 模块（及其变体）

Inception 模块背后的总体思路有两个方面：

（1）很难决定在给定的 CONV 层上需要学习的过滤器的大小。5×5还是3×3？或者1×1的过滤器来学习局部特征吗？ 相反，为什么不全部学习它们并让模型决定呢？在 Inception 模块中，我们学习了所有三个 5×5、3×3 和 1×1 过滤器（并行计算它们）沿着通道维度连接生成的特征图。GoogLeNet 架构中的下一层（可能是另一个 Inception 模块）接收这些串联的混合过滤器并执行相同的过程。总的来说，这个过程使 GoogLeNet 能够通过较小的卷积学习局部特征和具有较大卷积的抽象特征——我们不必以牺牲较小的特征为代价来牺牲我们的抽象水平。

（2）通过学习多个filter-size，我们可以把这个模块变成一个多级特征提取器。5×5个filters有更大的receptive size，可以学习更多的抽象特征。 1×1 过滤器根据定义是本地的。 3×3 过滤器作为两者之间的平衡。

1.1、Inception

现在我们已经讨论了 Inception 模块背后的动机，让我们看看图中的实际模块本身。

在每个 CONV 层之后隐式应用激活函数 (ReLU)。 为了节省篇幅，网络图中没有包含这个激活函数。 当我们实现 GoogLeNet 时，您将看到如何在 Inception 模块中使用此激活。

1.2、Miniception

最初的 Inception 模块是为 GoogLeNet 设计的，因此它可以在 ImageNet 数据集上进行训练（假设每个输入图像为 224×224×3）并获得最先进的精度。 对于需要较少网络参数的较小数据集（具有较小的图像空间维度），我们可以简化 Inception 模块。

·左：负责执行卷积、批量归一化和激活的卷积模块。

·中间：Miniception 模块执行两组卷积，一组用于 1×1 过滤器，另一组用于 3×3 过滤器，然后连接结果。 在 3×3 过滤器之前不执行降维，因为 (1) 输入量已经较小（因为我们将使用 CIFAR-10 数据集）和 (2) 以减少网络中的参数数量。

·右图：一个下采样模块，它应用卷积和最大池化来降低维度，然后跨过滤器维度连接。

2、MiniGoogLeNet在CIFAR-10上

在本节中，我们将使用 Miniception 模块实现 MiniGoogLeNet 架构。 然后我们将在 CIFAR-10 数据集上训练 MiniGoogLeNet。 正如我们的结果将证明的那样，该架构将在 CIFAR-10 上获得 > 90% 的准确率，远好于我们之前的所有尝试。

2.1、实施MiniGoogLeNet

创建minigooglenet.py

# import the necessary packages
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import AveragePooling2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Activation
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model
from tensorflow.keras.layers import concatenate
from tensorflow.keras import backend as Kclass MiniGoogLeNet:@staticmethoddef conv_module(x, K, kX, kY, stride, chanDim, padding="same"):# define a CONV => BN => RELU patternx = Conv2D(K, (kX, kY), strides=stride, padding=padding)(x)x = BatchNormalization(axis=chanDim)(x)x = Activation("relu")(x)# return the blockreturn x@staticmethoddef inception_module(x, numK1x1, numK3x3, chanDim):# define two CONV modules, then concatenate across the# channel dimensionconv_1x1 = MiniGoogLeNet.conv_module(x, numK1x1, 1, 1, (1, 1), chanDim)conv_3x3 = MiniGoogLeNet.conv_module(x, numK3x3, 3, 3, (1, 1), chanDim)x = concatenate([conv_1x1, conv_3x3], axis=chanDim)# return the blockreturn x@staticmethoddef downsample_module(x, K, chanDim):# define the CONV module and POOL, then concatenate# across the channel dimensionsconv_3x3 = MiniGoogLeNet.conv_module(x, K, 3, 3, (2, 2), chanDim, padding="valid")pool = MaxPooling2D((3, 3), strides=(2, 2))(x)x = concatenate([conv_3x3, pool], axis=chanDim)# return the blockreturn x@staticmethoddef build(width, height, depth, classes):# initialize the input shape to be "channels last" and the# channels dimension itselfinputShape = (height, width, depth)chanDim = -1# if we are using "channels first", update the input shape# and channels dimensionif K.image_data_format() == "channels_first":inputShape = (depth, height, width)chanDim = 1# define the model input and first CONV moduleinputs = Input(shape=inputShape)x = MiniGoogLeNet.conv_module(inputs, 96, 3, 3, (1, 1), chanDim)# two Inception modules followed by a downsample modulex = MiniGoogLeNet.inception_module(x, 32, 32, chanDim)x = MiniGoogLeNet.inception_module(x, 32, 48, chanDim)x = MiniGoogLeNet.downsample_module(x, 80, chanDim)# four Inception modules followed by a downsample modulex = MiniGoogLeNet.inception_module(x, 112, 48, chanDim)x = MiniGoogLeNet.inception_module(x, 96, 64, chanDim)x = MiniGoogLeNet.inception_module(x, 80, 80, chanDim)x = MiniGoogLeNet.inception_module(x, 48, 96, chanDim)x = MiniGoogLeNet.downsample_module(x, 96, chanDim)# two Inception modules followed by global POOL and dropoutx = MiniGoogLeNet.inception_module(x, 176, 160, chanDim)x = MiniGoogLeNet.inception_module(x, 176, 160, chanDim)x = AveragePooling2D((7, 7))(x)x = Dropout(0.5)(x)# softmax classifierx = Flatten()(x)x = Dense(classes)(x)x = Activation("softmax")(x)# create the modelmodel = Model(inputs, x, name="googlenet")# return the constructed network architecturereturn model

2.2、在CIFAR-10上训练和评估

创建googlenet_cifar10.py

使用 INIT_LR 初始学习率初始化 SGD 优化器。 一旦训练开始，这个学习率将通过 LearningRateScheduler 更新。 MiniGoogLeNet 架构本身将接受宽度为 32 像素、高度为 32 像素、深度为 3 通道、总共 10 个类别标签的输入图像。 使用 64 的小批量开始训练过程，训练总共 NUM_EPOCHS。 训练完成后，将我们的模型序列化到磁盘。

# set the matplotlib backend so figures can be saved in the background
import matplotlib
matplotlib.use("Agg")# import the necessary packages
from sklearn.preprocessing import LabelBinarizer
from models.minigooglenet.minigooglenet import MiniGoogLeNet
from customize.tools.trainingmonitor import TrainingMonitor
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.callbacks import LearningRateScheduler
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.datasets import cifar10
import numpy as np
import argparse
import os# define the total number of epochs to train for along with the
# initial learning rate
NUM_EPOCHS = 70
INIT_LR = 5e-3def poly_decay(epoch):# initialize the maximum number of epochs, base learning rate,# and power of the polynomialmaxEpochs = NUM_EPOCHSbaseLR = INIT_LRpower = 1.0# compute the new learning rate based on polynomial decayalpha = baseLR * (1 - (epoch / float(maxEpochs))) ** power# return the new learning ratereturn alpha# construct the argument parse and parse the arguments
# ap = argparse.ArgumentParser()
# ap.add_argument("-m", "--model", required=True, help="path to output model")
# ap.add_argument("-o", "--output", required=True, help="path to output directory (logs, plots, etc.)")
# args = vars(ap.parse_args())# load the training and testing data, converting the images from
# integers to floats
print("[INFO] loading CIFAR-10 data...")
((trainX, trainY), (testX, testY)) = cifar10.load_data()
trainX = trainX.astype("float")
testX = testX.astype("float")# apply mean subtraction to the data
mean = np.mean(trainX, axis=0)
trainX -= mean
testX -= mean# convert the labels from integers to vectors
lb = LabelBinarizer()
trainY = lb.fit_transform(trainY)
testY = lb.transform(testY)# construct the image generator for data augmentation
aug = ImageDataGenerator(width_shift_range=0.1, height_shift_range=0.1, horizontal_flip=True, fill_mode="nearest")# construct the set of callbacks
figPath = os.path.sep.join(["output", "{}.png".format(os.getpid())])
jsonPath = os.path.sep.join(["output", "{}.json".format(os.getpid())])
callbacks = [TrainingMonitor(figPath, jsonPath=jsonPath), LearningRateScheduler(poly_decay)]# initialize the optimizer and model
print("[INFO] compiling model...")
opt = SGD(lr=INIT_LR, momentum=0.9)
model = MiniGoogLeNet.build(width=32, height=32, depth=3, classes=10)
model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])# train the network
print("[INFO] training network...")
model.fit_generator(aug.flow(trainX, trainY, batch_size=64), validation_data=(testX, testY), steps_per_epoch=len(trainX) // 64, epochs=NUM_EPOCHS, callbacks=callbacks, verbose=1)# save the network to disk
print("[INFO] serializing network...")
model.save("model/model.hdf5")

2.3、不同学习率的比较

实验1：学习率1e-3

实验2：学习率1e-2

实验3：学习率5e-3

3、Tiny ImageNet

3.1、下载Tiny-ImageNet

您可以从这里的官方 cs231n 排行榜页面下载 Tiny ImageNet 数据集：
https://tiny-imagenet.herokuapp.com/https://tiny-imagenet.herokuapp.com/        .zip 文件为 237MB，因此在尝试下载之前请确保您有合理的互联网连接。

3.2、Tiny-ImageNet 目录结构

下载并解压 tiny-imagenet-200.zip 文件后，解压它，您会发现以下目录结构：

可以通过解析 words.txt 文件来查找 WordNet ID 的人类可读标签，该文件只是一个制表符分隔的文件，第一列是 WordNet ID，第二列是人类可读的词/对象。 wnids.txt 文件列出了 ImageNet 数据集中的 200 个 WordNet ID（每行一个）。

3.3、构建Tiny-ImageNet 数据集

新建一个配置文件tiny_imagenet_config.py

# import the necessary packages
from os import path# define the paths to the training and validation directories
TRAIN_IMAGES = "D:/Project/deeplearn/dataset/tiny-imagenet-200/datasets/tiny-imagenet-200/train"
VAL_IMAGES = "D:/Project/deeplearn/dataset/tiny-imagenet-200/datasets/tiny-imagenet-200/val/images"# define the path to the file that maps validation filenames to
# their corresponding class labels
VAL_MAPPINGS = "D:/Project/deeplearn/dataset/tiny-imagenet-200/datasets/tiny-imagenet-200/val/val_annotations.txt"# define the paths to the WordNet hierarchy files which are used
# to generate our class labels
WORDNET_IDS = "D:/Project/deeplearn/dataset/tiny-imagenet-200/datasets/tiny-imagenet-200/wnids.txt"
WORD_LABELS = "D:/Project/deeplearn/dataset/tiny-imagenet-200/datasets/tiny-imagenet-200/words.txt"# since we do not have access to the testing data we need to
# take a number of images from the training data and use it instead
NUM_CLASSES = 200
NUM_TEST_IMAGES = 50 * NUM_CLASSES# define the path to the output training, validation, and testing
# HDF5 files
TRAIN_HDF5 = "hdf5/train.hdf5"
VAL_HDF5 = "hdf5/val.hdf5"
TEST_HDF5 = "hdf5/test.hdf5"# define the path to the dataset mean
DATASET_MEAN = "output/tiny-image-net-200-mean.json"# define the path to the output directory used for storing plots,
# classification reports, etc.
OUTPUT_PATH = "output"
MODEL_PATH = path.sep.join([OUTPUT_PATH, "checkpoints/epoch_70.hdf5"])
FIG_PATH = path.sep.join([OUTPUT_PATH, "deepergooglenet_tinyimagenet.png"])
JSON_PATH = path.sep.join([OUTPUT_PATH, "deepergooglenet_tinyimagenet.json"])

创建一个生成hdf5的文件build_tiny_imagenet.py，并运行生成hdf5文件

# import the necessary packages
import tiny_imagenet_config as config
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from customize.tools.hdf5DatasetWriter import HDF5DatasetWriter
from imutils import paths
import numpy as np
import progressbar
import json
import cv2
import os# grab the paths to the training images, then extract the training
# class labels and encode them
trainPaths = list(paths.list_images(config.TRAIN_IMAGES))
trainLabels = [p.split(os.path.sep)[-3] for p in trainPaths]
le = LabelEncoder()
trainLabels = le.fit_transform(trainLabels)# perform stratified sampling from the training set to construct a
# a testing set
split = train_test_split(trainPaths, trainLabels, test_size=config.NUM_TEST_IMAGES, stratify=trainLabels, random_state=42)
(trainPaths, testPaths, trainLabels, testLabels) = split# load the validation filename => class from file and then use these
# mappings to build the validation paths and label lists
M = open(config.VAL_MAPPINGS).read().strip().split("\n")
M = [r.split("\t")[:2] for r in M]
valPaths = [os.path.sep.join([config.VAL_IMAGES, m[0]]) for m in M]
valLabels = le.transform([m[1] for m in M])# construct a list pairing the training, validation, and testing
# image paths along with their corresponding labels and output HDF5
# files
datasets = [("train", trainPaths, trainLabels, config.TRAIN_HDF5),("val", valPaths, valLabels, config.VAL_HDF5),("test", testPaths, testLabels, config.TEST_HDF5)]# initialize the lists of RGB channel averages
(R, G, B) = ([], [], [])# loop over the dataset tuples
for (dType, paths, labels, outputPath) in datasets:# create HDF5 writerprint("[INFO] building {}...".format(outputPath))writer = HDF5DatasetWriter((len(paths), 64, 64, 3), outputPath)# initialize the progress barwidgets = ["Building Dataset: ", progressbar.Percentage(), " ", progressbar.Bar(), " ", progressbar.ETA()]pbar = progressbar.ProgressBar(maxval=len(paths), widgets=widgets).start()# loop over the image pathsfor (i, (path, label)) in enumerate(zip(paths, labels)):# load the image from diskimage = cv2.imread(path)# if we are building the training dataset, then compute the# mean of each channel in the image, then update the# respective listsif dType == "train":(b, g, r) = cv2.mean(image)[:3]R.append(r)G.append(g)B.append(b)# add the image and label to the HDF5 datasetwriter.add([image], [label])pbar.update(i)# close the HDF5 writerpbar.finish()writer.close()# construct a dictionary of averages, then serialize the means to a
# JSON file
print("[INFO] serializing means...")
D = {"R": np.mean(R), "G": np.mean(G), "B": np.mean(B)}
f = open(config.DATASET_MEAN, "w")
f.write(json.dumps(D))
f.close()

4、Tiny-ImageNet 上的 DeeperGoogLeNet

4.1、实施DeeperGoogLeNet

创建deeergooglenet.py文件：

# import the necessary packages
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import AveragePooling2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Activation
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model
from tensorflow.keras.layers import concatenate
from tensorflow.keras.regularizers import l2
from tensorflow.keras import backend as Kclass DeeperGoogLeNet:@staticmethoddef conv_module(x, K, kX, kY, stride, chanDim, padding="same", reg=0.0005, name=None):# initialize the CONV, BN, and RELU layer names(convName, bnName, actName) = (None, None, None)# if a layer name was supplied, prepend itif name is not None:convName = name + "_conv"bnName = name + "_bn"actName = name + "_act"# define a CONV => BN => RELU patternx = Conv2D(K, (kX, kY), strides=stride, padding=padding, kernel_regularizer=l2(reg), name=convName)(x)x = BatchNormalization(axis=chanDim, name=bnName)(x)x = Activation("relu", name=actName)(x)# return the blockreturn x@staticmethoddef inception_module(x, num1x1, num3x3Reduce, num3x3, num5x5Reduce, num5x5, num1x1Proj, chanDim, stage, reg=0.0005):# define the first branch of the Inception module which# consists of 1x1 convolutionsfirst = DeeperGoogLeNet.conv_module(x, num1x1, 1, 1, (1, 1), chanDim, reg=reg, name=stage + "_first")# define the second branch of the Inception module which# consists of 1x1 and 3x3 convolutionssecond = DeeperGoogLeNet.conv_module(x, num3x3Reduce, 1, 1, (1, 1), chanDim, reg=reg, name=stage + "_second1")second = DeeperGoogLeNet.conv_module(second, num3x3, 3, 3, (1, 1), chanDim, reg=reg, name=stage + "_second2")# define the third branch of the Inception module which# are our 1x1 and 5x5 convolutionsthird = DeeperGoogLeNet.conv_module(x, num5x5Reduce, 1, 1, (1, 1), chanDim, reg=reg, name=stage + "_third1")third = DeeperGoogLeNet.conv_module(third, num5x5, 5, 5, (1, 1), chanDim, reg=reg, name=stage + "_third2")# define the fourth branch of the Inception module which# is the POOL projectionfourth = MaxPooling2D((3, 3), strides=(1, 1), padding = "same", name = stage + "_pool")(x)fourth = DeeperGoogLeNet.conv_module(fourth, num1x1Proj, 1, 1, (1, 1), chanDim, reg = reg, name = stage + "_fourth")# concatenate across the channel dimensionx = concatenate([first, second, third, fourth], axis=chanDim, name = stage + "_mixed")# return the blockreturn x@staticmethoddef build(width, height, depth, classes, reg=0.0005):# initialize the input shape to be "channels last" and the# channels dimension itselfinputShape = (height, width, depth)chanDim = -1# if we are using "channels first", update the input shape# and channels dimensionif K.image_data_format() == "channels_first":inputShape = (depth, height, width)chanDim = 1# define the model input, followed by a sequence of CONV =># POOL => (CONV * 2) => POOL layersinputs = Input(shape=inputShape)x = DeeperGoogLeNet.conv_module(inputs, 64, 5, 5, (1, 1), chanDim, reg = reg, name = "block1")x = MaxPooling2D((3, 3), strides=(2, 2), padding="same", name = "pool1")(x)x = DeeperGoogLeNet.conv_module(x, 64, 1, 1, (1, 1), chanDim, reg = reg, name = "block2")x = DeeperGoogLeNet.conv_module(x, 192, 3, 3, (1, 1), chanDim, reg = reg, name = "block3")x = MaxPooling2D((3, 3), strides=(2, 2), padding="same", name = "pool2")(x)# apply two Inception modules followed by a POOLx = DeeperGoogLeNet.inception_module(x, 64, 96, 128, 16, 32, 32, chanDim, "3a", reg = reg)x = DeeperGoogLeNet.inception_module(x, 128, 128, 192, 32, 96, 64, chanDim, "3b", reg = reg)x = MaxPooling2D((3, 3), strides=(2, 2), padding="same", name = "pool3")(x)# apply five Inception modules followed by POOLx = DeeperGoogLeNet.inception_module(x, 192, 96, 208, 16, 48, 64, chanDim, "4a", reg = reg)x = DeeperGoogLeNet.inception_module(x, 160, 112, 224, 24, 64, 64, chanDim, "4b", reg = reg)x = DeeperGoogLeNet.inception_module(x, 128, 128, 256, 24, 64, 64, chanDim, "4c", reg = reg)x = DeeperGoogLeNet.inception_module(x, 112, 144, 288, 32, 64, 64, chanDim, "4d", reg = reg)x = DeeperGoogLeNet.inception_module(x, 256, 160, 320, 32, 128, 128, chanDim, "4e", reg = reg)x = MaxPooling2D((3, 3), strides=(2, 2), padding="same", name = "pool4")(x)# apply a POOL layer (average) followed by dropoutx = AveragePooling2D((4, 4), name="pool5")(x)x = Dropout(0.4, name="do")(x)# softmax classifierx = Flatten(name="flatten")(x)x = Dense(classes, kernel_regularizer=l2(reg), name="labels")(x)x = Activation("softmax", name="softmax")(x)# create the modelmodel = Model(inputs, x, name="googlenet")# return the constructed network architecturereturn model

4.2、在Tiny-ImageNet上训练

创建train.py

# set the matplotlib backend so figures can be saved in the background
import matplotlib
matplotlib.use("Agg")# import the necessary packages
import tiny_imagenet_config as config
from customize.tools.imagetoarraypreprocessor import ImageToArrayPreprocessor
from customize.tools.simplepreprocessor import SimplePreprocessor
from customize.tools.meanpreprocessor import MeanPreprocessor
from tensorflow.keras.callbacks import ModelCheckpoint
from customize.tools.trainingmonitor import TrainingMonitor
from customize.tools.hdf5datasetgenerator import HDF5DatasetGenerator
from models.deepergooglenet.deeergooglenet import DeeperGoogLeNet
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.models import load_model
import tensorflow.keras.backend as K
import argparse
import json# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-c", "--checkpoints", required=True, help="path to output checkpoint directory")
ap.add_argument("-m", "--model", type=str, help="path to *specific* model checkpoint to load")
ap.add_argument("-s", "--start-epoch", type=int, default=0, help="epoch to restart training at")
args = vars(ap.parse_args())# construct the training image generator for data augmentation
aug = ImageDataGenerator(rotation_range=18, zoom_range=0.15, width_shift_range=0.2,height_shift_range=0.2, shear_range=0.15,horizontal_flip=True, fill_mode="nearest")# load the RGB means for the training set
means = json.loads(open(config.DATASET_MEAN).read())# initialize the image preprocessors
sp = SimplePreprocessor(64, 64)
mp = MeanPreprocessor(means["R"], means["G"], means["B"])
iap = ImageToArrayPreprocessor()# initialize the training and validation dataset generators
trainGen = HDF5DatasetGenerator(config.TRAIN_HDF5, 64, aug=aug, preprocessors=[sp, mp, iap], classes=config.NUM_CLASSES)
valGen = HDF5DatasetGenerator(config.VAL_HDF5, 64, preprocessors=[sp, mp, iap], classes=config.NUM_CLASSES)# if there is no specific model checkpoint supplied, then initialize
# the network and compile the model
if args["model"] is None:print("[INFO] compiling model...")model = DeeperGoogLeNet.build(width=64, height=64, depth=3,classes=config.NUM_CLASSES, reg=0.0002)opt = Adam(1e-3)model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
# otherwise, load the checkpoint from disk
else:print("[INFO] loading {}...".format(args["model"]))model = load_model(args["model"])# update the learning rateprint("[INFO] old learning rate: {}".format(K.get_value(model.optimizer.lr)))K.set_value(model.optimizer.lr, 1e-5)print("[INFO] new learning rate: {}".format(K.get_value(model.optimizer.lr)))# construct the set of callbacks
#callbacks = [EpochCheckpoint(args["checkpoints"], every=5, startAt=args["start_epoch"]),
#    TrainingMonitor(config.FIG_PATH, jsonPath=config.JSON_PATH, startAt=args["start_epoch"])]
model_checkpoint = ModelCheckpoint('unet_membrane.hdf5', monitor='loss', verbose=1, save_best_only=True)# train the network
model.fit_generator(trainGen.generator(), steps_per_epoch=trainGen.numImages // 64, validation_data=valGen.generator(),validation_steps=valGen.numImages // 64, epochs=10, max_queue_size=64 * 2, callbacks=[model_checkpoint], verbose=1)# close the databases
trainGen.close()
valGen.close()

4.3、训练结果

我决定使用 SGD 训练 DeeperGoogLenet，初始学习率为 1e-2，动量项为 0.9（未应用 Nesterov 加速）。 我总是在我的第一个实验中使用 SGD。 根据我在第 7 章中的指导方针和经验法则，您应该首先尝试 SGD 以获得基线，然后如果需要，使用更高级的优化方法。

在 epoch 25 之后，我停止训练，将学习率降低到 1e3，然后继续训练10个epoch。

在 epoch 35 之后，我再次停止训练，将学习率降低到 1e

Python视觉深度学习系列教程 第二卷 第10章 GoogLeNet相关推荐

1. Python视觉深度学习系列教程 第二卷 第4章 微调网络

           第二卷 第四章 微调网络         在上一章中,我们学习了如何将预训练的卷积神经网络视为特征提取器.使用这个特征提取器,我们通过网络向前传播我们的图像数据集,提取给定层的激活,并 ...

2. Python视觉深度学习系列教程 第二卷 第9章 Kaggle竞赛：Cat与Dog

   第二卷 第九章 Kaggle竞赛:Cat与Dog 在本章中,我们将扩展我们的工作并学习如何为HDF5数据集定义一个图像生成器,适用于使用Keras训练卷积神经网络.该生成器将打开HDF5数据集,为要训 ...

3. Python视觉深度学习系列教程 第三卷 第8章 在ImageNet上训练SqueezeNet

           第三卷 第八章 在ImageNet上训练SqueezeNet         关于在ImageNet大规模视觉识别挑战 (ILSVRC) 上训练深度神经网络的最后一章中,将讨论Sque ...

4. Python视觉深度学习系列教程 第三卷 第5章 在ImageNet上训练VGGNet

           第三卷 第五章 在ImageNet上训练VGGNet 在本章中,我们将从头开始学习如何在 ImageNet 数据集上训练 VGG16 网络架构. 该网络的特点是简单,仅使用3*3 卷积 ...

5. Python视觉深度学习系列教程 第三卷 第2章 什么是ImageNet？

           第三卷 第二章 什么是ImageNet? 在本章中,我们将讨论 ImageNet 数据集和相关的 ImageNet 大规模视觉识别挑战 (ILSVRC) . 这一挑战是评估图像分类算法 ...

6. Python视觉深度学习系列教程 第三卷 第9章 Kaggle竞赛：情绪识别

           第三卷 第九章 Kaggle竞赛:情绪识别 在本章中,我们将解决Kaggle的面部表情识别挑战.为了完成这项任务,我们将在训练数据上从头开始训练一个类似VGG的网络,同时考虑到我们的网 ...

7. Python视觉深度学习系列教程 第三卷 第12章 年龄和性别预测

           第三卷 第十二章 年龄和性别预测 为了构建一个能够识别照片中人物年龄和性别的系统,我们将使用 Adience 数据集.我们训练两个模型,一个用于年龄识别,另一个用于性别识别.此外,我们 ...

8. Python视觉深度学习系列教程 第三卷 第14章 从头开始训练Faster R-CNN

           第三卷 第十四章 从头开始训练Faster R-CNN 本章的目的是达到以下四点: 1.在您的系统上安装和配置 TensorFlow Object Detection API. 2.在 ...

9. Python视觉深度学习系列教程 第一卷 第21章 案例：使用CNN破解验证码

           第一卷 第二十一章 案例:使用CNN破解验证码 Breaking captchas with deep learning, Keras, and TensorFlow - PyImag ...

最新文章

1. [实现] 利用 Seq2Seq 预测句子后续字词 （Pytorch）
2. PCL中点云的超体素(SuperVoxel)
3. 【ACM】杭电OJ 2039
4. Linux CENTOS7 Linux网络性能测试工具-iperf 安装过程以及示例！
5. [JSOI2008]火星人
6. Android-广播接收者简介
7. Android经常使用的五种弹出对话框
8. mysql recovery = 4_强制MySQL InnoDB恢复参数innodb_force_recovery
9. Java程序利用POJ读写Excel的.xls或.xlsx文件所需的3个jar包
10. Java描述设计模式(16)：代理模式
11. server2003 IIS6.0 网站不可用
12. H3C 模拟器 pc与sw直连 开启telnet
13. UE4 蓝图教程(一) 开始，一个转动的香蕉
14. qq 病毒 html,盗qq号软件里有病毒吗
15. 萬惡淫為首，百善孝為先
16. 四个角不是直角的四边形_四边形的特点是有四条直的边和四个直角对吗
17. H2O with R 简明使用手记·上篇
18. 创业布局移动端App，应该先做Android还是iOS？
19. 华三交换机如何进入配置_H3C交换机如何配置登录用户名和密码
20. 工作日计算器_.NET工作日计算器

热门文章

1. 迷宫问题寻找最短路径（BFS）
2. php内容怎么设置隐藏,在PHP中设置隐藏元素的值 - php
3. Ubuntu 无法连接上 cn.archive.ubuntu.com:80
4. 瞬间把自己家里的ipad或华为平板、手机变成电脑副屏
5. Jam-packed（思维）
6. 【Vue】时间戳转换为年月日 格式为 yyyy-MM-dd hh:mm js转换时间戳 时间戳转换为年月日 Fri Apr 01 2022 00:00:00 GMT+0800 (中国标准时间)转换
7. 567网盘 飞猫网盘 星耀网盘 kufile expfile rardisk RoseFile 雪球网盘 77file 迅牛网盘 ownfile 1988网盘 520网盘解析演示
8. META-INF/MANIFEST.MF介绍 _
9. 电子科技大学2021计算机考研复试科目,2021西安电子科技大学考研复试时间_西安交通大学考研复试科目...
10. sql 大于号小于号显示问题