Deep Neural Network for Image Classification: Application

实现一个深层的Neural Network需要用到，C1_week4_1中的辅助函数．

After this assignment you will be able to:

Build and apply a deep neural network to supervised learning.

Let’s get started!

1 - Packages

Let’s first import all the packages that you will need during this assignment.

numpy is the fundamental package for scientific computing with Python.
matplotlib is a library to plot graphs in Python.
h5py is a common package to interact with a dataset that is stored on an H5 file.
dnn_app_utils provides the functions implemented in the “Building your Deep Neural Network: Step by Step” assignment to this notebook.
np.random.seed(1) is used to keep all the random function calls consistent. It will help us grade your work.
注意:imread is deprecated!
imread is deprecated in SciPy 1.0.0.Use matplotlib.pyplot.imread instead.
注意:imresize is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.Use skimage.transform.resize instead.

import time
import numpy as np
import h5py
from skimage.transform import resize
import matplotlib.pyplot as plt
#import scipy
#from PIL import Image
#from scipy import ndimage
from dnn_app_utils_v2 import *
np.random.seed(1)

2 - Dataset

使用的数据集是上次Logistic Regression中相同的数据集．

Problem Statement: 数据集保存在h５格式的文件中．

a training set of m_train images labelled as cat (1) or non-cat (0)
a test set of m_test images labelled as cat and non-cat
each image is of shape (num_px, num_px, 3) where 3 is for the 3 channels (RGB).

def load_data():train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r")train_set_x_orig = np.array(train_dataset["train_set_x"][:]) # your train set featurestrain_set_y_orig = np.array(train_dataset["train_set_y"][:]) # your train set labelstest_dataset = h5py.File('datasets/test_catvnoncat.h5', "r")test_set_x_orig = np.array(test_dataset["test_set_x"][:]) # your test set featurestest_set_y_orig = np.array(test_dataset["test_set_y"][:]) # your test set labelsclasses = np.array(test_dataset["list_classes"][:]) # the list of classestrain_set_y_orig = train_set_y_orig.reshape((1, train_set_y_orig.shape[0]))test_set_y_orig = test_set_y_orig.reshape((1, test_set_y_orig.shape[0]))return train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, classes

train_x_orig, train_y, test_x_orig, test_y, classes = load_data()

训练集中的一张照片

# Example of a picture
index = 2
label = train_y[0, index]
plt.imshow(train_x_orig[index])
print ("y = " + str(label) + ". It's a " + classes[label].decode("utf-8") +  " picture.")

y = 1. It's a cat picture.

# Explore your dataset
m_train = train_x_orig.shape[0]
num_px = train_x_orig.shape[1]
m_test = test_x_orig.shape[0]print ("Number of training examples: " + str(m_train))
print ("Number of testing examples: " + str(m_test))
print ("Each image is of size: (" + str(num_px) + ", " + str(num_px) + ", 3)")
print ("train_x_orig shape: " + str(train_x_orig.shape))
print ("train_y shape: " + str(train_y.shape))
print ("test_x_orig shape: " + str(test_x_orig.shape))
print ("test_y shape: " + str(test_y.shape))

Number of training examples: 209
Number of testing examples: 50
Each image is of size: (64, 64, 3)
train_x_orig shape: (209, 64, 64, 3)
train_y shape: (1, 209)
test_x_orig shape: (50, 64, 64, 3)
test_y shape: (1, 50)

在训练之前需要对数据进行一些预处理：

reshape
standardize

Figure 1: Image to vector conversion.

# Reshape the training and test examples
train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T
test_x_flatten = test_x_orig.reshape(test_x_orig.shape[0], -1).T# Standardize data to have feature values between 0 and 1.
train_x = train_x_flatten/255.
test_x = test_x_flatten/255.print ("train_x's shape: " + str(train_x.shape))
print ("test_x's shape: " + str(test_x.shape))

train_x's shape: (12288, 209)
test_x's shape: (12288, 50)

12,28812,28812,288 equals 64×64×364×64×364 \times 64 \times 3 which is the size of one reshaped image vector.

3 - Architecture of your model

搭建两个不同的神经网络模型:

一个两层的NN
一个L层的DNN

3.1 - 2-layer neural network

Figure 2: 2-layer neural network.
The model can be summarized as: INPUT -> LINEAR -> RELU -> LINEAR -> SIGMOID -> OUTPUT.

3.2 - L-layer deep neural network

很难表述一个深层的网络模型，下面是一个简化版本的模型．

Figure 3: L-layer neural network.
The model can be summarized as: [LINEAR -> RELU] ××\times (L-1) -> LINEAR -> SIGMOID

3.3 - General methodology

As usual you will follow the Deep Learning methodology to build the model:

1. Initialize parameters / Define hyperparameters
2. Loop for num_iterations:a. Forward propagationb. Compute cost functionc. Backward propagationd. Update parameters (using parameters, and grads from backprop)
4. Use trained parameters to predict labels

Let’s now implement those two models!

4 - Two-layer neural network

Use the helper functions 在week4_1中都介绍过 to build a 2-layer neural network with the following structure: LINEAR -> RELU -> LINEAR -> SIGMOID. The functions you may need and their inputs are:

def initialize_parameters(n_x, n_h, n_y):...return parameters
def linear_activation_forward(A_prev, W, b, activation):...return A, cache
def compute_cost(AL, Y):...return cost
def linear_activation_backward(dA, cache, activation):...return dA_prev, dW, db
def update_parameters(parameters, grads, learning_rate):...return parameters

### CONSTANTS DEFINING THE MODEL ####
n_x = 12288     # num_px * num_px * 3
n_h = 7
n_y = 1
layers_dims = (n_x, n_h, n_y)

# GRADED FUNCTION: two_layer_modeldef two_layer_model(X, Y, layers_dims, learning_rate = 0.0075, num_iterations = 3000,print_cost=False):"""Implements a two-layer neural network: LINEAR->RELU->LINEAR->SIGMOID.Arguments:X -- input data, of shape (n_x, number of examples)Y -- true "label" vector (containing 0 if cat, 1 if non-cat), of shape (1, number of examples)layers_dims -- dimensions of the layers (n_x, n_h, n_y)num_iterations -- number of iterations of the optimization looplearning_rate -- learning rate of the gradient descent update ruleprint_cost -- If set to True, this will print the cost every 100 iterations Returns:parameters -- a dictionary containing W1, W2, b1, and b2"""np.random.seed(1)grads = {}costs = []                              # to keep track of the costm = X.shape[1]                           # number of examples(n_x, n_h, n_y) = layers_dims# Initialize parameters dictionary, by calling one of the functions you'd previously implementedparameters = initialize_parameters(n_x, n_h, n_y)# Get W1, b1, W2 and b2 from the dictionary parameters.W1 = parameters["W1"]b1 = parameters["b1"]W2 = parameters["W2"]b2 = parameters["b2"]# Loop (gradient descent)for i in range(0, num_iterations):# Forward propagation: LINEAR -> RELU -> LINEAR -> SIGMOID. Inputs: "X, W1, b1". Output: "A1, cache1, A2, cache2".A1, cache1 = linear_activation_forward(X, W1, b1, 'relu')A2, cache2 = linear_activation_forward(A1, W2, b2, 'sigmoid')# Compute costcost = compute_cost(A2, Y)# Initializing backward propagationdA2 = - (np.divide(Y, A2) - np.divide(1 - Y, 1 - A2))# Backward propagation. Inputs: "dA2, cache2, cache1". Outputs: "dA1, dW2, db2; also dA0 (not used), dW1, db1".dA1, dW2, db2 = linear_activation_backward(dA2, cache2, 'sigmoid')dA0, dW1, db1 = linear_activation_backward(dA1, cache1, 'relu')# Set grads['dWl'] to dW1, grads['db1'] to db1, grads['dW2'] to dW2, grads['db2'] to db2grads['dW1'] = dW1grads['db1'] = db1grads['dW2'] = dW2grads['db2'] = db2# Update parameters.### START CODE HERE ### (approx. 1 line of code)parameters = update_parameters(parameters, grads, learning_rate)### END CODE HERE #### Retrieve W1, b1, W2, b2 from parametersW1 = parameters["W1"]b1 = parameters["b1"]W2 = parameters["W2"]b2 = parameters["b2"]# Print the cost every 100 training exampleif print_cost and i % 100 == 0:print("Cost after iteration {}: {}".format(i, np.squeeze(cost)))if print_cost and i % 100 == 0:costs.append(cost)# plot the costplt.plot(np.squeeze(costs))plt.ylabel('cost')plt.xlabel('iterations (per tens)')plt.title("Learning rate =" + str(learning_rate))plt.show()return parameters

Training 2 layers NN model

parameters = two_layer_model(train_x, train_y, layers_dims = (n_x, n_h, n_y),num_iterations = 2500, print_cost=True)

Cost after iteration 0: 0.6930497356599888
Cost after iteration 100: 0.6464320953428849
Cost after iteration 200: 0.6325140647912678
Cost after iteration 300: 0.6015024920354665
Cost after iteration 400: 0.5601966311605748
Cost after iteration 500: 0.5158304772764729
Cost after iteration 600: 0.4754901313943325
Cost after iteration 700: 0.43391631512257495
Cost after iteration 800: 0.4007977536203889
Cost after iteration 900: 0.3580705011323798
Cost after iteration 1000: 0.33942815383664127
Cost after iteration 1100: 0.3052753636196264
Cost after iteration 1200: 0.27491377282130164
Cost after iteration 1300: 0.24681768210614818
Cost after iteration 1400: 0.19850735037466105
Cost after iteration 1500: 0.17448318112556663
Cost after iteration 1600: 0.17080762978096006
Cost after iteration 1700: 0.11306524562164738
Cost after iteration 1800: 0.09629426845937156
Cost after iteration 1900: 0.08342617959726865
Cost after iteration 2000: 0.07439078704319083
Cost after iteration 2100: 0.06630748132267933
Cost after iteration 2200: 0.05919329501038172
Cost after iteration 2300: 0.05336140348560558
Cost after iteration 2400: 0.04855478562877019

在traing和test数据上检验一下模型预测效果

predictions_train = predict(train_x, train_y, parameters)

Accuracy: 1.0

predictions_test = predict(test_x, test_y, parameters)

Accuracy: 0.72

在测试集上的准确率要比logistic regression要好一些，接下来看看跟深层的网络怎么样．

5 - L-layer Neural Network

LLL-layer neural network with the following structure: [LINEAR -> RELU]×" role="presentation" style="position: relative;">××\times(L-1) -> LINEAR -> SIGMOID. The functions you may need and their inputs are:

def initialize_parameters_deep(layer_dims):...return parameters
def L_model_forward(X, parameters):...return AL, caches
def compute_cost(AL, Y):...return cost
def L_model_backward(AL, Y, caches):...return grads
def update_parameters(parameters, grads, learning_rate):...return parameters

### CONSTANTS ###
layers_dims = [12288, 20, 7, 5, 1] #  5-layer model

# GRADED FUNCTION: L_layer_modeldef L_layer_model(X, Y, layers_dims, learning_rate = 0.0075, num_iterations = 3000,print_cost=False):#lr was 0.009"""Implements a L-layer neural network: [LINEAR->RELU]*(L-1)->LINEAR->SIGMOID.Arguments:X -- data, numpy array of shape (number of examples, num_px * num_px * 3)Y -- true "label" vector (containing 0 if cat, 1 if non-cat), of shape (1, number of examples)layers_dims -- list containing the input size and each layer size, of length (number of layers + 1).learning_rate -- learning rate of the gradient descent update rulenum_iterations -- number of iterations of the optimization loopprint_cost -- if True, it prints the cost every 100 stepsReturns:parameters -- parameters learnt by the model. They can then be used to predict."""np.random.seed(1)costs = []                         # keep track of cost# Parameters initialization.parameters = initialize_parameters_deep(layers_dims)# Loop (gradient descent)for i in range(0, num_iterations):# Forward propagation: [LINEAR -> RELU]*(L-1) -> LINEAR -> SIGMOID.AL, caches = L_model_forward(X, parameters)# Compute cost.cost = compute_cost(AL, Y)# Backward propagation.grads = L_model_backward(AL, Y, caches)# Update parameters.parameters = update_parameters(parameters, grads, learning_rate)# Print the cost every 100 training exampleif print_cost and i % 100 == 0:print ("Cost after iteration %i: %f" %(i, cost))if print_cost and i % 100 == 0:costs.append(cost)# plot the costplt.figure(figsize=(10,6))plt.plot(np.squeeze(costs))plt.ylabel('cost')plt.xlabel('iterations (per tens)')plt.title("Learning rate =" + str(learning_rate))plt.grid()plt.show()return parameters

train the model as a 5-layer neural network.

parameters = L_layer_model(train_x, train_y, layers_dims, num_iterations = 2500,print_cost = True)

Cost after iteration 0: 0.771749
Cost after iteration 100: 0.672053
Cost after iteration 200: 0.648263
Cost after iteration 300: 0.611507
Cost after iteration 400: 0.567047
Cost after iteration 500: 0.540138
Cost after iteration 600: 0.527930
Cost after iteration 700: 0.465477
Cost after iteration 800: 0.369126
Cost after iteration 900: 0.391747
Cost after iteration 1000: 0.315187
Cost after iteration 1100: 0.272700
Cost after iteration 1200: 0.237419
Cost after iteration 1300: 0.199601
Cost after iteration 1400: 0.189263
Cost after iteration 1500: 0.161189
Cost after iteration 1600: 0.148214
Cost after iteration 1700: 0.137775
Cost after iteration 1800: 0.129740
Cost after iteration 1900: 0.121225
Cost after iteration 2000: 0.113821
Cost after iteration 2100: 0.107839
Cost after iteration 2200: 0.102855
Cost after iteration 2300: 0.100897
Cost after iteration 2400: 0.092878

pred_train = predict(train_x, train_y, parameters)

Accuracy: 0.985645933014

pred_test = predict(test_x, test_y, parameters)

Accuracy: 0.8

5-layer neural network 的准确率要高于 2-layer neural network (72%)在同一测试集上．

6) Results Analysis

下面是分类错误的图片

def print_mislabeled_images(classes, X, y, p):"""Plots images where predictions and truth were different.X -- datasety -- true labelsp -- predictions"""a = p + ymislabeled_indices = np.asarray(np.where(a == 1))plt.rcParams['figure.figsize'] = (40.0, 40.0) # set default size of plotsnum_images = len(mislabeled_indices[0])for i in range(num_images):index = mislabeled_indices[1][i]plt.subplot(2, num_images, i + 1)plt.imshow(X[:,index].reshape(64,64,3), interpolation='nearest')plt.axis('off')plt.title("Prediction: " + classes[int(p[0,index])].decode("utf-8") + " \n Class: " + classes[y[0,index]].decode("utf-8"))

print_mislabeled_images(classes, test_x, test_y, pred_test)

A few type of images the model tends to do poorly on include:

Cat body in an unusual position
Cat appears against a background of a similar color
Unusual cat color and species
Camera Angle
Brightness of the picture
Scale variation (cat is very large or small in image)

7) Test with your own image (optional/ungraded exercise)

my_label_y = [0] # the true class of your image (1 -> cat, 0 -> non-cat)
fname = "datasets/ty.jpg"
image = np.array(plt.imread(fname))
my_image = resize(image, (64, 64, 3)).reshape((64*64*3,1))
my_predicted_image = predict(my_image, my_label_y, parameters)plt.imshow(image)
print ("y = " + str(np.squeeze(my_predicted_image)) + ", your L-layer model predicts a \"" + classes[int(np.squeeze(my_predicted_image)),].decode("utf-8") +  "\" picture.")

Accuracy: 1.0
y = 0.0, your L-layer model predicts a "non-cat" picture.

DL_C1_week4_2(build a deep neural network 2)相关推荐

5.深度学习练习：Deep Neural Network for Image Classification: Application
本文节选自吴恩达老师<深度学习专项课程>编程作业,在此表示感谢. 课程链接:https://www.deeplearning.ai/deep-learning-specialization ...
4.深度学习练习：Building your Deep Neural Network: Step by Step（强烈推荐）
本文节选自吴恩达老师<深度学习专项课程>编程作业,在此表示感谢. 课程链接:https://www.deeplearning.ai/deep-learning-specialization ...
文献记录(part33)-Hierarchical deep neural network for mental stress state detection using IoT ...
学习笔记,仅供参考,有错必纠仅记录实验文章目录 Hierarchical deep neural network for mental stress state detection using I ...
论文阅读（XiangBai——【AAAI2017】TextBoxes_A Fast Text Detector with a Single Deep Neural Network）...
XiangBai--[AAAI2017]TextBoxes:A Fast Text Detector with a Single Deep Neural Network 目录作者和相关链接方法概括 ...
论文阅读 [TPAMI-2022] ManifoldNet: A Deep Neural Network for Manifold-Valued Data With Applications
论文阅读 [TPAMI-2022] ManifoldNet: A Deep Neural Network for Manifold-Valued Data With Applications 论文搜索 ...
论文翻译：2022_PACDNN: A phase-aware composite deep neural network for speech enhancement
论文地址:PACDNN:一种用于语音增强的相位感知复合深度神经网络相似代码:https://github.com/phpstorm1/SE-FCN 引用格式:Hasannezhad M,Yu H,Z ...
《3D Point Cloud Registration for Localization using a Deep Neural Network Auto-Encoder》读书笔记
3D Point Cloud Registration for Localization using a Deep Neural Network Auto-Encoder 题目:基于深度神经网络自编码 ...
论文笔记：Identifying Lung Cancer Risk Factors in the Elderly Using Deep Neural Network - Chen, Wu
论文笔记:Identifying Lung Cancer Risk Factors in the Elderly Using Deep Neural Network - Chen, Wu 原文链接 I ...
抓取检测之 End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation
前言: 最近研究没有进度,就想着找几篇质量高点的最近的关于抓取检测的论文,这篇文章很新,联合了抓取检测和语义分割,正好自己也想着将语义分割应用到自己的抓取网络中,奈何自己编码基本功不到位,只能先看看思 ...
《TextBoxes: A Fast Text Detector with a Single Deep Neural Network》论文笔记
参考博文: 日常阅读论文,这是在谷歌学术上搜索其引用CRNN的相关文献中被引数量比较高的一篇OCR方向的文章,这里拿来读一读. 文章目录 make decision step1:读摘要 step2:读 ...

DL_C1_week4_2(build a deep neural network 2)