使用神经网络对手写数字进行分类

学习目标：

• 训练线性模型和神经网络，以对传统 MNIST 数据集中的手写数字进行分类
• 比较线性分类模型和神经网络分类模型的效果
• 可视化神经网络隐藏层的权重

---

我们的目标是将每个输入图片与正确的数字相对应。我们会创建一个包含几个隐藏层的神经网络，并在顶部放置一个归一化指数层，以选出最合适的类别。

第1步：设置：加载必要的库+加载数据+数据预处理

from __future__ import print_functionimport glob
import math
import osfrom IPython import display
from matplotlib import cm
from matplotlib import gridspec
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn import metrics
import tensorflow as tf
from tensorflow.python.data import Datasettf.logging.set_verbosity(tf.logging.ERROR)
pd.options.display.max_rows = 10
pd.options.display.float_format = '{:.1f}'.formatmnist_dataframe = pd.read_csv("https://download.mlcc.google.com/mledu-datasets/mnist_train_small.csv",sep=",",header=None)# Use just the first 10,000 records for training/validation
mnist_dataframe = mnist_dataframe.head(10000)mnist_dataframe = mnist_dataframe.reindex(np.random.permutation(mnist_dataframe.index))
mnist_dataframe.head()

第一列中包含类别标签。其余列中包含特征值，每个像素对应一个特征值，有 28×28=784 个像素值，其中大部分像素值都为零；您也许需要花一分钟时间来确认它们不全部为零。

这些样本都是分辨率相对较低、对比度相对较高的手写数字图片。0-9 这十个数字中的每个可能出现的数字均由唯一的类别标签表示。因此，这是一个具有 10 个类别的多类别分类问题。

现在，我们解析一下标签和特征，并查看几个样本。注意 loc 的使用，借助 loc，我们能够基于原来的位置抽出各列，因为此数据集中没有标题行。

def parse_labels_and_features(dataset):"""Extracts labels and features.This is a good place to scale or transform the features if needed.Args:dataset: A Pandas `Dataframe`, containing the label on the first column andmonochrome pixel values on the remaining columns, in row major order.Returns:A `tuple` `(labels, features)`:labels: A Pandas `Series`.features: A Pandas `DataFrame`."""labels = dataset[0]    #数据的第0列为标签列# DataFrame.loc index ranges are inclusive at both ends.features = dataset.loc[:,1:784]    #数据第1~784行为数据内容列# Scale the data to [0, 1] by dividing out the max value, 255.features = features / 255      #通过除以255把特征值缩小到[0，1]之间return labels, features

预览数据集：

training_targets, training_examples = parse_labels_and_features(mnist_dataframe[:7500])
training_examples.describe()

第2步：样本特征列的选择

rand_example = np.random.choice(training_examples.index)
_, ax = plt.subplots()
ax.matshow(training_examples.loc[rand_example].values.reshape(28, 28))
ax.set_title("Label: %i" % training_targets.loc[rand_example])
ax.grid(False)

第三步：样本特征列的选择

def construct_feature_columns():"""Construct the TensorFlow Feature Columns.Returns:A set of feature columns""" # There are 784 pixels in each image return set([tf.feature_column.numeric_column('pixels', shape=784)])

*对比示范第3步*：为 MNIST 构建线性模型

在本次练习中，我们会对训练和预测使用单独的输入函数，并将这些函数分别嵌套在 create_training_input_fn() 和 create_predict_input_fn() 中，这样一来，我们就可以调用这些函数，以返回相应的 _input_fn，并将其传递到 .train() 和 .predict() 调用。

def create_training_input_fn(features, labels, batch_size, num_epochs=None, shuffle=True):"""A custom input_fn for sending MNIST data to the estimator for training.Args:features: The training features.labels: The training labels.batch_size: Batch size to use during training.Returns:A function that returns batches of training features and labels duringtraining."""def _input_fn(num_epochs=None, shuffle=True):# Input pipelines are reset with each call to .train(). To ensure model# gets a good sampling of data, even when steps is small, we # shuffle all the data before creating the Dataset objectidx = np.random.permutation(features.index)raw_features = {"pixels":features.reindex(idx)}#打乱顺序并重新索引raw_targets = np.array(labels[idx])ds = Dataset.from_tensor_slices((raw_features,raw_targets)) # warning: 2GB limitds = ds.batch(batch_size).repeat(num_epochs)if shuffle:ds = ds.shuffle(10000)# Return the next batch of datafeature_batch, label_batch = ds.make_one_shot_iterator().get_next()return feature_batch, label_batchreturn _input_fn

def create_predict_input_fn(features, labels, batch_size):"""A custom input_fn for sending mnist data to the estimator for predictions.Args:features: The features to base predictions on.labels: The labels of the prediction examples.Returns:A function that returns features and labels for predictions."""def _input_fn():raw_features = {"pixels": features.values}raw_targets = np.array(labels)ds = Dataset.from_tensor_slices((raw_features, raw_targets)) # warning: 2GB limitds = ds.batch(batch_size)# Return the next batch of datafeature_batch, label_batch = ds.make_one_shot_iterator().get_next()return feature_batch, label_batchreturn _input_fn

def train_linear_classification_model(learning_rate,steps,batch_size,training_examples,training_targets,validation_examples,validation_targets):"""Trains a linear classification model for the MNIST digits dataset.In addition to training, this function also prints training progress information,a plot of the training and validation loss over time, and a confusionmatrix.Args:learning_rate: An `int`, the learning rate to use.steps: A non-zero `int`, the total number of training steps. A training stepconsists of a forward and backward pass using a single batch.batch_size: A non-zero `int`, the batch size.training_examples: A `DataFrame` containing the training features.training_targets: A `DataFrame` containing the training labels.validation_examples: A `DataFrame` containing the validation features.validation_targets: A `DataFrame` containing the validation labels.Returns:The trained `LinearClassifier` object."""periods = 10steps_per_period = steps / periods  # Create the input functions.predict_training_input_fn = create_predict_input_fn(training_examples, training_targets, batch_size)predict_validation_input_fn = create_predict_input_fn(validation_examples, validation_targets, batch_size)training_input_fn = create_training_input_fn(training_examples, training_targets, batch_size)# Create a LinearClassifier object.my_optimizer = tf.train.AdagradOptimizer(learning_rate=learning_rate)my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)classifier = tf.estimator.LinearClassifier(feature_columns=construct_feature_columns(),n_classes=10,optimizer=my_optimizer,config=tf.estimator.RunConfig(keep_checkpoint_max=1))# Train the model, but do so inside a loop so that we can periodically assess# loss metrics.print("Training model...")print("LogLoss error (on validation data):")training_errors = []validation_errors = []for period in range (0, periods):# Train the model, starting from the prior state.classifier.train(input_fn=training_input_fn,steps=steps_per_period)# Take a break and compute probabilities.training_predictions = list(classifier.predict(input_fn=predict_training_input_fn))training_probabilities = np.array([item['probabilities'] for item in training_predictions])training_pred_class_id = np.array([item['class_ids'][0] for item in training_predictions])training_pred_one_hot = tf.keras.utils.to_categorical(training_pred_class_id,10)validation_predictions = list(classifier.predict(input_fn=predict_validation_input_fn))validation_probabilities = np.array([item['probabilities'] for item in validation_predictions])    validation_pred_class_id = np.array([item['class_ids'][0] for item in validation_predictions])validation_pred_one_hot = tf.keras.utils.to_categorical(validation_pred_class_id,10)    # Compute training and validation errors.training_log_loss = metrics.log_loss(training_targets, training_pred_one_hot)validation_log_loss = metrics.log_loss(validation_targets, validation_pred_one_hot)# Occasionally print the current loss.print("  period %02d : %0.2f" % (period, validation_log_loss))# Add the loss metrics from this period to our list.training_errors.append(training_log_loss)validation_errors.append(validation_log_loss)print("Model training finished.")# Remove event files to save disk space._ = map(os.remove, glob.glob(os.path.join(classifier.model_dir, 'events.out.tfevents*')))# Calculate final predictions (not probabilities, as above).final_predictions = classifier.predict(input_fn=predict_validation_input_fn)final_predictions = np.array([item['class_ids'][0] for item in final_predictions])accuracy = metrics.accuracy_score(validation_targets, final_predictions)print("Final accuracy (on validation data): %0.2f" % accuracy)# Output a graph of loss metrics over periods.plt.ylabel("LogLoss")plt.xlabel("Periods")plt.title("LogLoss vs. Periods")plt.plot(training_errors, label="training")plt.plot(validation_errors, label="validation")plt.legend()plt.show()# Output a plot of the confusion matrix.cm = metrics.confusion_matrix(validation_targets, final_predictions)# Normalize the confusion matrix by row (i.e by the number of samples# in each class)cm_normalized = cm.astype("float") / cm.sum(axis=1)[:, np.newaxis]ax = sns.heatmap(cm_normalized, cmap="bone_r")ax.set_aspect(1)plt.title("Confusion matrix")plt.ylabel("True label")plt.xlabel("Predicted label")plt.show()return classifier

*对比示范第4步*：调整训练模型参数开始训练

_ = train_linear_classification_model(learning_rate=0.03,steps=1000,batch_size=30,training_examples=training_examples,training_targets=training_targets,validation_examples=validation_examples,validation_targets=validation_targets)

该线性模型经过多次调试，最后获得了一个比较高的准确率90%

使用神经网络替换线性分类器

第3步：创建神经网络模型

def train_nn_classification_model(learning_rate,steps,batch_size,hidden_units,         #配置隐藏单元training_examples,training_targets,validation_examples,validation_targets):"""Trains a neural network classification model for the MNIST digits dataset.In addition to training, this function also prints training progress information,a plot of the training and validation loss over time, as well as a confusionmatrix.Args:learning_rate: An `int`, the learning rate to use.steps: A non-zero `int`, the total number of training steps. A training stepconsists of a forward and backward pass using a single batch.batch_size: A non-zero `int`, the batch size.hidden_units: A `list` of int values, specifying the number of neurons in each layer.#隐藏单元training_examples: A `DataFrame` containing the training features.training_targets: A `DataFrame` containing the training labels.validation_examples: A `DataFrame` containing the validation features.validation_targets: A `DataFrame` containing the validation labels.Returns:The trained `DNNClassifier` object."""periods = 10# Caution: input pipelines are reset with each call to train. # If the number of steps is small, your model may never see most of the data.  # So with multiple `.train` calls like this you may want to control the length # of training with num_epochs passed to the input_fn. Or, you can do a really-big shuffle, # or since it's in-memory data, shuffle all the data in the `input_fn`.steps_per_period = steps / periods  # Create the input functions.predict_training_input_fn = create_predict_input_fn(training_examples, training_targets, batch_size)predict_validation_input_fn = create_predict_input_fn(validation_examples, validation_targets, batch_size)training_input_fn = create_training_input_fn(training_examples, training_targets, batch_size)# Create the input functions.predict_training_input_fn = create_predict_input_fn(training_examples, training_targets, batch_size)predict_validation_input_fn = create_predict_input_fn(validation_examples, validation_targets, batch_size)training_input_fn = create_training_input_fn(training_examples, training_targets, batch_size)# Create feature columns.feature_columns = [tf.feature_column.numeric_column('pixels', shape=784)]# Create a DNNClassifier object.my_optimizer = tf.train.AdagradOptimizer(learning_rate=learning_rate)my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)classifier = tf.estimator.DNNClassifier(                   #DNNClassifier feature_columns=feature_columns,n_classes=10,hidden_units=hidden_units,#隐藏单元optimizer=my_optimizer,config=tf.contrib.learn.RunConfig(keep_checkpoint_max=1)#隐藏单元配置)# Train the model, but do so inside a loop so that we can periodically assess# loss metrics.print("Training model...")print("LogLoss error (on validation data):")training_errors = []validation_errors = []for period in range (0, periods):# Train the model, starting from the prior state.classifier.train(input_fn=training_input_fn,steps=steps_per_period)# Take a break and compute probabilities.training_predictions = list(classifier.predict(input_fn=predict_training_input_fn))training_probabilities = np.array([item['probabilities'] for item in training_predictions])training_pred_class_id = np.array([item['class_ids'][0] for item in training_predictions])training_pred_one_hot = tf.keras.utils.to_categorical(training_pred_class_id,10)validation_predictions = list(classifier.predict(input_fn=predict_validation_input_fn))validation_probabilities = np.array([item['probabilities'] for item in validation_predictions])    validation_pred_class_id = np.array([item['class_ids'][0] for item in validation_predictions])validation_pred_one_hot = tf.keras.utils.to_categorical(validation_pred_class_id,10)    # Compute training and validation errors.training_log_loss = metrics.log_loss(training_targets, training_pred_one_hot)validation_log_loss = metrics.log_loss(validation_targets, validation_pred_one_hot)# Occasionally print the current loss.print("  period %02d : %0.2f" % (period, validation_log_loss))# Add the loss metrics from this period to our list.training_errors.append(training_log_loss)validation_errors.append(validation_log_loss)print("Model training finished.")# Remove event files to save disk space._ = map(os.remove, glob.glob(os.path.join(classifier.model_dir, 'events.out.tfevents*')))# Calculate final predictions (not probabilities, as above).final_predictions = classifier.predict(input_fn=predict_validation_input_fn)final_predictions = np.array([item['class_ids'][0] for item in final_predictions])accuracy = metrics.accuracy_score(validation_targets, final_predictions)print("Final accuracy (on validation data): %0.2f" % accuracy)# Output a graph of loss metrics over periods.plt.ylabel("LogLoss")plt.xlabel("Periods")plt.title("LogLoss vs. Periods")plt.plot(training_errors, label="training")plt.plot(validation_errors, label="validation")plt.legend()plt.show()# Output a plot of the confusion matrix.cm = metrics.confusion_matrix(validation_targets, final_predictions)# Normalize the confusion matrix by row (i.e by the number of samples# in each class)cm_normalized = cm.astype("float") / cm.sum(axis=1)[:, np.newaxis]ax = sns.heatmap(cm_normalized, cmap="bone_r")ax.set_aspect(1)plt.title("Confusion matrix")plt.ylabel("True label")plt.xlabel("Predicted label")plt.show()return classifier

第4步：调整训练模型参数开始训练

classifier = train_nn_classification_model(learning_rate=0.05,steps=1000,batch_size=30,hidden_units=[100, 100],training_examples=training_examples,training_targets=training_targets,validation_examples=validation_examples,validation_targets=validation_targets)

第5步：验证测试集的准确率

mnist_test_dataframe = pd.read_csv("https://download.mlcc.google.com/mledu-datasets/mnist_test.csv",sep=",",header=None)test_targets, test_examples = parse_labels_and_features(mnist_test_dataframe)
test_examples.describe()

predict_test_input_fn = create_predict_input_fn(test_examples, test_targets, batch_size=100)test_predictions = classifier.predict(input_fn=predict_test_input_fn)
test_predictions = np.array([item['class_ids'][0] for item in test_predictions])accuracy = metrics.accuracy_score(test_targets, test_predictions)
print("Accuracy on test data: %0.2f" % accuracy)

使用深度神经网络之后，准确率达到了94%。效果有明显提升。

额外任务 3：可视化第一个隐藏层的权重

我们来花几分钟时间看看模型的 weights_ 属性，以深入探索我们的神经网络，并了解它学到了哪些规律。

模型的输入层有 784 个权重，对应于 28×28 像素输入图片。第一个隐藏层将有 784×N 个权重，其中 N 指的是该层中的节点数。我们可以将这些权重重新变回 28×28像素的图片，具体方法是将 N 个 1×784 权重数组变形为 N 个 28×28 大小数组。

运行以下单元格，绘制权重曲线图。请注意，此单元格要求名为 "classifier" 的 DNNClassifier 已经过训练。

print(classifier.get_variable_names())weights0 = classifier.get_variable_value("dnn/hiddenlayer_0/kernel")print("weights0 shape:", weights0.shape)num_nodes = weights0.shape[1]
num_rows = int(math.ceil(num_nodes / 10.0))
fig, axes = plt.subplots(num_rows, 10, figsize=(20, 2 * num_rows))
for coef, ax in zip(weights0.T, axes.ravel()):# Weights in coef is reshaped from 1x784 to 28x28.ax.matshow(coef.reshape(28, 28), cmap=plt.cm.pink)ax.set_xticks(())ax.set_yticks(())plt.show()

神经网络的第一个隐藏层应该会对一些级别特别低的特征进行建模，因此可视化权重可能只显示一些模糊的区域，也可能只显示数字的某几个部分。此外，您可能还会看到一些基本上是噪点（这些噪点要么不收敛，要么被更高的层忽略）的神经元。

好吧，反正我是看得一脸懵逼！只能看出图中的凹凸部分很像是写字时候的部分轨迹。还有部分很像二维码的应该是噪点吧。

本文仅为个人学习笔记记录，请结合Google 机器学习，编程练习：使用神经网络对手写数字进行分类，进行阅读

编程练习地址：https://colab.research.google.com/notebooks/mlcc/multi-class_classification_of_handwritten_digits.ipynb?hl=zh-cn#scrollTo=ZfzsTYGPPU8I

Google TensorFlow课程 编程笔记（10）———使用神经网络对手写数字进行分类相关推荐

1. 机器学习速成课程 | 练习 | Google Development——编程练习：使用神经网络对手写数字进行分类

   使用神经网络对手写数字进行分类 学习目标: 训练线性模型和神经网络,以对传统 MNIST 数据集中的手写数字进行分类 比较线性分类模型和神经网络分类模型的效果 可视化神经网络隐藏层的权重 我们的目标是 ...

2. PyTorch之LeNet-5：利用PyTorch实现最经典的LeNet-5卷积神经网络对手写数字图片识别CNN

   PyTorch之LeNet-5:利用PyTorch实现最经典的LeNet-5卷积神经网络对手写数字图片识别CNN 目录 训练过程 代码设计 训练过程 代码设计 #PyTorch:利用PyTorch实现 ...

3. 深度学习实战——利用卷积神经网络对手写数字二值图像分类（附代码）

   系列文章目录 深度学习实战--利用卷积神经网络对手写数字二值图像分类(附代码) 目录 系列文章目录 前言 一.案例需求 二.MATLAB算法实现 三.MATLAB源代码 参考文献 前言 本案例利用MA ...

4. 《人工智能实践：Tensorflow笔记》听课笔记22_6.1输入手写数字图片输出识别结果

   附:课程链接 第六讲.全连接网络实践 6.1输入手写数字图片输出识别结果 由于个人使用Win7系统,并未完全按照课程所讲,以下记录的也基本是我的结合课程做的Windows系统+PyCharm操作.且本 ...

5. tensorflow应用：双向LSTM神经网络手写数字识别

   tensorflow应用:双向LSTM神经网络手写数字识别 思路 Python程序1.建模训练保存 Tensorboard检查计算图及训练结果 打开训练好的模型进行预测 思路 将28X28的图片看成2 ...

6. 前馈神经网络手写数字识别

   前馈神经网络手写数字识别 今天就来说说手写数字识别,我们上课的时候老师要求我们使用前馈神经网络和卷积神经网络两种神经网络实现手写数字识别.做一下这两个实验还真的挺有意思啊. 举个例子,识别图片中的 : ...

7. 机器学习Tensorflow基于MNIST数据集识别自己的手写数字（读取和测试自己的模型）

   机器学习Tensorflow基于MNIST数据集识别自己的手写数字(读取和测试自己的模型)

8. 【手写数字识别】RBM神经网络手写数字识别【含GUI Matlab源码 1109期】

   ⛄一.手写数字识别技术简介 1 案例背景 手写体数字识别是图像识别学科下的一个分支,是图像处理和模式识别研究领域的重要应用之一,并且具有很强的通用性.由于手写体数字的随意性很大,如笔画粗细.字体大小. ...

9. cs224w（图机器学习）2021冬季课程学习笔记10 Applications of Graph Neural Networks

   诸神缄默不语-个人CSDN博文目录 cs224w(图机器学习)2021冬季课程学习笔记集合 文章目录 1. Graph Augmentation for GNNs 1.1 图特征增强Feature A ...

最新文章

1. 数据库定义语言常见关键字
2. 汉字转拼音（VB版）
3. 基于JAVA+Servlet+JSP+MYSQL的企业车库管理系统
4. 模块公有包中不能导入私有包的内容
5. HDU2027 统计元音【文本处理】
6. python免费教学视频400集-如何入门 Python 爬虫？400集免费教程视频带你从0-1全面掌握...
7. 【NOIP2009】【codevs1174】靶形数独
8. 04 - Masonry使用练习
9. 布局篇(1)—If you love css …
10. 他们连忙跟了上去ZuL1VsLa5X67XOI
11. 联想笔记本进入不了BIOS的解决方法
12. Photoshop2019 系统错误 无法启动此程序修复教程
13. Unity3d模型，加入Animator后，播放动画，物体回归原位处理
14. 如何用python可视化疫情风险区？地图标记位置信息：数据爬取+地址解析+可视化
15. matlab画图中颜色渐变
16. Rocky Linux 8 安装 socks5 服务器步骤
17. 2023年广东数据分析师CPDA认证招生简章（理论+实战）
18. jacob操作word和excel
19. linux樱桃树软件名字,名字带“樱”的3种水果，多数人最先想到樱桃，第3种最少见！...
20. Linux用键盘演奏音乐,分享|让我们使用 PC 键盘在终端演奏钢琴

热门文章

1. python 串口示波器
2. C语言控制桌面背景图
3. c语言中怎么对圆环填充颜色,ps里怎么把圆环填充两种颜色
4. bat脚本执行sql脚本
5. Vector使用报错:Expression:vector subscript out of range
6. 【JVM · GC】垃圾回收器
7. JVM上篇（13）：GC分类器
8. C#如何判断某个日期是否为今天
9. 2014522420145238 《信息安全系统设计基础》 第五次实验
10. 弱电时间同步系统（卫星同步时钟）医院学校标准时钟系统建设的意义