深度学习2.0-33.BatchNorm

文章目录

BatchNorm
- Batch from Image
- 效果
- 实战

BatchNorm

Batch from Image

效果

实战

import osos.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'# 解决了UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [Op:Conv2D]
from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSessionconfig = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, optimizers# 2 images with 4x4 size, 3 channels
# we explicitly enforce the mean and stddev to N(1, 0.5)
x = tf.random.normal([2, 4, 4, 3], mean=1., stddev=0.5)net = layers.BatchNormalization(axis=-1, center=True, scale=True,trainable=True)out = net(x)
print('forward in test mode:', net.variables)out1 = net(x, training=True)
print('forward in train mode(1 step):', net.variables)for i in range(100):out = net(x, training=True)
print('forward in train mode(100 steps):', net.variables)optimizer = optimizers.SGD(lr=1e-2)
for i in range(10):with tf.GradientTape() as tape:out = net(x, training=True)loss = tf.reduce_mean(tf.pow(out, 2)) - 1grads = tape.gradient(loss, net.trainable_variables)optimizer.apply_gradients(zip(grads, net.trainable_variables))
# 测试集中没有backward-更新β、γ
print('backward(10 steps):', net.variables)

forward in test mode: [<tf.Variable 'batch_normalization/gamma:0' shape=(3,) dtype=float32, numpy=array([1., 1., 1.], dtype=float32)>, <tf.Variable 'batch_normalization/beta:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>, <tf.Variable 'batch_normalization/moving_mean:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>, <tf.Variable 'batch_normalization/moving_variance:0' shape=(3,) dtype=float32, numpy=array([1., 1., 1.], dtype=float32)>]
forward in train mode(1 step): [<tf.Variable 'batch_normalization/gamma:0' shape=(3,) dtype=float32, numpy=array([1., 1., 1.], dtype=float32)>, <tf.Variable 'batch_normalization/beta:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>, <tf.Variable 'batch_normalization/moving_mean:0' shape=(3,) dtype=float32, numpy=array([0.00976992, 0.0118203 , 0.00990692], dtype=float32)>, <tf.Variable 'batch_normalization/moving_variance:0' shape=(3,) dtype=float32, numpy=array([0.9930444, 0.9923902, 0.9919932], dtype=float32)>]
forward in train mode(100 steps): [<tf.Variable 'batch_normalization/gamma:0' shape=(3,) dtype=float32, numpy=array([1., 1., 1.], dtype=float32)>, <tf.Variable 'batch_normalization/beta:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>, <tf.Variable 'batch_normalization/moving_mean:0' shape=(3,) dtype=float32, numpy=array([0.6229577 , 0.75369585, 0.6316934 ], dtype=float32)>, <tf.Variable 'batch_normalization/moving_variance:0' shape=(3,) dtype=float32, numpy=array([0.55648977, 0.51477736, 0.48946366], dtype=float32)>]
backward(10 steps): [<tf.Variable 'batch_normalization/gamma:0' shape=(3,) dtype=float32, numpy=array([0.9355103, 0.9355681, 0.9356216], dtype=float32)>, <tf.Variable 'batch_normalization/beta:0' shape=(3,) dtype=float32, numpy=array([ 3.4645198e-09, -1.7657877e-08,  1.3411044e-09], dtype=float32)>, <tf.Variable 'batch_normalization/moving_mean:0' shape=(3,) dtype=float32, numpy=array([0.6568097 , 0.7946524 , 0.66602015], dtype=float32)>, <tf.Variable 'batch_normalization/moving_variance:0' shape=(3,) dtype=float32, numpy=array([0.5323891 , 0.4884099 , 0.46172065], dtype=float32)>]

深度学习2.0-33.BatchNorm相关推荐

深度学习笔记（33）特征点检测
深度学习笔记(33) 特征点检测 1. 特征点 2. 设定特征点 3. 关键特征点 1. 特征点利用神经网络进行对象定位,即通过输出四个参数值bx.by.bh和bw给出图片中对象的边界框概括地说, ...
halcon 深度学习标注_HALCON深度学习工具0.4 早鸟版发布了
原标题:HALCON深度学习工具0.4 早鸟版发布了 HALOCN深度学习工具在整个深度学习过程中扮演着重要的作用,而且在将来将扮演更重要的辅助作用,大大加快深度学习的开发流程,目前发布版本工具的主要 ...
halcon显示坐标_HALCON深度学习工具0.4 早鸟版发布了
HALOCN深度学习工具在整个深度学习过程中扮演着重要的作用,而且在将来将扮演更重要的辅助作用,大大加快深度学习的开发流程,目前发布版本工具的主要作用是图像数据处理和目标检测和分类中的标注. 标注训练 ...
【深度学习】重新思考BatchNorm中的 “Batch”
这篇很有趣且很有用,激动的赶紧把文章看了一遍,不愧是FAIR,实验看的太爽了. 之前对于Norm的研究主要在于改变Norm的维度,然后衍生出了BatchNorm.GroupNorm.InstanceN ...
神经网络与深度学习——TensorFlow2.0实战(笔记)(二)(开发环境介绍)
开发环境介绍 Python3 1.结构清晰,简单易学 2.丰富的标准库 3.强大的的第三方生态系统 4.开源.开放体系 5.高可扩展性:胶水语言 6.高可扩展性:胶水语言 7.解释型语言,实现复杂算法 ...
资源下载| 深度学习Pytoch1.0如何玩？这一门含900页ppt和代码实例的深度学习课程带你飞
本文来自专知近日,在NeurIPS 2018 大会上,Facebook 官方宣布 PyTorch 1.0 正式版发布了.如何用Pytorch1.0搞深度学习?对很多小白学生是个问题.瑞士非盈利研究机 ...
深度学习_TensorFlow2.0基础_张量创建，运算，维度变换，采样
Tensorflow2.0 基础一:TensorFlow特性 1.TensorFlow An end-to-end open source machine learning platform end ...
深度学习-计算机视觉-0基础-学习历程
周志华<机器学习>------------------------若是想从基础算法公式开始可以先试着看一下周志华的<机器学习>,由于我对公式推导很头疼,看了几页就跳过了.(在经 ...
动手学深度学习V2.0(Pytorch)——11.模型选择+过拟合和欠拟合
文章目录 1. 模型选择 2. 过拟合和欠拟合 3. 代码 4. Q&A 4.1 SVM和神经网络相比,缺点在哪里 4.2 训练集验证集测试集比例 4.3 时序预测问题中的测试集训练集 4.4 ...
《深度学习从0开始》
1.感知机给定输入x, 权重w, 和偏移 b, 感知机输出: o=σ(<w,x>+b)o = \sigma(<w,x> + b)o=σ(<w,x>+b) (< ...

深度学习2.0-33.BatchNorm

文章目录

BatchNorm

Batch from Image

效果

实战

深度学习2.0-33.BatchNorm相关推荐

最新文章

热门文章