slim入门教程

slim入门教程
- 1. Variable
- 2. Layers
  - 2.1 slim.bias_add
  - 2.2 slim.batch_norm
  - 2.3 slim.conv2d
  - 2.4 slim.conv2d_in_plane
  - 2.5 slim.conv2d_transpose
  - 2.6 slim.fully_connected
  - 2.7 slim.avg_pool2d
  - 2.8 slim.dropout
  - 2.9 slim.flatten
  - 2.10 slim.max_pool2d
  - 2.11 slim.one_hot_encoding
  - 2.12 slim.separable_conv2d
  - 2.13 slim.unit_norm
  - 2.14 slim.repeat
  - 2.15 slim.stack

import tensorflow.contrib.slim as slim

1. Variable

TF Slim提供一个高级封装：

weights = slim.variable('weights',shape=[10, 10, 3 , 3],initializer=tf.truncated_normal_initializer(stddev=0.1),regularizer=slim.l2_regularizer(0.05),device='/CPU:0')

参数说明：
- [x] slim.variable( )
- [x] slim.model_variable( )默认加入tf.GraphKeys.MODEL_VARIABLES

Args:
name: 变量名称.
shape: shape of the new or existing variable.
dtype: type of the new or existing variable (defaults to DT_FLOAT).
initializer: initializer for the variable if one is created.
regularizer: a (Tensor -> Tensor or None) function; the result of
applying it on a newly created variable will be added to the collection
GraphKeys.REGULARIZATION_LOSSES and can be used for regularization.
trainable: If True also add the variable to the graph collection
GraphKeys.TRAINABLE_VARIABLES (see tf.Variable).
collections: A list of collection names to which the Variable will be added.
If None it would default to tf.GraphKeys.GLOBAL_VARIABLES.
caching_device: Optional device string or function describing where the
Variable should be cached for reading. Defaults to the Variable’s
device.
device: Optional device to place the variable. It can be an string or a
function that is called to get the device for the variable.
partitioner: Optional callable that accepts a fully defined TensorShape
and dtype of the Variable to be created, and returns a list of
partitions for each axis (currently only one axis can be partitioned).
custom_getter: Callable that allows overwriting the internal
get_variable method and has to have the same signature.
**use_resource: If True use a ResourceVariable instead of a Variable.
**use_resource: If True use a ResourceVariable instead of a Variable.

Returns:
The created or existing variable.

Tensorflow里提供了regular variable（可以通过saver保存的）和local variable（只在会话期间的临时变量）。在此基础上，TF Slim提供了model variable（模型参数，可以训练和微调的从checkpoint导入的参数），比如global_step就不是model variable，类似的，moving average可以mirror全局变量，但是它本身不是全局变量。

# Model Variables
weights = slim.model_variable('weights',shape=[10, 10, 3 , 3],initializer=tf.truncated_normal_initializer(stddev=0.1),regularizer=slim.l2_regularizer(0.05),device='/CPU:0')
model_variables = slim.get_model_variables()# Regular variables
my_var = slim.variable('my_var',shape=[20, 1],initializer=tf.zeros_initializer())
regular_variables_and_model_variables = slim.get_variables()

2. Layers

原生的TensorFlow操作比较底层，TF Slim提供了一些高级封装。

2.1 slim.bias_add

@add_arg_scope
def bias_add(inputs,activation_fn=None,initializer=init_ops.zeros_initializer(),regularizer=None,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,data_format=DATA_FORMAT_NHWC,scope=None):"""Adds a bias to the inputs.Can be used as a normalizer function for conv2d and fully_connected.Args:inputs: A tensor of with at least rank 2 and value for the last dimension,e.g. `[batch_size, depth]`, `[None, None, None, depth]`.activation_fn: Activation function, default set to None to skip it andmaintain a linear activation.initializer: An initializer for the bias, defaults to 0.regularizer: A regularizer like the result of`l1_regularizer` or `l2_regularizer`.reuse: Whether or not the layer and its variables should be reused. To beable to reuse the layer scope must be given.variables_collections: Optional collections for the variables.outputs_collections: Collections to add the outputs.trainable: If `True` also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES` (see tf.Variable).data_format: A string. 'NHWC' and 'NCHW' are supported.scope: Optional scope for variable_scope.Returns:A tensor representing the result of adding biases to the inputs.Raises:ValueError: If `data_format` is neither `NHWC` nor `NCHW`.ValueError: If `data_format` is `NCHW` and rank of `inputs` is not 4.ValueError: If the rank of `inputs` is undefined.ValueError: If rank or `C` dimension of `inputs` is undefined."""

2.2 slim.batch_norm

@add_arg_scope
def batch_norm(inputs,decay=0.999,center=True,scale=False,epsilon=0.001,activation_fn=None,param_initializers=None,param_regularizers=None,updates_collections=ops.GraphKeys.UPDATE_OPS,is_training=True,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,batch_weights=None,fused=None,data_format=DATA_FORMAT_NHWC,zero_debias_moving_mean=False,scope=None,renorm=False,renorm_clipping=None,renorm_decay=0.99,adjustment=None):"""Adds a Batch Normalization layer from http://arxiv.org/abs/1502.03167."Batch Normalization: Accelerating Deep Network Training by ReducingInternal Covariate Shift"Sergey Ioffe, Christian SzegedyCan be used as a normalizer function for conv2d and fully_connected. Thenormalization is over all but the last dimension if `data_format` is `NHWC`and all but the second dimension if `data_format` is `NCHW`.  In case of a 2Dtensor this corresponds to the batch dimension, while in case of a 4D tensorthiscorresponds to the batch and space dimensions.Note: when training, the moving_mean and moving_variance need to be updated.By default the update ops are placed in `tf.GraphKeys.UPDATE_OPS`, so theyneed to be added as a dependency to the `train_op`. For example:update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)with tf.control_dependencies(update_ops):train_op = optimizer.minimize(loss)One can set updates_collections=None to force the updates in place, but thatcan have a speed penalty, especially in distributed settings.Args:inputs: A tensor with 2 or more dimensions, where the first dimension has`batch_size`. The normalization is over all but the last dimension if`data_format` is `NHWC` and the second dimension if `data_format` is`NCHW`.decay: Decay for the moving average. Reasonable values for `decay` are closeto 1.0, typically in the multiple-nines range: 0.999, 0.99, 0.9, etc.Lower `decay` value (recommend trying `decay`=0.9) if model experiencesreasonably good training performance but poor validation and/or testperformance. Try zero_debias_moving_mean=True for improved stability.center: If True, add offset of `beta` to normalized tensor. If False, `beta`is ignored.scale: If True, multiply by `gamma`. If False, `gamma` isnot used. When the next layer is linear (also e.g. `nn.relu`), this can bedisabled since the scaling can be done by the next layer.epsilon: Small float added to variance to avoid dividing by zero.activation_fn: Activation function, default set to None to skip it andmaintain a linear activation.param_initializers: Optional initializers for beta, gamma, moving mean andmoving variance.param_regularizers: Optional regularizer for beta and gamma.updates_collections: Collections to collect the update ops for computation.The updates_ops need to be executed with the train_op.If None, a control dependency would be added to make sure the updates arecomputed in place.is_training: Whether or not the layer is in training mode. In training modeit would accumulate the statistics of the moments into `moving_mean` and`moving_variance` using an exponential moving average with the given`decay`. When it is not in training mode then it would use the values ofthe `moving_mean` and the `moving_variance`.reuse: Whether or not the layer and its variables should be reused. To beable to reuse the layer scope must be given.variables_collections: Optional collections for the variables.outputs_collections: Collections to add the outputs.trainable: If `True` also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES` (see `tf.Variable`).batch_weights: An optional tensor of shape `[batch_size]`,containing a frequency weight for each batch item. If present,then the batch normalization uses weighted mean andvariance. (This can be used to correct for bias in trainingexample selection.)fused: if `None` or `True`, use a faster, fused implementation if possible.If `False`, use the system recommended implementation.data_format: A string. `NHWC` (default) and `NCHW` are supported.zero_debias_moving_mean: Use zero_debias for moving_mean. It creates a newpair of variables 'moving_mean/biased' and 'moving_mean/local_step'.scope: Optional scope for `variable_scope`.renorm: Whether to use Batch Renormalization(https://arxiv.org/abs/1702.03275). This adds extra variables duringtraining. The inference is the same for either value of this parameter.renorm_clipping: A dictionary that may map keys 'rmax', 'rmin', 'dmax' toscalar `Tensors` used to clip the renorm correction. The correction`(r, d)` is used as `corrected_value = normalized_value * r + d`, with`r` clipped to [rmin, rmax], and `d` to [-dmax, dmax]. Missing rmax, rmin,dmax are set to inf, 0, inf, respectively.renorm_decay: Momentum used to update the moving means and standarddeviations with renorm. Unlike `momentum`, this affects trainingand should be neither too small (which would add noise) nor too large(which would give stale estimates). Note that `decay` is still appliedto get the means and variances for inference.adjustment: A function taking the `Tensor` containing the (dynamic) shape ofthe input tensor and returning a pair (scale, bias) to apply to thenormalized values (before gamma and beta), only during training. Forexample,`adjustment = lambda shape: (tf.random_uniform(shape[-1:], 0.93, 1.07),tf.random_uniform(shape[-1:], -0.1, 0.1))`will scale the normalized value by up to 7% up or down, then shift theresult by up to 0.1 (with independent scaling and bias for each featurebut shared across all examples), and finally apply gamma and/or beta. If`None`, no adjustment is applied.Returns:A `Tensor` representing the output of the operation.Raises:ValueError: If `data_format` is neither `NHWC` nor `NCHW`.ValueError: If the rank of `inputs` is undefined.ValueError: If rank or channels dimension of `inputs` is undefined."""

2.3 slim.conv2d

@add_arg_scope
def convolution2d(inputs,num_outputs,kernel_size,stride=1,padding='SAME',data_format=None,rate=1,activation_fn=nn.relu,normalizer_fn=None,normalizer_params=None,weights_initializer=initializers.xavier_initializer(),weights_regularizer=None,biases_initializer=init_ops.zeros_initializer(),biases_regularizer=None,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,scope=None)Returns:A tensor representing the output of the operation.

2.4 slim.conv2d_in_plane

def convolution2d_in_plane(inputs,kernel_size,stride=1,padding='SAME',activation_fn=nn.relu,normalizer_fn=None,normalizer_params=None,weights_initializer=initializers.xavier_initializer(),weights_regularizer=None,biases_initializer=init_ops.zeros_initializer(),biases_regularizer=None,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,scope=None)

2.5 slim.conv2d_transpose

def convolution2d_transpose(inputs,num_outputs,kernel_size,stride=1,padding='SAME',data_format=DATA_FORMAT_NHWC,activation_fn=nn.relu,normalizer_fn=None,normalizer_params=None,weights_initializer=initializers.xavier_initializer(),weights_regularizer=None,biases_initializer=init_ops.zeros_initializer(),biases_regularizer=None,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,scope=None):

2.6 slim.fully_connected

def fully_connected(inputs,num_outputs,activation_fn=nn.relu,normalizer_fn=None,normalizer_params=None,weights_initializer=initializers.xavier_initializer(),weights_regularizer=None,biases_initializer=init_ops.zeros_initializer(),biases_regularizer=None,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,scope=None):

2.7 slim.avg_pool2d

def avg_pool2d(inputs,kernel_size,stride=2,padding='VALID',data_format=DATA_FORMAT_NHWC,outputs_collections=None,scope=None)

2.8 slim.dropout

def dropout(inputs,keep_prob=0.5,noise_shape=None,is_training=True,outputs_collections=None,scope=None,seed=None)

2.9 slim.flatten

def flatten(inputs, outputs_collections=None, scope=None)

2.10 slim.max_pool2d

def max_pool2d(inputs,kernel_size,stride=2,padding='VALID',data_format=DATA_FORMAT_NHWC,outputs_collections=None,scope=None)

2.11 slim.one_hot_encoding

def one_hot_encoding(labels,num_classes,on_value=1.0,off_value=0.0,outputs_collections=None,scope=None)

2.12 slim.separable_conv2d

def separable_convolution2d(inputs,num_outputs,kernel_size,depth_multiplier,stride=1,padding='SAME',data_format=DATA_FORMAT_NHWC,rate=1,activation_fn=nn.relu,normalizer_fn=None,normalizer_params=None,weights_initializer=initializers.xavier_initializer(),pointwise_initializer=None,weights_regularizer=None,biases_initializer=init_ops.zeros_initializer(),biases_regularizer=None,reuse=None,variables_collections=None,outputs_collections=None,trainable=True,scope=None)

2.13 slim.unit_norm

def unit_norm(inputs, dim, epsilon=1e-7, scope=None)

2.14 slim.repeat

def repeat(inputs, repetitions, layer, *args, **kwargs):"""Applies the same layer with the same arguments repeatedly.

y = repeat(x, 3, conv2d, 64, [3, 3], scope='conv1')
'''
相当于：
'''
x = conv2d(x, 64, [3, 3], scope='conv1/conv1_1')
x = conv2d(x, 64, [3, 3], scope='conv1/conv1_2')
y = conv2d(x, 64, [3, 3], scope='conv1/conv1_3')

repeat会自动分配scope嵌套。

2.15 slim.stack

# Verbose way:
x = slim.conv2d(x, 32, [3, 3], scope='core/core_1')
x = slim.conv2d(x, 32, [1, 1], scope='core/core_2')
x = slim.conv2d(x, 64, [3, 3], scope='core/core_3')
x = slim.conv2d(x, 64, [1, 1], scope='core/core_4')# Using stack:
slim.stack(x, slim.conv2d, [(32, [3, 3]), (32, [1, 1]), (64, [3, 3]), (64, [1, 1])], scope='core')

Tensorflow Slim入门教程（1）相关推荐

Tensorflow keras入门教程
目录 1.Tensorflow与Keras 2.安装内置Keras的Tensorflow 3.Tensorflow内置的Keras教程 3.1.导入tf.keras 3.2.创建一个简单的模型 3.2 ...
tensorflow slim 安装教程
参考链接: https://www.cnblogs.com/zyly/p/9145081.html https://blog.csdn.net/qq_27882063/article/details/ ...
TensorFlow v1 入门教程
目录 Tensor 计算图 Eager mode Tensorboard Save and Restore 模型保存模型恢复 tensorflow v2 版本现在已经全面用keras 封装了,运行时 ...
TensorFlow人工智能入门教程之十一最强网络DLSTM 双向长短期记忆网络（阿里小AI实现）...
2019独角兽企业重金招聘Python工程师标准>>> 失眠 ....上一章讲了最强网络之一 RSNN 深度残差网络这一章节我们来讲讲还有一个很强的网络模型,就是双向LS ...
python tensorflow教程_TensorFlow入门教程TensorFlow 基本使用T
该楼层疑似违规已被系统折叠隐藏此楼查看此楼 TensorFlow入门教程 TensorFlow 基本使用 TensorFlow官方中文教程 TensorFlow 的特点: 使用图 (graph) 来 ...
Tensorflow 入门教程
Tensorflow 入门教程 http://tensornews.cn/ 深度学习发展史特征工程深度学习之激活函数损失函数反向传播算法 [上] 反向传播算法 [下] Tensorflow ...
python中tensorflow_TensorFlow入门教程TensorFlow 基本使用T
该楼层疑似违规已被系统折叠隐藏此楼查看此楼 TensorFlow入门教程 TensorFlow 基本使用 TensorFlow官方中文教程 TensorFlow 的特点: 使用图 (graph) 来 ...
TensorFlow发布语音识别入门教程，附1GB数据集代码
原标题:TensorFlow发布语音识别入门教程,附1GB数据集&代码机械鸡的鸡友经常问:如何开始入门深度学习语音和其他音频识别,例如关键字检测或语音命令. 虽然有一些伟大的开源语音识别系统 ...
（转）tensorflow入门教程(二十六)人脸识别(上)
https://blog.csdn.net/rookie_wei/article/details/81676177 1.概述查看全文 http://www.taodudu.cc/news/show- ...

Tensorflow Slim入门教程（1）