Squeeze Net

1602_Squeeze Net

图：

网络描述：

Squeeze Net 发表于ICLR-2017，作者分别来自Berkeley和Stanford，Squeeze Net不是模型压缩技术，而是 “design strategies for CNN architectures with few parameters” 。 Squeeze Net是Han等提出的一种轻量且高效的CNN模型，它参数比AlexNet少50x，但模型性能（accuracy）与AlexNet接近。SqueezeNet的核心在于Fire module，Fire module 由两层构成，分别是squeeze层+expand层，如上图所示，squeeze层是一个1x1卷积核的卷积层，expand层是1x1和3x3卷积核的卷积层，expand层中，把1x1 和3x3 得到的feature map 进行concat。

特点，优点：

(1) 提出了新的网络架构Fire Module，通过减少参数来进行模型压缩

(2) 使用其他方法对提出的Squeeze Net模型进行进一步压缩，

(3) 对参数空间进行了探索，主要研究了压缩比和3∗3，3∗3卷积比例的影响

(4) 更高效的分布式训练，小模型参数小，网络通信量减少；

(5) 便于模型更新，模型小，客户端程序容易更新；

(6) 利于部署在特定硬件如FPGA，因为其内存受限。因此研究小模型是很有现实意义的。

其他方法：使用以下三个策略来减少Squeeze Net设计参数

1.使用1∗1卷积代替3∗3 卷积：参数减少为原来的1/9

2.减少输入通道数量：这一部分使用squeeze layers来实现

3.将池化层操作延后，可以给卷积层提供更大的激活图：更大的激活图保留了更多的信息，可以提供更高的分类准确率

代码：

keras实现：

def fire_module(x, fire_id, squeeze=16, expand=64):s_id = 'fire' + str(fire_id) + '/'if K.image_data_format() == 'channels_first':channel_axis = 1else:channel_axis = 3x = Convolution2D(squeeze, (1, 1), padding='valid', name=s_id + sq1x1)(x)x = Activation('relu', name=s_id + relu + sq1x1)(x)left = Convolution2D(expand, (1, 1), padding='valid', name=s_id + exp1x1)(x)left = Activation('relu', name=s_id + relu + exp1x1)(left)right = Convolution2D(expand, (3, 3), padding='same', name=s_id + exp3x3)(x)right = Activation('relu', name=s_id + relu + exp3x3)(right)x = concatenate([left, right], axis=channel_axis, name=s_id + 'concat')return x# Original SqueezeNet from paper.
def SqueezeNet(include_top=True, weights='imagenet',input_tensor=None, input_shape=None,pooling=None,classes=1000):"""Instantiates the SqueezeNet architecture."""if weights not in {'imagenet', None}:raise ValueError('The `weights` argument should be either ''`None` (random initialization) or `imagenet` ''(pre-training on ImageNet).')if weights == 'imagenet' and classes != 1000:raise ValueError('If using `weights` as imagenet with `include_top`'' as true, `classes` should be 1000')input_shape = _obtain_input_shape(input_shape,default_size=227,min_size=48,data_format=K.image_data_format(),require_flatten=include_top)if input_tensor is None:img_input = Input(shape=input_shape)else:if not K.is_keras_tensor(input_tensor):img_input = Input(tensor=input_tensor, shape=input_shape)else:img_input = input_tensorx = Convolution2D(64, (3, 3), strides=(2, 2), padding='valid', name='conv1')(img_input)x = Activation('relu', name='relu_conv1')(x)x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x)x = fire_module(x, fire_id=2, squeeze=16, expand=64)x = fire_module(x, fire_id=3, squeeze=16, expand=64)x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool3')(x)x = fire_module(x, fire_id=4, squeeze=32, expand=128)x = fire_module(x, fire_id=5, squeeze=32, expand=128)x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool5')(x)x = fire_module(x, fire_id=6, squeeze=48, expand=192)x = fire_module(x, fire_id=7, squeeze=48, expand=192)x = fire_module(x, fire_id=8, squeeze=64, expand=256)x = fire_module(x, fire_id=9, squeeze=64, expand=256)if include_top:# It's not obvious where to cut the network... # Could do the 8th or 9th layer... some work recommends cutting earlier layers.x = Dropout(0.5, name='drop9')(x)x = Convolution2D(classes, (1, 1), padding='valid', name='conv10')(x)x = Activation('relu', name='relu_conv10')(x)x = GlobalAveragePooling2D()(x)x = Activation('softmax', name='loss')(x)else:if pooling == 'avg':x = GlobalAveragePooling2D()(x)elif pooling=='max':x = GlobalMaxPooling2D()(x)elif pooling==None:passelse:raise ValueError("Unknown argument for 'pooling'=" + pooling)# Ensure that the model takes into account# any potential predecessors of `input_tensor`.if input_tensor is not None:inputs = get_source_inputs(input_tensor)else:inputs = img_inputmodel = Model(inputs, x, name='squeezenet')# load weightsif weights == 'imagenet':if include_top:weights_path = get_file('squeezenet_weights_tf_dim_ordering_tf_kernels.h5',WEIGHTS_PATH,cache_subdir='models')else:weights_path = get_file('squeezenet_weights_tf_dim_ordering_tf_kernels_notop.h5',WEIGHTS_PATH_NO_TOP,cache_subdir='models')model.load_weights(weights_path)if K.backend() == 'theano':layer_utils.convert_all_kernels_in_model(model)if K.image_data_format() == 'channels_first':if K.backend() == 'tensorflow':warnings.warn('You are using the TensorFlow backend, yet you ''are using the Theano ''image data format convention ''(`image_data_format="channels_first"`). ''For best performance, set ''`image_data_format="channels_last"` in ''your Keras config ''at ~/.keras/keras.json.')return model

pytorch实现：

class Fire(nn.Module):def __init__(self, inplanes, squeeze_planes,expand1x1_planes, expand3x3_planes):super(Fire, self).__init__()self.inplanes = inplanesself.squeeze = nn.Conv2d(inplanes, squeeze_planes, kernel_size=1)self.squeeze_activation = nn.ReLU(inplace=True)self.expand1x1 = nn.Conv2d(squeeze_planes, expand1x1_planes,kernel_size=1)self.expand1x1_activation = nn.ReLU(inplace=True)self.expand3x3 = nn.Conv2d(squeeze_planes, expand3x3_planes,kernel_size=3, padding=1)self.expand3x3_activation = nn.ReLU(inplace=True)def forward(self, x):x = self.squeeze_activation(self.squeeze(x))return torch.cat([self.expand1x1_activation(self.expand1x1(x)),self.expand3x3_activation(self.expand3x3(x))], 1)class SqueezeNet(nn.Module):def __init__(self, version='1_0', num_classes=1000):super(SqueezeNet, self).__init__()self.num_classes = num_classesif version == '1_0':self.features = nn.Sequential(nn.Conv2d(3, 96, kernel_size=7, stride=2),nn.ReLU(inplace=True),nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True),Fire(96, 16, 64, 64),Fire(128, 16, 64, 64),Fire(128, 32, 128, 128),nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True),Fire(256, 32, 128, 128),Fire(256, 48, 192, 192),Fire(384, 48, 192, 192),Fire(384, 64, 256, 256),nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True),Fire(512, 64, 256, 256),)elif version == '1_1':self.features = nn.Sequential(nn.Conv2d(3, 64, kernel_size=3, stride=2),nn.ReLU(inplace=True),nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True),Fire(64, 16, 64, 64),Fire(128, 16, 64, 64),nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True),Fire(128, 32, 128, 128),Fire(256, 32, 128, 128),nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True),Fire(256, 48, 192, 192),Fire(384, 48, 192, 192),Fire(384, 64, 256, 256),Fire(512, 64, 256, 256),)else:# FIXME: Is this needed? SqueezeNet should only be called from the# FIXME: squeezenet1_x() functions# FIXME: This checking is not done for the other modelsraise ValueError("Unsupported SqueezeNet version {version}:""1_0 or 1_1 expected".format(version=version))# Final convolution is initialized differently from the restfinal_conv = nn.Conv2d(512, self.num_classes, kernel_size=1)self.classifier = nn.Sequential(nn.Dropout(p=0.5),final_conv,nn.ReLU(inplace=True),nn.AdaptiveAvgPool2d((1, 1)))for m in self.modules():if isinstance(m, nn.Conv2d):if m is final_conv:init.normal_(m.weight, mean=0.0, std=0.01)else:init.kaiming_uniform_(m.weight)if m.bias is not None:init.constant_(m.bias, 0)def forward(self, x):x = self.features(x)x = self.classifier(x)return torch.flatten(x, 1)

Squeeze Net相关推荐

pytorch学习中 torch.squeeze() 和torch.unsqueeze()的用法
squeeze的用法主要就是对数据的维度进行压缩或者解压. 先看torch.squeeze() 这个函数主要对数据的维度进行压缩,去掉维数为1的的维度,比如是一行或者一列这种,一个一行三列(1,3)的 ...
pytorch之expand，gather，squeeze，sum，contiguous，softmax，max，argmax
目录 gather squeeze expand sum contiguous softmax max argmax gather torch.gather(input,dim,index,out=N ...
PyTorch 笔记（04）— Tensor 属性方法（获取元素个数numel/neleme、查看形状size()/shape、增减维度squeeze()/unsqueeze()、resize形状）
1. 获取 Tensor 元素个数获取 Tensor 的元素个数 ,a.numel() 等价 a.nelement() In [1]: import torch as t In [5]: a = t ...
tf.squeeze示例代码
官网上对此方法有介绍,但缺少验证示例,而百度上搜到的都是从官网上翻译过来的,正确倒都是正确的,只是缺少验证示例,这边写一个示例代码,希望能帮助理解 import tensorflow as tfk = ...
报错解决：InvalidArgumentError: Can not squeeze dim[1], expected a dimension of 1, got
报错解决:InvalidArgumentError: Can not squeeze dim[1], expected a dimension of 1, got 101 晚上在使用tensorflo ...
pytorch中tensor的unsqueeze()函数和squeeze()函数的用处
unsqueeze()用于增加一个维度. 先假设有如下一维的Tensor. a=torch.Tensor([1,2]) print(a.shape) 假设我们现在有一个2*2的矩阵b,要与a相乘,最规 ...
Debian6 Squeeze上安装xen
2019独角兽企业重金招聘Python工程师标准>>> 现在服务器虚拟化趋势越来越明显了,一方面服务器硬件越来越强大,一台服务器只跑一个系统非常浪费,另一方面虚拟机管理比物理机方便得 ...
Pytorch view()、squeeze()、unsqueeze()、torch.max()
本篇博客主要向大家介绍Pytorch中view().squeeze().unsqueeze().torch.max()函数,这些函数虽然简单,但是在神经网络编程总却经常用到,希望大家看了这篇博文能够 ...
Can't create layer 289 of type Squeeze in function 'cv::dnn::dnn4_v20190902::LayerData::getLaye
Can't create layer "289" of type "Squeeze" in function 'cv::dnn::dnn4_v20190902: ...
torch.squeeze()和unsqueeze()
torch.squeeze()和unsqueeze() unsqueeze() 函数功能:与squeeze()函数功能相反,用于添加维度. queeze() 函数功能:去除size为1的维度,包括行和 ...