一、over roll archi

Alexnet——2012年——标志性网络
该网络的亮点在于：
（1）首次利用GPU进行网络加速训练。
（2）使用了ReLU激活函数，而不是传统的Sigmoid激活函数以及Tanh激活函数。（smoid求导比较麻烦而且当网路比较深的时候会出现梯度消失）
（3）使用了LRN局部响应归一化。
（4）在全连接层的前两层中使用了Dropout随机失活神经元操作，以减少过拟合。

dropout解释：使用dropout后，在每一层中随机失活一些神经元——减少训练参数从而减少over fitting

二、结构详解

1. Conv1

2. Maxpool1

3. Conv2

4. Maxpool2

5. Conv3

6. Conv4

7. Conv5

8. Maxpool3

9. FC1

描述一下：这里使用4096个神经元，对256个大小为66特征图，进行一个全链接，也就是将66大小的特征图，进行卷积变为一个特征点,然后对于4096个神经元中的一个点，是由256个特征图中某些个特征图卷积之后得到的特征点乘以相应的权重之后，再加上一个偏置得到.　再进行一个dropout随机从4096个节点中丢掉一些节点信息（也就是值清0），然后就得到新的4096个神经元.

10. FC2

和fc1类似.

11. FC3

采用的是1000个神经元，然后对fc7中4096个神经元进行全链接，然后会通过高斯过滤器，得到1000个float型的值，也就是我们所看到的预测的可能性,

如果是训练模型的话，会通过标签label进行对比误差，然后求解出残差，再通过链式求导法则，将残差通过求解偏导数逐步向上传递，并将权重进行推倒更改，类似与BP网络思虑，然后会逐层逐层的调整权重以及偏置.

10. summary

三、tensorflow实现

model：

tensorflow实现：

import tensorflow as tfdef alexnet(x, keep_prob, num_classes):# conv1with tf.name_scope('conv1') as scope:kernel = tf.Variable(tf.truncated_normal([11, 11, 3, 96], dtype=tf.float32,stddev=1e-1), name='weights')conv = tf.nn.conv2d(x, kernel, [1, 4, 4, 1], padding='SAME')biases = tf.Variable(tf.constant(0.0, shape=[96], dtype=tf.float32),trainable=True, name='biases')bias = tf.nn.bias_add(conv, biases)conv1 = tf.nn.relu(bias, name=scope)# lrn1with tf.name_scope('lrn1') as scope:lrn1 = tf.nn.local_response_normalization(conv1,alpha=1e-4,beta=0.75,depth_radius=2,bias=2.0)# pool1with tf.name_scope('pool1') as scope:pool1 = tf.nn.max_pool(lrn1,ksize=[1, 3, 3, 1],strides=[1, 2, 2, 1],padding='VALID')# conv2with tf.name_scope('conv2') as scope:pool1_groups = tf.split(axis=3, value = pool1, num_or_size_splits = 2)kernel = tf.Variable(tf.truncated_normal([5, 5, 48, 256], dtype=tf.float32,stddev=1e-1), name='weights')kernel_groups = tf.split(axis=3, value = kernel, num_or_size_splits = 2)conv_up = tf.nn.conv2d(pool1_groups[0], kernel_groups[0], [1,1,1,1], padding='SAME')conv_down = tf.nn.conv2d(pool1_groups[1], kernel_groups[1], [1,1,1,1], padding='SAME')biases = tf.Variable(tf.constant(0.0, shape=[256], dtype=tf.float32),trainable=True, name='biases')biases_groups = tf.split(axis=0, value=biases, num_or_size_splits=2)bias_up = tf.nn.bias_add(conv_up, biases_groups[0])bias_down = tf.nn.bias_add(conv_down, biases_groups[1])bias = tf.concat(axis=3, values=[bias_up, bias_down])conv2 = tf.nn.relu(bias, name=scope)# lrn2with tf.name_scope('lrn2') as scope:lrn2 = tf.nn.local_response_normalization(conv2,alpha=1e-4,beta=0.75,depth_radius=2,bias=2.0)# pool2with tf.name_scope('pool2') as scope:pool2 = tf.nn.max_pool(lrn2,ksize=[1, 3, 3, 1],strides=[1, 2, 2, 1],padding='VALID')                         # conv3with tf.name_scope('conv3') as scope:kernel = tf.Variable(tf.truncated_normal([3, 3, 256, 384],dtype=tf.float32,stddev=1e-1), name='weights')conv = tf.nn.conv2d(pool2, kernel, [1, 1, 1, 1], padding='SAME')biases = tf.Variable(tf.constant(0.0, shape=[384], dtype=tf.float32),trainable=True, name='biases')bias = tf.nn.bias_add(conv, biases)conv3 = tf.nn.relu(bias, name=scope)# conv4with tf.name_scope('conv4') as scope:conv3_groups = tf.split(axis=3, value=conv3, num_or_size_splits=2)kernel = tf.Variable(tf.truncated_normal([3, 3, 192, 384],dtype=tf.float32,stddev=1e-1), name='weights')kernel_groups = tf.split(axis=3, value=kernel, num_or_size_splits=2)conv_up = tf.nn.conv2d(conv3_groups[0], kernel_groups[0], [1, 1, 1, 1], padding='SAME')conv_down = tf.nn.conv2d(conv3_groups[1], kernel_groups[1], [1,1,1,1], padding='SAME')biases = tf.Variable(tf.constant(0.0, shape=[384], dtype=tf.float32),trainable=True, name='biases')biases_groups = tf.split(axis=0, value=biases, num_or_size_splits=2)bias_up = tf.nn.bias_add(conv_up, biases_groups[0])bias_down = tf.nn.bias_add(conv_down, biases_groups[1])bias = tf.concat(axis=3, values=[bias_up,bias_down])conv4 = tf.nn.relu(bias, name=scope)# conv5with tf.name_scope('conv5') as scope:conv4_groups = tf.split(axis=3, value=conv4, num_or_size_splits=2)kernel = tf.Variable(tf.truncated_normal([3, 3, 192, 256],dtype=tf.float32,stddev=1e-1), name='weights')kernel_groups = tf.split(axis=3, value=kernel, num_or_size_splits=2)conv_up = tf.nn.conv2d(conv4_groups[0], kernel_groups[0], [1, 1, 1, 1], padding='SAME')conv_down = tf.nn.conv2d(conv4_groups[1], kernel_groups[1], [1,1,1,1], padding='SAME')biases = tf.Variable(tf.constant(0.0, shape=[256], dtype=tf.float32),trainable=True, name='biases')biases_groups = tf.split(axis=0, value=biases, num_or_size_splits=2)bias_up = tf.nn.bias_add(conv_up, biases_groups[0])bias_down = tf.nn.bias_add(conv_down, biases_groups[1])bias = tf.concat(axis=3, values=[bias_up,bias_down])conv5 = tf.nn.relu(bias, name=scope)# pool5with tf.name_scope('pool5') as scope:pool5 = tf.nn.max_pool(conv5,ksize=[1, 3, 3, 1],strides=[1, 2, 2, 1],padding='VALID',)# flattened6with tf.name_scope('flattened6') as scope:flattened = tf.reshape(pool5, shape=[-1, 6*6*256])# fc6with tf.name_scope('fc6') as scope:weights = tf.Variable(tf.truncated_normal([6*6*256, 4096],dtype=tf.float32,stddev=1e-1), name='weights')biases = tf.Variable(tf.constant(0.0, shape=[4096], dtype=tf.float32),trainable=True, name='biases')bias = tf.nn.xw_plus_b(flattened, weights, biases)fc6 = tf.nn.relu(bias)# dropout6with tf.name_scope('dropout6') as scope:dropout6 = tf.nn.dropout(fc6, keep_prob)# fc7with tf.name_scope('fc7') as scope:weights = tf.Variable(tf.truncated_normal([4096,4096],dtype=tf.float32,stddev=1e-1), name='weights')biases = tf.Variable(tf.constant(0.0, shape=[4096], dtype=tf.float32),trainable=True, name='biases')bias = tf.nn.xw_plus_b(dropout6, weights, biases)fc7 = tf.nn.relu(bias)# dropout7with tf.name_scope('dropout7') as scope:dropout7 = tf.nn.dropout(fc7, keep_prob)# fc8with tf.name_scope('fc8') as scope:weights = tf.Variable(tf.truncated_normal([4096, num_classes],dtype=tf.float32,stddev=1e-1), name='weights')biases = tf.Variable(tf.constant(0.0, shape=[num_classes], dtype=tf.float32),trainable=True, name='biases')fc8 = tf.nn.xw_plus_b(dropout7, weights, biases)return fc8

keras实现：（v1是是使用keras function api的方法搭建、v2是subclasses也就是子类的方法搭建-类似pytorch）

from tensorflow.keras import layers, models, Model, Sequentialdef AlexNet_v1(im_height=224, im_width=224, num_classes=1000):# tensorflow中的tensor通道排序是NHWCinput_image = layers.Input(shape=(im_height, im_width, 3), dtype="float32")  # output(None, 224, 224, 3)x = layers.ZeroPadding2D(((1, 2), (1, 2)))(input_image)                      # output(None, 227, 227, 3)x = layers.Conv2D(48, kernel_size=11, strides=4, activation="relu")(x)       # output(None, 55, 55, 48)x = layers.MaxPool2D(pool_size=3, strides=2)(x)                              # output(None, 27, 27, 48)x = layers.Conv2D(128, kernel_size=5, padding="same", activation="relu")(x)  # output(None, 27, 27, 128)x = layers.MaxPool2D(pool_size=3, strides=2)(x)                              # output(None, 13, 13, 128)x = layers.Conv2D(192, kernel_size=3, padding="same", activation="relu")(x)  # output(None, 13, 13, 192)x = layers.Conv2D(192, kernel_size=3, padding="same", activation="relu")(x)  # output(None, 13, 13, 192)x = layers.Conv2D(128, kernel_size=3, padding="same", activation="relu")(x)  # output(None, 13, 13, 128)x = layers.MaxPool2D(pool_size=3, strides=2)(x)                              # output(None, 6, 6, 128)x = layers.Flatten()(x)                         # output(None, 6*6*128)x = layers.Dropout(0.2)(x)x = layers.Dense(2048, activation="relu")(x)    # output(None, 2048)x = layers.Dropout(0.2)(x)x = layers.Dense(2048, activation="relu")(x)    # output(None, 2048)x = layers.Dense(num_classes)(x)                  # output(None, 5)predict = layers.Softmax()(x)model = models.Model(inputs=input_image, outputs=predict)return modelclass AlexNet_v2(Model):def __init__(self, num_classes=1000):super(AlexNet_v2, self).__init__()self.features = Sequential([layers.ZeroPadding2D(((1, 2), (1, 2))),                                 # output(None, 227, 227, 3)layers.Conv2D(48, kernel_size=11, strides=4, activation="relu"),        # output(None, 55, 55, 48)layers.MaxPool2D(pool_size=3, strides=2),                               # output(None, 27, 27, 48)layers.Conv2D(128, kernel_size=5, padding="same", activation="relu"),   # output(None, 27, 27, 128)layers.MaxPool2D(pool_size=3, strides=2),                               # output(None, 13, 13, 128)layers.Conv2D(192, kernel_size=3, padding="same", activation="relu"),   # output(None, 13, 13, 192)layers.Conv2D(192, kernel_size=3, padding="same", activation="relu"),   # output(None, 13, 13, 192)layers.Conv2D(128, kernel_size=3, padding="same", activation="relu"),   # output(None, 13, 13, 128)layers.MaxPool2D(pool_size=3, strides=2)])                              # output(None, 6, 6, 128)self.flatten = layers.Flatten()self.classifier = Sequential([layers.Dropout(0.2),layers.Dense(1024, activation="relu"),                                  # output(None, 2048)layers.Dropout(0.2),layers.Dense(128, activation="relu"),                                   # output(None, 2048)layers.Dense(num_classes),                                                # output(None, 5)layers.Softmax()])def call(self, inputs, **kwargs):x = self.features(inputs)x = self.flatten(x)x = self.classifier(x)return x

————————————————————————————————————————————————————————————————————————-
pytorch实现-reference：
https://github.com/dansuh17/alexnet-pytorch

https://github.com/sloth2012/AlexNet

Alexnet详解以及tesnsorflow实现alexnet；什么是alexnet alexnet能做什么；alexnet教程相关推荐

Hadoop-HDFS详解与HA，完全分布式集群搭建(细到令人发指的教程)
前言本篇篇幅较长,有许多集群搭建干货,和枯燥乏味但是面试可能问到的理论知识. 思来想去不知道怎样才能鼓励自己加油学习,想想要面对的生活还是假吧意思打开学习视频吧. 目录一.引入 hdfs是什么 h ...
mysql通配符escape使用_详解MySQL like如何查询包含#39;%#39;的字段(ESCAPE用法)-MySQL教程-Web开发者网...
在SQl like语句中,比如 SELECT * FROM user WHERE username LIKE '%luchi%' SELECT * FROM user WHERE username L ...
图像分类篇——AlexNet详解
一.概述 AlexNet是由2012年ImageNet竞赛参赛者Hinton和他的学生Alex Krizhevsky设计的.AlexNet在当年赢得了ImageNet图像分类竞赛的冠军,使得CNN成为 ...
详解centos7虚拟机安装elasticsearch5.0.x-安装篇(自己做测试了，es启动有错误可以在这上面找)
本篇文章主要介绍了centos7虚拟机安装elasticsearch5.0.x-安装篇,具有一定的参考价值,感兴趣的小伙伴们可以参考一下. centos7虚拟机安装elasticsearch5.0.x ...
系统部署文档_详解window系统下安装部署SWFTools（文档转换）工具教程
概述 SWFTools是包括pdf2swf, jpeg2swf, png2swf, avi2swf, font2swf, and wav2swf等在内的Adobe Flash工具集合,我们可以用它非常 ...
python email模块详解_Python使用email模块对邮件进行编码和解码的实例教程
解码邮件python自带的email模块是个很有意思的东西,它可以对邮件编码解码,用来处理邮件非常好用. 处理邮件是一个很细致的工作,尤其是解码邮件,因为它的格式变化太多了,下面先看看一个邮件的源文件 ...
【转】canvas save restore详解（包你懂，绝对不是百度里千篇一律的教程）
如果你是要进来看我怎么写的教程,那就错了!! 因为有人写得太好了,只要你懂英语,那就进来传送门吧! 转载于:https://www.cnblogs.com/joyho/articles/58300 ...
twig模板引擎详解（下集：开发者篇）【twig模板引擎中文使用教程】
上集重点介绍了twig模板的使用,供模板设计者阅读,下集供php开发者阅读,讲解如何调用和扩展twig,这一篇安装和采用版本将衔接上集内容. Twig_Environment: 该类的实例是twig模 ...
卷积神经网络之AlexNet网络详解
一介绍 Alex Krizhevsky等人训练了一个大型的卷积神经网络用来把ImageNet LSVRC-2010比赛中120万张高分辨率的图像分为1000个不同的类别.在测试卷上,获得很高准确率 ...

Alexnet详解以及tesnsorflow实现alexnet；什么是alexnet alexnet能做什么；alexnet教程