人群场景分析--Slicing Convolutional Neural Network for Crowd Video Understanding

Slicing Convolutional Neural Network for Crowd Video Understanding CVPR2016
http://www.ee.cuhk.edu.hk/~jshao/SCNN.html
Caffe code: https://github.com/amandajshao/Slicing-CNN

本文也是使用CNN网络对一段视频进行分析，网络输出是关于视频的 94个属性，主要关于 “ Who is in the crowd?”, “Where is the crowd?”, and “Why is crowd here?“
类似 Deeply Learned Attributes for Crowded Scene Understanding，只不过使用的网络结构不一样。
本文最大的特色是 decomposition of 3D feature maps into 2D spatio- and 2D temporal-slices representations，
将视频中的 xyt 信息分解为 xy, xt 和 yt 三类信息。这里的 t 表示时间轴， xy 表示空间坐标

1 Introduction
理解人群的行为模式和运动规律是一个很重要的任务。提取人群中的 appearance and dynamic information 还是比较难得。理论上，我们首先将感兴趣的目标从背景中分割出来，将其分门别类，然后再对每个物体进行跟踪，最后综合所有信息得到一个全局的理解。但是这么做似乎太难了。

深度学习能够解决上面的挑战吗？当前CNN网络可以提取图像中的 strong generic appearance 信息。但是他们缺乏学习 dynamic representation 的关键能力。当前采取的做法一般是将 2D 的 CNN 拓展到 3D CNN，使用 3D 滤波器将 appearance and dynamic feature 混在一起学习表示。但是 appearance and dynamic feature 应该被区分对待，因为他们的表达方式不一样，传递的信息也不一样。也有尝试区分对待 appearance and dynamic 特征，但是这些方法在输入层就丢失了关键的 dynamic information
these methods lose critical dynamic information at the input layer.

我们提出了一个 Slicing CNN (S-CNN) 网络， appearance and dynamics 即被独立的提取，有在semantic 层面上有机的融合到一起
appearance and dynamics have separate representations yet they interact seamlessly at semantic level

3 Slicing CNN Model

我们首先对视频中的每一帧图像使用一个 2D CNN模型来学习 appearance 特征，得到一个 semantic feature cuboids 集合（一组滤波器学习到的特征图），每个feature cuboid 捕获一个独立的信息 visual pattern, or an object instance/category。基于这些提取到的 feature cuboids，我们分别引入三类滤波器（xy-, xt-, and yt-）来学习不同维度的 appearance and dynamic features，接着我们在每个分支后面使用一个 1D 的 temporal pooling layer，接着我们将所有的特征向量 concatenated，再输入到一个 SVM 分类器中，完成 crowd attribute 的分类工作。

3.1. Semantic Selectiveness of Feature Maps
最近的研究显示图像相关的CNN模型中的 2D 滤波器具有很强的选择性 posses strong selectiveness on patterns corresponding to object categories and object identities

通过上面观察到的现象，我们可以从上面的特征集中将运动物体和背景的干扰区分开来，这样可以更好的分析人群的行为
Motivated by the aforementioned observations, we could actually exploit such feature cuboids to separately monitor the movements of different object categories, both spatially and temporally, while reducing the interference caused by the background clutter and irrelevant objects

3.2. Feature Map Pruning
这里主要是将背景的运动信息剔除，提取人群的动态信息
The selectiveness of feature cuboids allows us to design models on a particular set of feature cuboids so as to capture crowd-related dynamic patterns and reject motions from irrelevant background contents.

通过定义 affinity score and conspicuous score 来度量哪个滤波器是我们需要的，哪个滤波器不是我们需要的。

3.3. Semantic Temporal Slices
这里我们将 3D 的信息用 2D temporal slices 来表示
dynamic feature representations can also be described from 2D temporal slices that cut across 3D volume from another two orthogonal planes, as xt- or yt-slices

但是 xt 或 yt 没有将不同物体的运动区分开来，因为它们没有提取将物体分割独立出来
It is not a trivial task to divide their motion patterns apart without identifying these objects at first

这里我们提出从 semantic feature cuboids 提取 Semantic Temporal Slice (STS)
Semantic Temporal Slice (STS) extracted from semantic feature cuboids, which are obtained from the xy convolutional layers,

这么做的好处是：不同物体的信息可以区分开来
such kind of slices can distinguish and purify the dynamic representation for a certain semantic pattern without the interference from other objects, instances or visual patterns inside one temporal slice. Furthermore, given multiple STSs extracted from different horizontal and vertical probe lines and fed into S-CNN, their information can be combined to learn long-range dynamic features.

4 S-CNN Deep Architecture
4.1. Single Branch of S-CNN Model
首先我们用 VGG 网络的前面卷积层提取 spatial semantic feature maps
VGG-16 from conv1_1 to conv4_3_1 to extract spatial semantic feature maps

接着是三个分支：
S-CNN-xy branch：传统意义上的卷积 The S-CNN-xy branch learns spatio- temporal features from the xy-plane by xy-convolutional filters.
convolving feature maps with xy-filters from VGG16 conv5_1 to conv5_3

S-CNN-xt / -yt branch

这里主要是维度的变换 we first swap dimensions of the original xy-plane to the corresponding xt- or yt-plane. 卷积完之后还要变回来

4.2. Combined S-CNN Model
三个分支信息的融合，每个属性使用一个 SVM 分类器，一共 94个分类器
We train a SVM independently for each attribute, thus there are 94 models in total.

5 Experiments

人群场景分析--Slicing Convolutional Neural Network for Crowd Video Understanding相关推荐

人群计数--Switching Convolutional Neural Network for Crowd Counting
Switching Convolutional Neural Network for Crowd Counting CVPR2017 Code for SCNN is based on Lasagne ...
Switching Convolutional Neural Network for Crowd Counting-论文笔记
Switching Convolutional Neural Network for Crowd Counting:用于人群计数的转换卷积神经网络 Switching Convolutional Ne ...
2017_Switching convolutional neural network for crowd counting
Switching convolutional neural network for crowd counting 说明概括一.Switch-CNN简介二.CrowdNet[2]和MCNN[3] ...
快速人群密度估计--Multi-scale Convolutional Neural Networks for Crowd Counting
Multi-scale Convolutional Neural Networks for Crowd Counting https://arxiv.org/abs/1702.02359 对于人群密度 ...
人群计数--Single-Image Crowd Counting via Multi-Column Convolutional Neural Network
Single-Image Crowd Counting via Multi-Column Convolutional Neural Network CVPR2016 https://github.co ...
一步一步分析讲解深度神经网络基础-Convolutional Neural Network
Convolutional Neural Network 参考http://cs231n.github.io/convolutional-networks/ history Convolutional ...
《Improved Crowd Counting Method Based onScale-Adaptive Convolutional Neural Network》论文笔记
<Improved Crowd Counting Method Based onScale-Adaptive Convolutional Neural Network>论文笔记论文地址 ...
论文笔记（三）：PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes 文章概括摘要 1. ...
笔记：PoseCNN:A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (Robotics: ...

人群场景分析--Slicing Convolutional Neural Network for Crowd Video Understanding

人群场景分析--Slicing Convolutional Neural Network for Crowd Video Understanding相关推荐

最新文章

热门文章