视频动作识别--Convolutional Two-Stream Network Fusion for Video Action Recognition

Convolutional Two-Stream Network Fusion for Video Action Recognition CVPR2016

http://www.robots.ox.ac.uk/~vgg/software/two_stream_action/
https://github.com/feichtenhofer/twostreamfusion

对视频动作识别采用 two steam CNN 分开处理时空信息，这里我们主要探讨这怎么在 CNN中更好的融合时空信息。
我们的发现有以下三点：
(i) that rather than fusing at the softmax layer, a spatial and temporal network can be fused at a convolution layer without loss of performance, but with a substantial saving in parameters;
在卷积层融合时空网络不会导致性能下降，但是可以减少网络参数

(ii) that it is better to fuse such networks spatially at the last convolutional layer than earlier, and that additionally fusing at the class prediction layer can boost accuracy;
在网络的后卷积层空间融合比浅层要好，在类别预测层融合会增加性能

(iii) that pooling of abstract convolutional features over spatiotemporal neighbourhoods further boosts performance.
在时空邻域加入池化可以增加性能

针对CNN网络为什么没有在视频动作识别中取得很好的结果，我们认为的原因是：1）训练数据可能太少了，2）时间信息利用的不够
current ConvNet architectures are not able to take full advantage of temporal information and their performance is consequently often dominated by spatial (appearance) recognition

至少以前的 two-stream architecture 不能很好的解决下面的问题：
1）recognizing what is moving where, i.e. registering appearance recognition (spatial cue) with optical flow recognition (temporal cue) 时空信息的对应
2）how these cues evolve over time. 信息是如何变化

3 Approach
以前的 two-stream architecture 不能很好的融合时空信息，没有时空对应关系
3.1. Spatial fusion 空间融合
这里介绍了好几种融合：Sum fusion，Max fusion，Concatenation fusion，Conv fusion，Bilinear fusion

3.2. Where to fuse the networks
这里的选择也是比较多的

3.3. Temporal fusion

3.4. Proposed architecture

We fuse the two networks, at the last convolutional layer (after ReLU) into the spatial stream to convert it into a spatiotemporal stream by using 3D Conv fusion followed by 3D pooling (see Fig. 4, left). Moreover, we do not truncate the temporal stream and also perform 3D Pooling in the temporal network (see Fig. 4, right). The losses of both streams are used for training and during testing we average the predictions of the two streams

有没有感觉搞复杂了啊！

视频动作识别--Convolutional Two-Stream Network Fusion for Video Action Recognition相关推荐

译：Convolutional Two-Stream Network Fusion for Video Action Recognition
这是双流神经网络的改进版,接下来会出他的读书笔记. 摘要:近年来,卷积神经网络(卷积神经网络)对视频中人类行为识别的应用提出了不同的解决方案,用于整合外观和运动信息.为了更好地利用 ...
[翻译]Convolutional Two-Stream Network Fusion for Video Action Recognition
摘要近年来,卷积神经网络(卷积神经网络)对视频中人类行为识别的应用提出了不同的解决方案,用于整合外观和运动信息.为了更好地利用时空信息,我们研究了许多在空间和时间上融合卷积塔的方法.我们得出以下结论 ...
Convolutional Two-Stream Network Fusion for Video Action Recognition翻译
本文转于:https://blog.csdn.net/weixin_42164269/article/details/80689653 若有侵权请联系删除摘要近年来,卷积神经网络(卷积神经网络)对 ...
NeurIPS 2022 | 基于对齐引导时间注意力机制的视频动作识别
©作者 | 张海涛单位 | 重庆邮电大学研究方向 | 视频理解论文标题: Alignment-guided Temporal Attention for Video Action Recogni ...
视频动作识别--Two-Stream Convolutional Networks for Action Recognition in Videos
Two-Stream Convolutional Networks for Action Recognition in Videos NIPS2014 http://www.robots.ox.ac. ...
视频时序动作识别（video action recognition）介绍
一.视频时序动作识别算法分类根据网络的工作方式,可以将视频时序动作识别算法大致分为四大类: 采用2D卷积的方法采用3D卷积的方法双流法引入VLAD的方法 1.1 采用2D卷积的方法 <T ...
计算机视觉和动作检测,计算机视觉技术深度解读之视频动作识别
来源:雪球App,作者: 我是张康,(https://xueqiu.com/6964725465/135165468) 作者:极链AI研究院徐宝函视频的理解与识别是计算机视觉的基础任务之一.随着视 ...
视频动作识别--Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition ECCV2016 https://githu ...
Tensorflow C3D完成视频动作识别
本文是视频动作识别领域经典的C3D网络的简易实现,可以作为动作识别的入门.论文为<Learning Spatiotemporal Features with 3D Convolutional N ...

视频动作识别--Convolutional Two-Stream Network Fusion for Video Action Recognition

视频动作识别--Convolutional Two-Stream Network Fusion for Video Action Recognition相关推荐

最新文章

热门文章