足球精彩时刻剪辑

转载自AI Studio

项目链接https://aistudio.baidu.com/aistudio/projectdetail/3473391?contributionType=1

1.项目概述

近期的全民热点话题离不开正在举办的冬奥会，每天都有赛场上的喜讯传来，谷爱凌夺中国第三金、武大靖和任子威的亮眼表现、苏翊鸣的“飞檐走壁”，可喜可贺！

本届冬奥会共有7个大项、15个分项和109个小项，每场比赛结束都有很多记录运动员比赛精彩瞬间的回放视频产生，这些短视频通常播放量很高，深受大众喜爱。除了服务于大众，让体育爱好者在欣赏时大饱眼福外，精彩视频剪辑技术也可以用于专业的体育训练，运动员可以通过比赛或日常训练回放来熟悉自己和对手，进行战术演练，达到知己知彼百战百胜的效果。

本项目运用动作检测技术，实现基于足球视频的高精度精彩视频剪辑方案，最终方案动作识别准确率达91%，动作检测F1-score达到76.2%。快一起来看看吧～

图1 足球精彩视频剪辑示例

欢迎报名直播课加入交流群，如需更多技术交流与合作可点击以下链接
https://paddleqiyeban.wjx.cn/vj/Qlb0uS3.aspx?udsid=531417

2.技术难点

目前精彩视频剪辑虽然需求量大应用广泛，但人工剪辑需要浏览整个比赛视频，工作量大、成本高，又要求剪辑人员同时具有一定的体育专业知识，专业需求高。如果用AI技术来实现会存在以下两个难点问题：

  1. 动作检测任务复杂度高：视频精彩片段剪辑任务的实现要点在于准确找到该类动作发生的起止点。但体育类视频内经常包含大量冗余的背景信息，动作类别多样且持续时长相对较短，要精准的判断出动作的起始点和对应的类别，任务难度高。
  2. 视频中的信息具有多样性，如何有效利用这些特征信息也是值得我们去考虑的。

3.解决方案

动作检测任务可以理解为动作定位+识别任务，需要在一个长视频中，先定位到一个动作发生的起止点，再识别出这个动作是什么。本案例使用PP-TSM+BMN+AttentionLSTM系列模型来实现动作检测任务，具体流程如下图：

图2 动作检测任务流程

整个方案可以分为三个步骤：

  1. 使用飞桨特色的高精度视频理解模型PP-TSM提取视频图像特征；使用VGGish网络提取音频特征；
  2. 将获得的音视频特征输入BMN网络，得到由动作开始时间和结束时间组合成的时序片段（proposal）；
  3. 得到时序片段提名后，根据动作开始和结束时间截断视频和音频特征，通过AttentionLSTM输出动作类别。

4.数据准备

4.1 数据集介绍

数据集由EuroCup2012, EuroCup2016, WorldCup2014, WorldCup2018四个赛事的比赛视频组成，共计272个训练集、25个测试集，支持15种足球精彩动作定位与识别，动作类别分别为：射门、进球、进球有欢呼、角球、任意球、黄牌、红牌、点球、换人、界外球、球门球、开球、越位挥旗、回放空中对抗和回放进球。本案例中我们提供一条视频数据供大家进行测试，可以通过如下命令，下载视频。

# 仅在第一次运行notebook时，执行以下命令
!wget https://bj.bcebos.com/v1/tmt-pub/datasets/EuroCup2016/63e51df254d2402fac703b6c4fdb4ea9.mp4
!mv 63e51df254d2402fac703b6c4fdb4ea9.mp4 football.mp4

--2022-02-18 14:00:03--  https://bj.bcebos.com/v1/tmt-pub/datasets/EuroCup2016/63e51df254d2402fac703b6c4fdb4ea9.mp4
正在解析主机 bj.bcebos.com (bj.bcebos.com)... 182.61.200.195, 182.61.200.229, 2409:8c04:1001:1002:0:ff:b001:368a
正在连接 bj.bcebos.com (bj.bcebos.com)|182.61.200.195|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度： 1562942061 (1.5G) [video/mp4]
正在保存至: “63e51df254d2402fac703b6c4fdb4ea9.mp4”63e51df254d2402fac7 100%[===================>]   1.46G  89.6MB/s    in 20s     2022-02-18 14:00:23 (75.0 MB/s) - 已保存 “63e51df254d2402fac703b6c4fdb4ea9.mp4” [1562942061/1562942061])

除视频文件外，数据集中还包含视频对应的标注文件，这里我们在PaddleVideo/applications/FootballAction/datasets/EuroCup2016 文件路径下提供了标注文件 label.json。标注格式为：

{"fps": 5,"gts": [{"url": "football.mp4","total_frames": 6189,"actions": [{"label_ids": [2],"label_names": ["角球"],"start_id": 393,"end_id": 399},...

文件中的 fps：5 代表以每秒5帧的频率提取视频帧。
actions 包含每段视频中的动作信息：动作类别标签（label_ids）、类别名称（label_names）、开始帧数（start_id）和结束帧数（end_id）。

该视频样本包含8种动作信息，动作类别标签与类别名称的对应关系如下：

    "0": "背景""1": "进球""2": "角球""3": "任意球""4": "黄牌""5": "红牌""6": "换人""7": "界外球"

4.2 视频采样

输入视频为mp4文件，我们提供的视频样本 football.mp4 时长1h43min。训练时如果使用全部视频文件，会消耗大量计算资源，一般预先做一些采样处理。

图像采样：以fps=5的频率抽取图像帧
音频采样：pcm音频文件，采样频率ar=16000

运行以下代码进行图像和音频采样。

# 移动视频football.mp4、label.json到指定位置
%cd /home/aistudio/PaddleVideo/applications/FootballAction/datasets/EuroCup2016/
!mkdir mp4
!mv /home/aistudio/football.mp4 /home/aistudio/PaddleVideo/applications/FootballAction/datasets/EuroCup2016/mp4/football.mp4

/home/aistudio/PaddleVideo/applications/FootballAction/datasets/EuroCup2016

# 开始运行代码前先安装好依赖
%cd /home/aistudio/PaddleVideo/
!python -m pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/

/home/aistudio/PaddleVideo
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/
Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (1.19.5)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (1.1.5)
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (4.27.0)
Requirement already satisfied: PyYAML>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (5.1.2)
Collecting opencv-python==4.2.0.32Downloading https://pypi.tuna.tsinghua.edu.cn/packages/34/a3/403dbaef909fee9f9f6a8eaff51d44085a14e5bb1a1ff7257117d744986a/opencv_python-4.2.0.32-cp37-cp37m-manylinux1_x86_64.whl (28.2 MB)|████████████████████████████████| 28.2 MB 9.3 MB/s
[?25hCollecting decord==0.4.2Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c0/0c/7d99cfcde7b85f80c9ea9b0b19441339ad3cef59ee7fa5386598db714efe/decord-0.4.2-py2.py3-none-manylinux1_x86_64.whl (11.8 MB)|████████████████████████████████| 11.8 MB 444 kB/s
[?25hCollecting av==8.0.3Downloading https://pypi.tuna.tsinghua.edu.cn/packages/66/ff/bacde7314c646a2bd2f240034809a10cc3f8b096751284d0828640fff3dd/av-8.0.3-cp37-cp37m-manylinux2010_x86_64.whl (37.2 MB)|████████████████████████████████| 37.2 MB 10.9 MB/s
[?25hRequirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (1.6.3)
Collecting ffmpeg-python==0.2.0Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d7/0c/56be52741f75bad4dc6555991fabd2e07b432d333da82c11ad701123888a/ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
Collecting scikit-imageDownloading https://pypi.tuna.tsinghua.edu.cn/packages/d2/d9/d16d4cbb4840e0fb3bd329b49184d240b82b649e1bd579489394fbc85c81/scikit_image-0.19.2-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.5 MB)|████████████████████████████████| 13.5 MB 5.9 MB/s
[?25hRequirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 11)) (2.2.3)
Requirement already satisfied: paddlenlp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 12)) (2.1.1)
Collecting lmdbDownloading https://pypi.tuna.tsinghua.edu.cn/packages/4d/cf/3230b1c9b0bec406abb85a9332ba5805bdd03a1d24025c6bbcfb8ed71539/lmdb-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (298 kB)|████████████████████████████████| 298 kB 10.9 MB/s
[?25hRequirement already satisfied: moviepy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 14)) (1.0.1)
Collecting SimpleITKDownloading https://pypi.tuna.tsinghua.edu.cn/packages/ad/ce/75d95eab0bc815f478b594add92090ff664e5cbc75ab9d8e2ffbaae1dc91/SimpleITK-2.1.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (48.4 MB)|████████████████████████████████| 48.4 MB 5.7 MB/s
[?25hCollecting paddledetDownloading https://pypi.tuna.tsinghua.edu.cn/packages/bb/27/eb6714bdc12e73544617b1b4b3481ae70bec644a722bcbedd5743531db97/paddledet-2.3.0-py3-none-any.whl (589 kB)|████████████████████████████████| 589 kB 6.2 MB/s
[?25hRequirement already satisfied: et_xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 17)) (1.0.1)
Collecting lapDownloading https://pypi.tuna.tsinghua.edu.cn/packages/bf/64/d9fb6a75b15e783952b2fec6970f033462e67db32dc43dfbb404c14e91c2/lap-0.4.0.tar.gz (1.5 MB)|████████████████████████████████| 1.5 MB 7.8 MB/s
[?25h  Preparing metadata (setup.py) ... [?25ldone
[?25hCollecting motmetricsDownloading https://pypi.tuna.tsinghua.edu.cn/packages/9c/28/9c3bc8e2a87f4c9e7b04ab72856ec7f9895a66681a65973ffaf9562ef879/motmetrics-1.2.0-py3-none-any.whl (151 kB)|████████████████████████████████| 151 kB 4.4 MB/s
[?25hCollecting xmltodictDownloading https://pypi.tuna.tsinghua.edu.cn/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB)
Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 21)) (3.0.5)
Requirement already satisfied: future in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from ffmpeg-python==0.2.0->-r requirements.txt (line 9)) (0.18.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas->-r requirements.txt (line 2)) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas->-r requirements.txt (line 2)) (2019.3)
Collecting PyWavelets>=1.1.1Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a1/9c/564511b6e1c4e1d835ed2d146670436036960d09339a8fa2921fe42dad08/PyWavelets-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (6.1 MB)|████████████████████████████████| 6.1 MB 3.6 MB/s
[?25hCollecting tifffile>=2019.7.26Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d8/38/85ae5ed77598ca90558c17a2f79ddaba33173b31cf8d8f545d34d9134f0d/tifffile-2021.11.2-py3-none-any.whl (178 kB)|████████████████████████████████| 178 kB 6.2 MB/s
[?25hRequirement already satisfied: pillow!=7.1.0,!=7.1.1,!=8.3.0,>=6.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 10)) (8.2.0)
Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 10)) (21.3)
Requirement already satisfied: imageio>=2.4.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 10)) (2.6.1)
Requirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 10)) (2.4)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 11)) (1.1.0)
Requirement already satisfied: six>=1.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 11)) (1.16.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 11)) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->-r requirements.txt (line 11)) (3.0.7)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (0.4.4)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (4.1.0)
Requirement already satisfied: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (2.9.0)
Requirement already satisfied: multiprocess in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (0.70.11.1)
Requirement already satisfied: paddlefsl==1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (1.0.0)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (0.42.1)
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp->-r requirements.txt (line 12)) (1.2.2)
Requirement already satisfied: requests~=2.24.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlefsl==1.0.0->paddlenlp->-r requirements.txt (line 12)) (2.24.0)
Requirement already satisfied: decorator<5.0,>=4.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->-r requirements.txt (line 14)) (4.4.2)
Requirement already satisfied: imageio-ffmpeg>=0.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->-r requirements.txt (line 14)) (0.3.0)
Requirement already satisfied: proglog<=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->-r requirements.txt (line 14)) (0.1.9)
Requirement already satisfied: Cython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddledet->-r requirements.txt (line 16)) (0.29)
Collecting cython-bboxDownloading https://pypi.tuna.tsinghua.edu.cn/packages/fa/b9/fc7d60e8c3b29cc0ff24a3bb3c4b7457e10b7610fbb2893741b623487b34/cython_bbox-0.1.3.tar.gz (41 kB)|████████████████████████████████| 41 kB 613 kB/s
[?25h  Preparing metadata (setup.py) ... [?25ldone
[?25hRequirement already satisfied: sklearn in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddledet->-r requirements.txt (line 16)) (0.0)
Requirement already satisfied: visualdl>=2.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddledet->-r requirements.txt (line 16)) (2.2.0)
Requirement already satisfied: setuptools>=42.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddledet->-r requirements.txt (line 16)) (56.2.0)
Collecting typeguardDownloading https://pypi.tuna.tsinghua.edu.cn/packages/9a/bb/d43e5c75054e53efce310e79d63df0ac3f25e34c926be5dffb7d283fb2a8/typeguard-2.13.3-py3-none-any.whl (17 kB)
Collecting terminaltablesDownloading https://pypi.tuna.tsinghua.edu.cn/packages/c4/fb/ea621e0a19733e01fe4005d46087d383693c0f4a8f824b47d8d4122c87e0/terminaltables-3.1.10-py2.py3-none-any.whl (15 kB)
Collecting shapelyDownloading https://pypi.tuna.tsinghua.edu.cn/packages/9d/4d/4b0d86ed737acb29c5e627a91449470a9fb914f32640db3f1cb7ba5bc19e/Shapely-1.8.1.post1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.0 MB)|████████████████████████████████| 2.0 MB 5.1 MB/s
[?25hCollecting pycocotoolsDownloading https://pypi.tuna.tsinghua.edu.cn/packages/75/5c/ac61ea715d7a89ecc31c090753bde28810238225ca8b71778dfe3e6a68bc/pycocotools-2.0.4.tar.gz (106 kB)|████████████████████████████████| 106 kB 434 kB/s
[?25h  Installing build dependencies ... [?25ldone
[?25h  Getting requirements to build wheel ... [?25ldone
[?25h  Preparing metadata (pyproject.toml) ... [?25ldone
[?25hCollecting pytest-benchmarkDownloading https://pypi.tuna.tsinghua.edu.cn/packages/2c/60/423a63fb190a0483d049786a121bd3dfd7d93bb5ff1bb5b5cd13e5df99a7/pytest_benchmark-3.4.1-py2.py3-none-any.whl (50 kB)|████████████████████████████████| 50 kB 217 kB/s
[?25hRequirement already satisfied: flake8 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from motmetrics->-r requirements.txt (line 19)) (4.0.1)
Collecting pytestDownloading https://pypi.tuna.tsinghua.edu.cn/packages/38/93/c7c0bd1e932b287fb948eb9ce5a3d6307c9fc619db1e199f8c8bc5dad95f/pytest-7.0.1-py3-none-any.whl (296 kB)|████████████████████████████████| 296 kB 4.0 MB/s
[?25hCollecting flake8-import-orderDownloading https://pypi.tuna.tsinghua.edu.cn/packages/ab/52/cf2d6e2c505644ca06de2f6f3546f1e4f2b7be34246c9e0757c6048868f9/flake8_import_order-0.18.1-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 21)) (1.4.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp->-r requirements.txt (line 12)) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp->-r requirements.txt (line 12)) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp->-r requirements.txt (line 12)) (1.25.6)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp->-r requirements.txt (line 12)) (2019.9.11)
Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.0.0)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (0.8.53)
Requirement already satisfied: flask>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.1.1)
Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.21.0)
Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (3.14.0)
Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (0.7.1.1)
Requirement already satisfied: importlib-metadata<4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->motmetrics->-r requirements.txt (line 19)) (4.2.0)
Requirement already satisfied: pycodestyle<2.9.0,>=2.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->motmetrics->-r requirements.txt (line 19)) (2.8.0)
Requirement already satisfied: pyflakes<2.5.0,>=2.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->motmetrics->-r requirements.txt (line 19)) (2.4.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->motmetrics->-r requirements.txt (line 19)) (0.6.1)
Requirement already satisfied: dill>=0.3.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from multiprocess->paddlenlp->-r requirements.txt (line 12)) (0.3.3)
Collecting iniconfigDownloading https://pypi.tuna.tsinghua.edu.cn/packages/9b/dd/b3c12c6d707058fa947864b67f0c4e0c39ef8610988d7baea9578f3c48f3/iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: attrs>=19.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pytest->motmetrics->-r requirements.txt (line 19)) (21.4.0)
Requirement already satisfied: pluggy<2.0,>=0.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pytest->motmetrics->-r requirements.txt (line 19)) (0.13.1)
Collecting py>=1.8.2Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f6/f0/10642828a8dfb741e5f3fbaac830550a518a775c7fff6f04a007259b0548/py-1.11.0-py2.py3-none-any.whl (98 kB)|████████████████████████████████| 98 kB 402 kB/s
[?25hCollecting tomli>=1.0.0Downloading https://pypi.tuna.tsinghua.edu.cn/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting py-cpuinfoDownloading https://pypi.tuna.tsinghua.edu.cn/packages/e6/ba/77120e44cbe9719152415b97d5bfb29f4053ee987d6cb63f55ce7d50fadc/py-cpuinfo-8.0.0.tar.gz (99 kB)|████████████████████████████████| 99 kB 2.3 MB/s
[?25h  Preparing metadata (setup.py) ... [?25ldone
[?25hRequirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp->-r requirements.txt (line 12)) (0.24.2)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.1.0)
Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (0.16.0)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (2.11.0)
Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (7.0)
Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (2.8.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata<4.3->flake8->motmetrics->-r requirements.txt (line 19)) (3.7.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata<4.3->flake8->motmetrics->-r requirements.txt (line 19)) (4.0.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp->-r requirements.txt (line 12)) (2.1.0)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp->-r requirements.txt (line 12)) (0.14.1)
Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (3.9.9)
Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (16.7.9)
Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.3.4)
Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.4.10)
Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (0.10.0)
Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (1.3.0)
Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (2.0.1)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.1->visualdl>=2.1.0->paddledet->-r requirements.txt (line 16)) (2.0.1)
Building wheels for collected packages: lap, cython-bbox, pycocotools, py-cpuinfoBuilding wheel for lap (setup.py) ... [?25ldone
[?25h  Created wheel for lap: filename=lap-0.4.0-cp37-cp37m-linux_x86_64.whl size=1593870 sha256=383f558707009f930cf7d69e2b4a3ae1ad2c8b6f1d80d7c7199b6d11efc96a69Stored in directory: /home/aistudio/.cache/pip/wheels/5c/d0/d2/e331d17a999666b1e2eb99743cfa1742629f9d26c55c657001Building wheel for cython-bbox (setup.py) ... [?25ldone
[?25h  Created wheel for cython-bbox: filename=cython_bbox-0.1.3-cp37-cp37m-linux_x86_64.whl size=61619 sha256=fee7bccbe8214d10185a65aac357bff45ad92e0eca525b7688617b6eb2dd4e3fStored in directory: /home/aistudio/.cache/pip/wheels/3e/b3/6a/aae8832326545645e65d643a2aaf223ffa3a7d01e1a1bae01bBuilding wheel for pycocotools (pyproject.toml) ... [?25ldone
[?25h  Created wheel for pycocotools: filename=pycocotools-2.0.4-cp37-cp37m-linux_x86_64.whl size=273789 sha256=c4019e01fa4dea259cf6072dd3b17f524a64d0955bf1cb09f94bc11504113fa7Stored in directory: /home/aistudio/.cache/pip/wheels/c0/01/5f/670dfd20204fc9cc6bf843db4e014acb998f411922e3abc49fBuilding wheel for py-cpuinfo (setup.py) ... [?25ldone
[?25h  Created wheel for py-cpuinfo: filename=py_cpuinfo-8.0.0-py3-none-any.whl size=22245 sha256=7a5755a906124d042677008953be77620053fd5ee6e03c0b043118fe42273e62Stored in directory: /home/aistudio/.cache/pip/wheels/88/c7/d0/6309c7cc9929894c11fe8e516c3e2a0d0a53ee4e198eac48b7
Successfully built lap cython-bbox pycocotools py-cpuinfo
Installing collected packages: tomli, py, iniconfig, pytest, py-cpuinfo, xmltodict, pytest-benchmark, flake8-import-order, typeguard, tifffile, terminaltables, shapely, PyWavelets, pycocotools, opencv-python, motmetrics, lap, cython-bbox, SimpleITK, scikit-image, paddledet, lmdb, ffmpeg-python, decord, avAttempting uninstall: opencv-pythonFound existing installation: opencv-python 4.1.1.26Uninstalling opencv-python-4.1.1.26:Successfully uninstalled opencv-python-4.1.1.26
Successfully installed PyWavelets-1.2.0 SimpleITK-2.1.1 av-8.0.3 cython-bbox-0.1.3 decord-0.4.2 ffmpeg-python-0.2.0 flake8-import-order-0.18.1 iniconfig-1.1.1 lap-0.4.0 lmdb-1.3.0 motmetrics-1.2.0 opencv-python-4.2.0.32 paddledet-2.3.0 py-1.11.0 py-cpuinfo-8.0.0 pycocotools-2.0.4 pytest-7.0.1 pytest-benchmark-3.4.1 scikit-image-0.19.2 shapely-1.8.1.post1 terminaltables-3.1.10 tifffile-2021.11.2 tomli-2.0.1 typeguard-2.13.3 xmltodict-0.12.0
[33mWARNING: You are using pip version 21.3.1; however, version 22.0.3 is available.
You should consider upgrading via the '/opt/conda/envs/python35-paddle120-env/bin/python -m pip install --upgrade pip' command.[0m

抽帧时间较长，大约需要执行10分钟。

%cd /home/aistudio/PaddleVideo/applications/FootballAction
!python3.7 datasets/script/get_frames_pcm.py

/home/aistudio/PaddleVideo/applications/FootballAction
mp4/football.mp4
done

数据预处理后得到的文件夹格式如下：

|--  datasets                   # 训练数据集和处理脚本|--  EuroCup2016   # 数据集|--  mp4               # 原始视频.mp4|--  frames           # 图像帧|--  pcm               # 音频pcm|--  url.list            # 视频列表|--  label.json       # 视频原始gts

5. 模型训练

5.1 PP-TSM训练

5.1.1 PP-TSM 训练数据处理

PP-TSM模型是一个视频理解模型，它可以用于包含单动作的视频段的视频分类任务。在本方案中，我们使用训练好的PP-TSM模型提取视频图像特征。

get_instance_for_pptsm.py 文件用于生成训练所需的正负样本。正样本为标注后的运动区间，该区间内的所有图像帧会生成一个pkl文件；负样本为标注后的非运动区间，因为足球赛事中无特殊动作的时间较长，负样本以随机取N个区间生成N个pkl的方式生成。

!python datasets/script/get_instance_for_pptsm.py

完成该步骤后，数据存储位置如下：

|--  datasets                   # 训练数据集和处理脚本|--  EuroCup2016            # 数据集|--  input_for_pptsm   # pptsm训练的数据

5.1.2 PP-TSM模型训练

在训练开始前，需要先下载图像蒸馏预训练模型ResNet50_vd_ssld_v2.pdparams 作为模型 backbone 初始化参数，通过如下命令下载：

# 在第一次运行代码时执行如下命令
!wget https://videotag.bj.bcebos.com/PaddleVideo/PretrainModel/ResNet50_vd_ssld_v2_pretrained.pdparams
!mkdir pretrain
!mv ResNet50_vd_ssld_v2_pretrained.pdparams pretrain/ResNet50_vd_ssld_v2_pretrained.pdparams

启动训练，可以在 pptsm_football_v2.0.yaml 中调节训练参数。

# 启动训练
%cd /home/aistudio/PaddleVideo/
!python -B -m paddle.distributed.launch \--gpus="0" \--log_dir=applications/FootballAction/train_pptsm/logs \main.py \--validate \-c applications/FootballAction/train_proposal/configs/pptsm_football_v2.0.yaml  \-o output_dir=applications/FootballAction/train_pptsm/

5.1.3 PP-TSM模型转为预测模式

使用如下命令生成预测所需的模型结构文件ppTSM.pdmodel和模型权重文件ppTSM.pdiparams。

!python tools/export_model.py -c applications/FootballAction/train_proposal/configs/pptsm_football_v2.0.yaml \-p applications/FootballAction/train_pptsm/ppTSM_best.pdparams \-o applications/FootballAction/checkpoints/ppTSM

5.1.4 基于PP-TSM的视频特征提取

基于刚训练好的PP-TSM模型提取视频特征。每次输入8帧图像，输出8维特征，最后将所有特征拼接在一起代表视频特征。

extract_feat.py 文件同时进行视频图像特征和音频特征提取，音频特征通过VGGish模型提取。该模型权重文件可通过如下命令下载：

# 仅在第一次运行notebook时下载
!wget https://videotag.bj.bcebos.com/PaddleVideo-release2.1/FootballAction/audio.tar
!tar -xvf audio.tar
!rm audio.tar
!mv AUDIO/ /home/aistudio/PaddleVideo/applications/FootballAction/checkpoints/

提取视频图像和音频特征：

%cd /home/aistudio/PaddleVideo/applications/FootballAction/extractor
!python extract_feat.py

特征提取完成后存储在EuroCup2016文件夹下

   |--  datasets                   # 训练数据集和处理脚本|--  EuroCup2016     # 数据集|--  features           # 视频的图像+音频特征

5.2 BMN训练

将得到的音视频特征整合输入BMN网络，得到由动作开始时间和结束时间组合成的时序片段（proposal）。

5.2.1 BMN训练数据处理

get_instance_for_bmn 文件用于提取二分类的proposal，根据标注文件和音视频特征得到BMN所需要的数据集。

%cd /home/aistudio/PaddleVideo/applications/FootballAction
!python datasets/script/get_instance_for_bmn.py

得到的数据格式如下：

duration_second 代表视频片段时长，duration_frame 代表涵盖多少帧，annotations为这段视频的真实标注，segment为动作起止时间，label及label_name 为动作类别和名称。

{"719b0a4bcb1f461eabb152298406b861_753_793": {"duration_second": 40.0,"duration_frame": 200,"feature_frame": 200,"subset": "train","annotations": [{"segment": [15.0,22.0],"label": "3.0","label_name": "任意球"}]},...
}

完成该步骤后，数据存储位置如下：

   |--  datasets                   # 训练数据集和处理脚本|--  EuroCup2016            # xx数据集|--  input_for_bmn     # bmn训练的proposal

5.2.2 BMN模型训练

该步骤训练与PP-TSM模型训练类似，可以调整 bmn_football_v2.0.yaml 文件，修改训练参数。

%cd /home/aistudio/PaddleVideo/
!python -B -m paddle.distributed.launch \--gpus='0' \--log_dir=applications/FootballAction/train_bmn/logs \main.py \--validate \-c applications/FootballAction/train_proposal/configs/bmn_football_v2.0.yaml \-o output_dir=applications/FootballAction/train_bmn/

5.2.3 BMN模型转为预测模式

生成预测所需的模型结构文件BMN.pdmodel和模型权重文件BMN.pdiparams。

!python tools/export_model.py \-c applications/FootballAction/train_proposal/configs/bmn_football_v2.0.yaml \-p applications/FootballAction/train_bmn/BMN_epoch_00001.pdparams \-o applications/FootballAction/checkpoints/BMN

5.2.4 BMN模型预测

进行模型预测，得到动作proposal的信息，包括动作开始时间、结束时间以及置信度。

%cd /home/aistudio/PaddleVideo/applications/FootballAction/extractor
!python extract_bmn.py

完成该步骤后，数据存储位置

   |--  datasets                   # 训练数据集和处理脚本|--  EuroCup2016            # xx数据集|--  feature_bmn|--  prop.json    # bmn 预测结果

5.3 LSTM训练

得到时序片段提名（proposal）后，根据动作开始和结束时间阶段视频和音频特征，通过AttentionLSTM输出动作类别。

5.3.1 LSTM训练数据处理

按照BMN预测到的proposal截断视频特征，生成训练AttentionLSTM所需数据集。

%cd /home/aistudio/PaddleVideo/applications/FootballAction/datasets/script/
!python get_instance_for_lstm.py

5.3.2 LSTM训练

通过如下命令训练AttentionLSTM网络。

cd /home/aistudio/PaddleVideo/applications/FootballAction/train_lstm/
python -u scenario_lib/train.py  \--model_name=ActionNet \--config=conf/conf.txt \--save_dir=../football_lstm

这里因为训练数据只提供了一条视频，直接进行LSTM训练会存在问题，如有需要可根据自己的数据集使用上述命令进行训练，并通过修改 conf/conf.txt 文件，更改模型参数。

5.3.3 LSTM模型转为预测模式

如需将训练模型转为预测模式，可通过以下命令生成模型结构文件。

cd /home/aistudio/PaddleVideo/applications/FootballAction/train_lstm/python inference_model.py --config=conf/conf.yaml --weights=../football_lstm/ActionNet.pdparams --save_dir=../checkpoints/LSTM

我们提供训练好的LSTM模型，共大家进行后续测试。

!wget https://videotag.bj.bcebos.com/PaddleVideo-release2.1/FootballAction/lstm.tar
!tar -xvf lstm.tar
!rm lstm.tar
!mv LSTM/ /home/aistudio/PaddleVideo/applications/FootballAction/checkpoints/

6. 模型推理

输入一条视频数据，以该网络结构进行推理。

%cd /home/aistudio/PaddleVideo/applications/FootballAction/predict
!python predict.py

7. 模型评估

通过如下命令开始模型评估。

# 包括bmn proposal 评估和最终action评估
%cd /home/aistudio/PaddleVideo/applications/FootballAction/predict
!python eval.py results.json

8. 模型优化

本案例中展示的PP-TSM+BMN+AttentionLSTM的实现方案是经过多次优化实验得来的。在实验的初始阶段，我们最先选取的是TSN+BMN+AttentionLSTM的实现方案。后续经过三个方面的大量优化处理，才在模型效果提升方面有了显著成果。可以通过阅读本节获得在模型效果提升方面的宝贵经验。

实验分为以下三个方面：

  • 更换图像特征提取模型，分别采用视频理解模型TSN、TSM、PP-TSM 进行实验，在动作识别任务上精度更高的模型，在动作检测任务上也有更好的表现。
  • 在BMN生成的时序提名基础上扩展proposal特征，更好的涵盖整个动作特征
  • 增加训练数据量，使网络具有更好的泛化能力

实验最初采用TSN+BMN+AttentionLSTM的解决方案。实验Baseline如下：

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSN 75.86 BMN LSTM 60.04 61.31 60.67

通过将TSN更换为在视频理解任务上精度更好的TSM模型得到如下效果提升：

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSN 75.86 BMN LSTM 60.04 61.31 60.67
TSM 90.24 BMN LSTM 71.06 65.93 68.4

可以看到，TSN和TSM模型在视频分类任务上的精度分别是75.86%和90.24%，精度大幅提升，使得提取的视频图像特征更为准确，最终在行为检测任务上F1-score由60.67%提升到68.4%。

可以通过如下代码训练TSN、TSM网络，进行数据对比。

# 训练TSN
%cd /home/aistudio/PaddleVideo/
!python -B -m paddle.distributed.launch \--gpus="0" \--log_dir=applications/FootballAction/train_tsn/logs \main.py \--validate \-c applications/FootballAction/train_proposal/configs/tsn_football.yaml  \-o output_dir=applications/FootballAction/train_tsn/

# 训练TSM
%cd /home/aistudio/PaddleVideo/
!python -B -m paddle.distributed.launch \--gpus="0" \--log_dir=applications/FootballAction/train_tsm/logs \main.py \--validate \-c applications/FootballAction/train_proposal/configs/tsm_football.yaml  \-o output_dir=applications/FootballAction/train_tsm/

除此之外，我们还查验了BMN生成的proposals，发现某些时间片段并不能完整涵盖动作。为更好的涵盖整个动作特征，我们将生成的proposal前后各扩展1s。

实验对比数据如下：

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSM 90.24 BMN LSTM 71.06 65.93 68.4
TSM 90.24 BMN LSTM + 扩展全部动作的proposal 80.3 66.31 72.64

对所有动作扩展proposal后我们发现，并不是所有动作扩展proposal特征都有明显的正向提升，因此我们选择只对部分特征扩展proposal，得到如下实验结果：

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSM 90.24 BMN LSTM + 扩展全部动作的proposal 80.3 66.31 72.64
TSM 90.24 BMN LSTM + 扩展部分动作的proposal 79.37 67.76 73.1

可通过以下代码扩展proposal特征：

%cd /home/aistudio/PaddleVideo/applications/FootballAction/datasets/script/
!python get_instance_for_lstm_long_proposal.py

以上所有实验均基于由156个训练集+15个测试集组成的数据集，经过一轮新的数据标注，我们将数据集扩充为272训练集+25测试集，并基于此进行了新的训练，得到如下结果：

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSM 90.24 BMN LSTM + 扩展部分动作的proposal 79.37 67.76 73.1
TSM + 数据扩展 90.85 BMN LSTM + 扩展部分动作的proposal 80.22 68.72 74.03

最后，我们将TSM模型更换为精度更高的PP-TSM模型，得到最终的实验结果。

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSM + 数据扩展 90.85 BMN LSTM + 扩展部分动作的proposal 80.22 68.72 74.03
PP-TSM + 数据扩展 91.03 BMN LSTM + 扩展部分动作的proposal 81.2 71.8 76.2

以上所有实验的对比数据如下：

图像特征提取模型 top1 acc 时序提名生成 序列分类 precision recall F1-score
TSN 75.86 BMN LSTM 60.04 61.31 60.67
TSM 90.24 BMN LSTM 71.06 65.93 68.4
TSM 90.24 BMN LSTM + 扩展全部动作的proposal 80.3 66.31 72.64
TSM 90.24 BMN LSTM + 扩展部分动作的proposal 79.37 67.76 73.1
TSM + 数据扩展 90.85 BMN LSTM + 扩展部分动作的proposal 80.22 68.72 74.03
PP-TSM + 数据扩展 91.03 BMN LSTM + 扩展部分动作的proposal 81.2 71.8 76.2

欢迎报名直播课加入交流群，如需更多技术交流与合作可点击以下链接
https://paddleqiyeban.wjx.cn/vj/Qlb0uS3.aspx?udsid=531417

参考文献

  • Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, Shilei Wen,2019. BMN: Boundary-Matching Network for Temporal Action Proposal Generation. https://arxiv.org/pdf/1907.09702.pdf

资源

更多资源请参考：

  • 更多深度学习知识、产业案例，请参考：awesome-DeepLearning

  • 更多动作识别、动作检测、多模态、视频目标分割、单目深度估计模型，请参考：PaddleVideo

  • 更多学习资料请参阅 飞桨深度学习平台

基于PP-TSM+BMN+LSTM实现足球精彩时刻剪辑_副本1相关推荐

  1. DL之LSTM：基于tensorflow框架利用LSTM算法对气温数据集训练并回归预测

    DL之LSTM:基于tensorflow框架利用LSTM算法对气温数据集训练并回归预测 目录 输出结果 核心代码 输出结果 数据集 tensorboard可视化 iter: 0 loss: 0.010 ...

  2. TF之LSTM：基于tensorflow框架自定义LSTM算法实现股票历史(1990~2015数据集,6112预测后100+单变量最高)行情回归预测

    TF之LSTM:基于tensorflow框架自定义LSTM算法实现股票历史(1990~2015数据集,6112预测后100+单变量最高)行情回归预测 目录 输出结果 LSTM代码 输出结果 数据集 L ...

  3. 【预测模型】基于BP神经网络、LSTM、GRNN实现风电功率预测附matlab代码

    1 简介 风电功率预测结果的准确性,不仅关系到风力发电厂的综合运行效率,也与区域运行成本具备直接联系,基于BP神经网络.LSTM.GRNN实现风电功率预测.​经过实例分析,证明设计的方法对风电功率的预 ...

  4. 基于RNN循环神经网络lstm的藏头诗制作

    基于RNN循环神经网络lstm的藏头诗制作 简单介绍 在一次偶然中接触到藏头诗,觉得十分有意思.但是好像都是利用古代本就有的诗句重新组合而成.比如输入清风袭来,结果如下图所示. 之后想到不如利用深度学 ...

  5. 基于YOLO的王者荣耀精彩时刻自动剪辑

    一.需求背景 企鹅电竞是一款游戏直播产品,其中游戏短视频社区是其重要组成部分. 为了丰富游戏短视频内容,针对王者荣耀,需要一套自动化剪辑精彩时刻的系统,以能够快速根据主播直播内容生成精彩时刻反馈到游戏 ...

  6. 时序预测 | MATLAB实现基于Adam算法优化LSTM长短期记忆神经网络时间序列预测

    时序预测 | MATLAB实现基于Adam算法优化LSTM长短期记忆神经网络时间序列预测 目录 时序预测 | MATLAB实现基于Adam算法优化LSTM长短期记忆神经网络时间序列预测 效果一览 基本 ...

  7. 基于长短期记忆网络(LSTM)对股票价格的涨跌幅度进行预测

    完整代码:https://download.csdn.net/download/qq_38735017/87536579 为对股票价格的涨跌幅度进行预测,本文使用了基于长短期记忆网络(LSTM)的方法 ...

  8. 基于麻雀算法优化LSTM回归预测（matlab）

    基于麻雀算法优化LSTM回归预测(matlab) 概述: 麻雀算法构思 lstm原理 麻雀优化lstm原理 代码及结果展示 第一部分 麻雀算法构思 众所周知,麻雀是常见的留鸟而且非常喜欢群居.这种生物 ...

  9. 基于keras的双层LSTM网络和双向LSTM网络

    1 前言 基于keras的双层LSTM网络和双向LSTM网络中,都会用到 LSTM层,主要参数如下: LSTM(units,input_shape,return_sequences=False) un ...

最新文章

  1. BroadcastReceiver 启动activity（在activity之外启动一个activity）
  2. 智能合约如何可信的与外部世界交互
  3. Azure认知服务之使用墨迹识别功能识别手写汉字
  4. Eclipse中将java类打成jar包形式运行
  5. 一张倾斜图片进行矫正 c++_专业性文章：10分钟矫正骨盆前倾
  6. Top20的OpenSSH服务器最佳安全实践--SSHD_CONFIG配置文件详细解读
  7. 【Android Demo】简单手机通讯录
  8. 中国制造强在哪儿？从美特斯邦威到Shein
  9. mina 和 xsocket
  10. iptables/arptables实现单IP一级二级路由
  11. Boost Asio Work类
  12. 【转】利用匿名namespace解决C++中重复定义的问题
  13. 计算机应用基础考试excel操作题,计算机应用基础上机操作试题
  14. 量子通信，到底是什么工作原理？
  15. 论文-OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Grap
  16. 2022最新u盘升级重装win10方法
  17. Android studio语音识别集成科大讯飞语音转文字
  18. 2010中国互联网十大搞笑段子，笑后请深思
  19. Linux输入命令不显示
  20. 【H3C】pc+交换机+路由器+连外网

热门文章

  1. Excel提取汇总所有工作表名
  2. 我写了一个语音识别引擎
  3. VBA编程图表（二十一）
  4. 仿热血江湖游戏Players装备加解锁功能未开放,请联系客服人员
  5. 由先序中序得后序，中序后序得先序代码实现
  6. Python-Django毕业设计汽车站售票管理系统（程序+Lw)
  7. 刨根问底：linux中bash shell中SIGHUP和SIGTERM信号的处理
  8. 实现炫酷的卡片式动画！
  9. 微信公众号12大互动技巧
  10. （32）【文件下载漏洞专题】Filedownload原理、漏洞出现、危害、漏洞利用……