关于NTU-RGB+D数据集skeleton数据

本人最近在做相关的研究，分享一下关于此数据集的经验。
这个数据集是来源于南洋理工大学的ROSE实验室，16年出了60种动作，后来又新增了60种，成为NTU RGB+D 120。随数据集发表的还有他们的文章——NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis 文章是在CVPR上发表的，算是权威的数据集了。数据集提供了60种动作，3种拍摄角度，多种互动场景和多种演员设置（年龄、性别覆盖比较全面），同一动作可能重复表演（站姿，坐姿）。拍摄设备来自于kinect 2.0。
数据集的获取方法是需要申请账号的，在表明研究者的身份后官方会邮件回复的。

数据集官网：
http://rose1.ntu.edu.sg/Datasets/actionRecognition.asp

该数据集包含每个样本的RGB视频，深度图序列，3D骨架数据和红外视频。这里我们只聊skeleton文件，根据官方在GitHub上公布的matlab源码可以获取整个skeleton的信息：


function []=show_skeleton_on_RGB_frames(...skeletonfilename,rgbfilename,outputvideofilename)
% Draws the skeleton data on RGB frames.
%
% Argrument:
%   skeletonfilename: full adress and filename of the .skeleton file.
%   rgbfilename: corresponding RGB video file
%   outputvideofilename (optional): the filename for output video file.
%
% For further information please refer to:
%   NTU RGB+D dataset's webpage:
%       http://rose1.ntu.edu.sg/Datasets/actionRecognition.asp
%   NTU RGB+D dataset's github page:
%        https://github.com/shahroudy/NTURGB-D
%   CVPR 2016 paper:
%       Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang,
%       "NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis",
%       in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016bodyinfo = read_skeleton_file(skeletonfilename);videofile = VideoReader(rgbfilename);
rgbvid = read(videofile);if nargin>2 % if the output video file is givenwriterObj = VideoWriter(outputvideofilename);open(writerObj);
end% in the skeleton structure, each joint is connected to some other joint:
connecting_joint = ...[2 1 21 3 21 5 6 7 21 9 10 11 1 13 14 15 1 17 18 19 2 8 8 12 12];% reapeat this for every frame
% for f=1:numel(bodyinfo)
for f=1:numel(bodyinfo)tryimrgb = rgbvid(:,:,:,f);% for all the detected skeletons in the current frame:for b=1:numel(bodyinfo(f).bodies)% for all the 25 joints within each skeleton:for j=1:25try% use red color for drawing joint connectionsrv=255;gv=0;bv=0;k = connecting_joint(j);joint = bodyinfo(f).bodies(b).joints(j);dx = joint.colorX;dy = joint.colorY;joint2 = bodyinfo(f).bodies(b).joints(k);dx2 = joint2.colorX;dy2 = joint2.colorY;xdist=abs(dx-dx2);ydist=abs(dy-dy2);% locate the pixels of the connecting line between the% two jointsif xdist>ydistxrange = [dx:sign(dx2-dx):dx2];yrange = [dy:sign(dy2-dy)*abs((dy2-dy)/(dx2-dx)):dy2];elseyrange = [dy:sign(dy2-dy):dy2];xrange = [dx:sign(dx2-dx)*abs((dx2-dx)/(dy2-dy)):dx2];end% draw the line!for i=1:numel(xrange)dx = int32(round(xrange(i)));dy = int32(round(yrange(i)));imrgb(dy-3:dy+3,dx-3:dx+3,1)=rv;imrgb(dy-3:dy+3,dx-3:dx+3,2)=gv;imrgb(dy-3:dy+3,dx-3:dx+3,3)=bv;endjoint = bodyinfo(f).bodies(b).joints(j);dx = int32(round(joint.colorX));dy = int32(round(joint.colorY));% use green color to draw jointsrv=0;gv=255;bv=0;imrgb(dy-7:dy+7,dx-7:dx+7,1)=rv;imrgb(dy-7:dy+7,dx-7:dx+7,2)=gv;imrgb(dy-7:dy+7,dx-7:dx+7,3)=bv;catch err1disp(err1);endendendimrgb = imrgb(1:1080,1:1920,:);%imshow(imrgb);writeVideo(writerObj,imrgb);pause(0.001);catch err2disp(err2);end
end
close(writerObj);
end

给一些我自己的注释方便理解：
% framecount当前文件一个有的帧数（1位）
% bodycount 当前帧内存在的人个数（1位）
% body.bodyID 当前人的ID（1位_17）
% arrayint 记录当前的身体信息（6位）
% lean 向X，Y方向偏移程度（2位）
% body.trackingState 身体追踪状态（1位）
% body.jointCount 骨骼点计数
% jointinfo 骨骼点信息（11位*25）
% joint.x,joint.y,joint.z 骨骼点的位置
% joint.depthX，joint.depthY 骨骼点红外位置
% joint.colorX，joint.colorY 骨骼点RGB位置

文件种可能会出现一人或者两人场景，也存在单人误判为两人的情况。也有因为损坏或者采集数据出现特别小的文件，官方给的建议是弃用。

关于数据处理，因为自己也是菜就先导成CSV格式再做其可视化和转成其他array格式训练，发一下批量导CSV的matlab：

% 通过skeleton生成csv文件，按照C和P分成两种数据集
clc;
clear all;
close all;skeleton_path='F:\NTU_RGB_D_DATABASE\nturgbd_skeletons_s001_to_s017\nturgb+d_skeletons';
% skeleton_path='C:\Users\DaiMoYU\Desktop\skeleton_test';
C_path='F:\skeleton2csv_stand\C';
P_path='F:\skeleton_to_csv\P';
flag_30frames=1;% 是否标准化为30帧
max_stand=0;%归一化信息：最大值
min_stand=0;%归一化信息：最小值
%==========================================================================
%创建文件夹
dir_C_path=dir(C_path);
[size_dir_C_path_a,size_dir_C_path_b]=size(dir_C_path);
for ii=3:size_dir_C_path_aCn_path=[dir_C_path(ii).folder,'\',dir_C_path(ii).name];create_filesbin(Cn_path);
end
%==========================================================================
%读取skeleton文件
dir_skeleton_path=dir(skeleton_path);
[size_dir_skeleton_path_a,size_dir_skeleton_path_b]=size(dir_skeleton_path);
n_skeleton=size_dir_skeleton_path_a-3;
for ii=3:size_dir_skeleton_path_askeleton_file_path=[dir_skeleton_path(ii).folder,'\',dir_skeleton_path(ii).name];skeletonfilename=skeleton_file_path;bodyinfo = read_skeleton_file(skeletonfilename);%读取每一帧frames=zeros(numel(bodyinfo),150);for f=1:numel(bodyinfo)% 读取每个人for b=1:numel(bodyinfo(f).bodies)for j=1:25joint_x(b,j) = bodyinfo(f).bodies(b).joints(j).x;joint_y(b,j) = bodyinfo(f).bodies(b).joints(j).y;joint_z(b,j) = bodyinfo(f).bodies(b).joints(j).z;endend%只有一个人if b==1for j=1:25frames(f,(j-1)*3+1)=joint_x(1,j);frames(f,(j-1)*3+2)=joint_y(1,j);frames(f,(j-1)*3+3)=joint_z(1,j);endend%只有两个人if b==2for j=1:25frames(f,(j-1)*3+1)=joint_x(1,j);frames(f,(j-1)*3+2)=joint_y(1,j);frames(f,(j-1)*3+3)=joint_z(1,j);frames(f,75+(j-1)*3+1)=joint_x(2,j);frames(f,75+(j-1)*3+2)=joint_y(2,j);frames(f,75+(j-1)*3+3)=joint_z(2,j);endendend% 此时的frames为[frames，joints]排列，joints=75*2为[body1，body2]排列% body1和body2都为X-Y-Z排列if flag_30frames==1%是否30帧标准化frames_sorce=numel(bodyinfo);distance_frames = fix(frames_sorce/30);for jj=1:30frames_30(jj,:)=frames((jj-1)*distance_frames+1,:);max_stand=max(max(max(frames_30)),max_stand);min_stand=min(min(min(frames_30)),min_stand);endelsedisp('未30帧标准化。')frames_30=frames;max_stand=max(max(max(frames_30)),max_stand);min_stand=min(min(min(frames_30)),min_stand);end%  分析文件名称file_name=dir_skeleton_path(ii).name(1:20);action = file_name(17:20);c_n=file_name(5:8);csv_name=[C_path,'\',c_n,'\',action,'\',file_name,'.csv'];%   存储csvif max(max(frames_30))==min(min(frames_30))disp('文件无效');elsecsvwrite(csv_name,frames_30);enddisp(csv_name);end
%==========================================================================

以及create_filesbin函数：

function create_filesbin(savepath)
% savepath='F:\train_all\';% 创建于此目录下savepath=[savepath,'\'];for i=1:60file_name = sprintf('%s',num2str(i));if length(file_name)==1file_name = ['A00',file_name];elsefile_name = ['A0',file_name];endfile_path_name = strcat(savepath,file_name);file_path_name_ = strcat(file_path_name,'\');if exist(file_path_name_)==0   %该文件夹不存在，则直接创建mkdir(file_path_name_);else                            %该文件夹存在，则先删除再创建rmdir(file_path_name_, 's'); %该文件夹中有没有文件均可mkdir(file_path_name_);endend
end

这里再给出我用前12帧在lstm网络对第13帧的预测的可视化代码，都是转成CSV格式做的读取：

% 绘出前12帧在LSTM输入和预测的结果;
clc;close all;clear all;
path_12 = 'F:\train_pre\testA001\TEST_7_12\S010C001P007R001A001.csv';
path_1_pre  = 'F:\train_pre\pre_S010C001P007R001A001_10_12_13.csv';
%load data
data1_12 = load(path_12);
data1_12_sorce=data1_12;
data_pre_13 = load(path_1_pre);
[size_data1_12_a,size_data1_12_b] = size(data1_12);
[size_data_1_pre_a,size_data_1_pre_b] = size(data_pre_13);
%处理归一化
distance = 4.720282+2.259431;
data1_12=data1_12+2.259431;
data1_12=data1_12/distance;
figure(1)
subplot(1,2,1)
for time_point=1:3%骨骼点赋值与绘制for joint=1:25point_x(joint)= data1_12((joint-1)*3+1,time_point);point_z(joint)= data1_12((joint-1)*3+2,time_point);point_y(joint)= data1_12((joint-1)*3+3,time_point);plot3(point_x(joint),point_y(joint),point_z(joint));point_pre_x(joint) = data_pre_13(1,(joint-1)*3+1);point_pre_z(joint) = data_pre_13(1,(joint-1)*3+2);point_pre_y(joint) = data_pre_13(1,(joint-1)*3+3);plot3(point_pre_x(joint),point_pre_y(joint),point_pre_z(joint));hold on;end%调色板temp1=1; temp2=0+time_point*1/3;temp3=0;
%    temp1=1;
%    temp2=0;
%    temp3=0;%躯干plot3([point_x(1),point_x(2)],[point_y(1),point_y(2)],[point_z(1),point_z(2)],'Color',[temp1 temp2 temp3]);plot3([point_x(1),point_x(17)],[point_y(1),point_y(17)],[point_z(1),point_z(17)],'Color',[temp1 temp2 temp3]);plot3([point_x(1),point_x(13)],[point_y(1),point_y(13)],[point_z(1),point_z(13)],'Color',[temp1 temp2 temp3]);plot3([point_x(21),point_x(2)],[point_y(21),point_y(2)],[point_z(21),point_z(2)],'Color',[temp1 temp2 temp3]);plot3([point_x(21),point_x(5)],[point_y(21),point_y(5)],[point_z(21),point_z(5)],'Color',[temp1 temp2 temp3]);plot3([point_x(21),point_x(9)],[point_y(21),point_y(9)],[point_z(21),point_z(9)],'Color',[temp1 temp2 temp3]);%头部plot3([point_x(21),point_x(3)],[point_y(21),point_y(3)],[point_z(21),point_z(3)],'Color',[temp1 temp2 temp3]);plot3([point_x(4),point_x(3)],[point_y(4),point_y(3)],[point_z(4),point_z(3)],'Color',[temp1 temp2 temp3]);%左臂plot3([point_x(9),point_x(10)],[point_y(9),point_y(10)],[point_z(9),point_z(10)],'Color',[temp1 temp2 temp3]);plot3([point_x(10),point_x(11)],[point_y(10),point_y(11)],[point_z(10),point_z(11)],'Color',[temp1 temp2 temp3]);plot3([point_x(11),point_x(12)],[point_y(11),point_y(12)],[point_z(11),point_z(12)],'Color',[temp1 temp2 temp3]);plot3([point_x(12),point_x(24)],[point_y(12),point_y(24)],[point_z(12),point_z(24)],'Color',[temp1 temp2 temp3]);plot3([point_x(12),point_x(25)],[point_y(12),point_y(25)],[point_z(12),point_z(25)],'Color',[temp1 temp2 temp3]);%右臂plot3([point_x(5),point_x(6)],[point_y(5),point_y(6)],[point_z(5),point_z(6)],'Color',[temp1 temp2 temp3]);plot3([point_x(6),point_x(7)],[point_y(6),point_y(7)],[point_z(6),point_z(7)],'Color',[temp1 temp2 temp3]);plot3([point_x(7),point_x(8)],[point_y(7),point_y(8)],[point_z(7),point_z(8)],'Color',[temp1 temp2 temp3]);plot3([point_x(8),point_x(22)],[point_y(8),point_y(22)],[point_z(8),point_z(22)],'Color',[temp1 temp2 temp3]);plot3([point_x(8),point_x(23)],[point_y(8),point_y(23)],[point_z(8),point_z(23)],'Color',[temp1 temp2 temp3]);%左腿plot3([point_x(17),point_x(18)],[point_y(17),point_y(18)],[point_z(17),point_z(18)],'Color',[temp1 temp2 temp3]);plot3([point_x(18),point_x(19)],[point_y(18),point_y(19)],[point_z(18),point_z(19)],'Color',[temp1 temp2 temp3]);plot3([point_x(20),point_x(19)],[point_y(20),point_y(19)],[point_z(20),point_z(19)],'Color',[temp1 temp2 temp3]);%右腿plot3([point_x(13),point_x(14)],[point_y(13),point_y(14)],[point_z(13),point_z(14)],'Color',[temp1 temp2 temp3]);plot3([point_x(14),point_x(15)],[point_y(14),point_y(15)],[point_z(14),point_z(15)],'Color',[temp1 temp2 temp3]);plot3([point_x(15),point_x(16)],[point_y(15),point_y(16)],[point_z(15),point_z(16)],'Color',[temp1 temp2 temp3]);
endtemp1=0;
temp2=0;
temp3=1;
%躯干
plot3([point_pre_x(1),point_pre_x(2)],[point_pre_y(1),point_pre_y(2)],[point_pre_z(1),point_pre_z(2)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(1),point_pre_x(17)],[point_pre_y(1),point_pre_y(17)],[point_pre_z(1),point_pre_z(17)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(1),point_pre_x(13)],[point_pre_y(1),point_pre_y(13)],[point_pre_z(1),point_pre_z(13)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(21),point_pre_x(2)],[point_pre_y(21),point_pre_y(2)],[point_pre_z(21),point_pre_z(2)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(21),point_pre_x(5)],[point_pre_y(21),point_pre_y(5)],[point_pre_z(21),point_pre_z(5)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(21),point_pre_x(9)],[point_pre_y(21),point_pre_y(9)],[point_pre_z(21),point_pre_z(9)],'Color',[temp1 temp2 temp3]);
%头部
plot3([point_pre_x(21),point_pre_x(3)],[point_pre_y(21),point_pre_y(3)],[point_pre_z(21),point_pre_z(3)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(4),point_pre_x(3)],[point_pre_y(4),point_pre_y(3)],[point_pre_z(4),point_pre_z(3)],'Color',[temp1 temp2 temp3]);
%左臂
plot3([point_pre_x(9),point_pre_x(10)],[point_pre_y(9),point_pre_y(10)],[point_pre_z(9),point_pre_z(10)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(10),point_pre_x(11)],[point_pre_y(10),point_pre_y(11)],[point_pre_z(10),point_pre_z(11)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(11),point_pre_x(12)],[point_pre_y(11),point_pre_y(12)],[point_pre_z(11),point_pre_z(12)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(12),point_pre_x(24)],[point_pre_y(12),point_pre_y(24)],[point_pre_z(12),point_pre_z(24)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(12),point_pre_x(25)],[point_pre_y(12),point_pre_y(25)],[point_pre_z(12),point_pre_z(25)],'Color',[temp1 temp2 temp3]);
%右臂
plot3([point_pre_x(5),point_pre_x(6)],[point_pre_y(5),point_pre_y(6)],[point_pre_z(5),point_pre_z(6)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(6),point_pre_x(7)],[point_pre_y(6),point_pre_y(7)],[point_pre_z(6),point_pre_z(7)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(7),point_pre_x(8)],[point_pre_y(7),point_pre_y(8)],[point_pre_z(7),point_pre_z(8)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(8),point_pre_x(22)],[point_pre_y(8),point_pre_y(22)],[point_pre_z(8),point_pre_z(22)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(8),point_pre_x(23)],[point_pre_y(8),point_pre_y(23)],[point_pre_z(8),point_pre_z(23)],'Color',[temp1 temp2 temp3]);
%左腿
plot3([point_pre_x(17),point_pre_x(18)],[point_pre_y(17),point_pre_y(18)],[point_pre_z(17),point_pre_z(18)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(18),point_pre_x(19)],[point_pre_y(18),point_pre_y(19)],[point_pre_z(18),point_pre_z(19)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(20),point_pre_x(19)],[point_pre_y(20),point_pre_y(19)],[point_pre_z(20),point_pre_z(19)],'Color',[temp1 temp2 temp3]);
%右腿
plot3([point_pre_x(13),point_pre_x(14)],[point_pre_y(13),point_pre_y(14)],[point_pre_z(13),point_pre_z(14)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(14),point_pre_x(15)],[point_pre_y(14),point_pre_y(15)],[point_pre_z(14),point_pre_z(15)],'Color',[temp1 temp2 temp3]);
plot3([point_pre_x(15),point_pre_x(16)],[point_pre_y(15),point_pre_y(16)],[point_pre_z(15),point_pre_z(16)],'Color',[temp1 temp2 temp3]);
grid on;%绘制帧间差图
mat_data=xlsread(path_12);
[a,b]=size(mat_data);for i=1:(b-1)D(i)=sum(sum(abs(mat_data(:,i)-mat_data(:,i+1))));maxD=max(D);
end
subplot(1,2,2)
title(['单位时刻动作位移程度']);
plot(1:b-1,D);
disp(path_12);
disp('frame_distance_max:')
disp(maxD);

暂时分享到这里，希望有大佬带带路。

关于NTU-RGB+D数据集skeleton数据相关推荐

NTU RGB+D动作识别数据集
GitHub:基于3D骨架数据的双流卷积网络动作识别算法数据集下载:3D骨架数据(60+60)类 NTU RGB + D动作识别数据集由56,880个动作样本组成,包含每个样本的RGB视频,深度图序 ...
Python学习笔记：使用PIL批量合成jpg+png图片，用于水印、合并图片，生成YOLO数据集+标注数据等等
[Python学习笔记] 使用PIL,批量合成jpg+png图片,用于水印.合并图片,生成YOLO数据集+标注数据等等起因:在了解很多关于YOLOv5的相关内容,于是尝试着去训练自己的游戏目标检测模 ...
R语言dplyr包的slice函数提取数据集的数据行实战
R语言dplyr包的slice函数提取数据集的数据行实战目录 R语言dplyr包的slice函数提取数据集的数据行实战 #导入dplyr包
ML之FE：对人类性别相关属性数据集进行数据特征分布可视化分析与挖掘
ML之FE:对人类性别相关属性数据集进行数据特征分布可视化分析与挖掘目录对人类性别相关属性数据集进行数据特征分布可视化分析与挖掘输出结果实现代码对人类性别相关属性数据集进行数据特征分布可视化 ...
Keras之MLPR：利用MLPR算法(3to1【窗口法】+【Input(3)→(12+8)(relu)→O(mse)】)实现根据历史航空旅客数量数据集(时间序列数据)预测下月乘客数量问题
Keras之MLPR:利用MLPR算法(3to1[窗口法]+[Input(3)→(12+8)(relu)→O(mse)])实现根据历史航空旅客数量数据集(时间序列数据)预测下月乘客数量问题目录输出 ...
Keras之MLPR：利用MLPR算法(1to1+【Input(1)→8(relu)→O(mse)】)实现根据历史航空旅客数量数据集(时间序列数据)预测下月乘客数量问题
Keras之MLPR:利用MLPR算法(1to1+[Input(1)→8(relu)→O(mse)])实现根据历史航空旅客数量数据集(时间序列数据)预测下月乘客数量问题目录输出结果设计思路实现 ...
处理自己的数据集_手写代码实现KDD CUP99数据集的数据归一化处理
归一化是数据处理的常用方法之一,目的是消除不同评价指标之间的量纲对数据分析结果的影响,使各指标处于同一数量级,以解决数据指标之间的可比性问题. 目前学术界关于归一化和标准化的概念还不统一,常常会把这两 ...
对PASCAL VOC 数据集进行数据增强
对PASCAL VOC 数据集进行数据增强 1.GitHub仓库位置目的:对VOC数据集的对象检测的数据进行数据增强.如果能帮到您请给本人一颗⭐,拜托了!!!!! https://github.co ...
数据预处理泰坦尼克号_了解泰坦尼克号数据集的数据预处理
数据预处理泰坦尼克号什么是数据预处理? (What is Data Pre-Processing?) We know from my last blog that data preprocessi ...

关于NTU-RGB+D数据集skeleton数据

关于NTU-RGB+D数据集skeleton数据

关于NTU-RGB+D数据集skeleton数据相关推荐

最新文章

热门文章