本文参考的文章：

ROS下开发

前提是需要提前配置好相机在ROS中的运行环境Ubuntu18.04+ROS+ 乐视三合一深度相机配置使用，然后通过程序对终端中发布的深度图节点和彩色图节点进行采集并保存。

运行ROS节点

配置好环境后，通过CTRL+ALT+T打开一个终端，并运行相机节点：

roslaunch astra_camera astrapro.launch

查看相机的话题及画面

通过CTRL+ALT+T打开一个终端,输入：

rqt_image_view

从图中可查看深度相机、彩色相机以及红外相机的节点，为接下来采集和保存深度和彩色图片作准备。

订阅画面并保存

相机话题启动，运行下方的程序，可实现自动采集深度图和彩色图，需要提前建立好image和depth文件夹，以防找不到地址。

#!/usr/bin/env python3   #不同的python环境，可修改编辑器地址
# -*- coding: utf-8 -*- import rospy
from geometry_msgs.msg import PointStamped
from nav_msgs.msg import Path
from sensor_msgs.msg import Image
import cv2 as cv
import numpy as np
from cv_bridge import CvBridgeclass save_image():def __init__(self):self.count = 0self.cvbridge = CvBridge()def message(self, data):print(data.encoding)def save_image(self, data):image = self.cvbridge.imgmsg_to_cv2(data,  desired_encoding='rgb8') #彩色图编码格式：rgb8image = cv.cvtColor(image,cv.COLOR_BGR2RGB)image = image[144:336]if self.count < 10:name = '00000{}'.format(self.count)      elif self.count < 100 and self.count >= 10:name = '0000{}'.format(self.count)      elif self.count < 1000 and self.count >= 100:name = '00{}'.format(self.count)      elif self.count < 10000 and self.count >= 1000:name = '0{}'.format(self.count)      elif self.count < 100000 and self.count >= 10000:name = '{}'.format(self.count)    else:name = '0000000000000'cv.imwrite('./scpict/image/{}.jpg'.format(name), image) #更改为自己彩色图的保存图片路径print('image:  {}'.format(name))self.count += 1def save_depth(self, data):depth = self.cvbridge.imgmsg_to_cv2(data,  desired_encoding='32FC1')#深度图编码格式：16UC1或32FC1depth = depth[144:336]if self.count < 10:name = '00000{}'.format(self.count)      elif self.count < 100 and self.count >= 10:name = '0000{}'.format(self.count)      elif self.count < 1000 and self.count >= 100:name = '000{}'.format(self.count)      elif self.count < 10000 and self.count >= 1000:name = '00{}'.format(self.count)      elif self.count < 100000 and self.count >= 10000:name = '0{}'.format(self.count)    else:name = '0000000000000'cv.imwrite('./scpict/depth/{}.png'.format(name), depth)#更改为自己深度图的保存图片路径print('depth:  {}'.format(name))'''-------------define main----------------'''
if __name__ == '__main__':try:a = save_image()rospy.init_node('save_image', anonymous = True)rospy.Subscriber("/camera/rgb/image_raw",  Image,  a.save_image)      #上边查看的节点彩色图的订阅话题rospy.Subscriber("/camera/depth/image_raw", Image,  a.save_depth)  #上边查看的节点深度图的订阅话题rospy.spin()except rospy.ROSInterruptException:pass

Python环境下开发

首先需要对linux系统下的环境进行配置，可参考Ubuntu下使用Python调用乐视三合一摄像头,需要安装opencv，numpy，openni2三个python的第三方库，用于程序的调用。

调用摄像头并保存采集画面

通过调用时间函数time，对采集到的画面进行分文件夹保存。

from openni import openni2
import numpy as np
import cv2, os, time
from imageio import imsavedef mousecallback(event,x,y,flags,param):if event==cv2.EVENT_LBUTTONDBLCLK:print(y, x, dpt[y,x])if __name__ == "__main__": openni2.initialize()dev = openni2.Device.open_any()print(dev.get_device_info())dev.set_depth_color_sync_enabled(True)# 按照日期创建文件夹save_path = os.path.join(os.getcwd(), "realsense/realsense/out", time.strftime("%Y_%m_%d_%H_%M_%S", time.localtime()))os.mkdir(save_path)depth_stream = dev.create_depth_stream()depth_stream.start()cap = cv2.VideoCapture(1)cv2.namedWindow('depth')cv2.setMouseCallback('depth',mousecallback)saved_count = 0while True:frame = depth_stream.read_frame()dframe_data = np.array(frame.get_buffer_as_triplet()).reshape([480, 640, 2])dpt1 = np.asarray(dframe_data[:, :, 0], dtype='float32')dpt2 = np.asarray(dframe_data[:, :, 1], dtype='float32')dpt2 *= 255dpt = dpt1 + dpt2dpt = dpt[:, ::-1]cv2.imshow('depth', dpt)ret,frame = cap.read()cv2.imshow('color', frame)key = cv2.waitKey(30)if key & 0xFF == ord('s'):imsave(os.path.join((save_path), "{:04d}r.png".format(saved_count)), frame)  # 保存RGB为png文件imsave(os.path.join((save_path), "{:04d}d.tiff".format(saved_count)), dpt)  # 保存深度图为tiff文件saved_count+=1cv2.imshow('save_dpt', dpt)cv2.imshow('save_color', frame)print("save ok " + "{:04d}r.".format(saved_count))if int(key) == ord('q'):breakdepth_stream.stop()dev.close()

程序运行后，鼠标放在图片上，每按一下s键，就自动保存一组深度和彩色图。

C++环境下开发

自己做实验中发现乐视三和一体感摄像头在ROS和Python开发过程中，并没有对深度相机和彩色相机进行配准，导致在采集到的画面中，深度图和彩色图子像素点是不匹配的。并且发现这款相机由于硬件原因，在python代码中，无法实现深度图和彩色图的配准。

C++环境配置

首先需要在自己环境中安装opencv，cmke的环境依赖，可参考全网最详细 Opencv + OpenNi + 奥比中光(Orbbec) Astra Pro /乐视三合一体感摄像头LeTMC-520 + linux 环境搭建来对现有的环境进行配置。

代码编译

本文在C++部分参考奥比中光 Astra Pro 一代（MX400）RGBD 摄像头彩色RGB及深度采集，并在此文源代码进行修改，通过键盘按一下来保存一组深度、彩色和融合图像，并对保存的深度图和彩色图不匹配进行调整。同时，本文程序可直接输出深度图，不再通过生成xml文件再进行转化。

采集和保存程序如下所示：

#include <opencv2/opencv.hpp>
#include <sstream>
#include <fstream>
#include <stdlib.h>
#include <iostream>
#include <string>
#include "/home/lts/OpenNI-Linux-x64-2.3.0.66/Include/OpenNI.h"
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include<sys/stat.h>
#include<sys/types.h>using namespace std;
using namespace cv;
using namespace openni;void writeMatToXML(const std::string xmlName, const cv::Mat & mat)
{FileStorage fs(xmlName, FileStorage::WRITE);fs << "Mat" << mat;fs.release();
};void CheckOpenNIError( Status result, string status )
{ if( result != STATUS_OK ) cerr << status << " Error: " << OpenNI::getExtendedError() << endl;
};class ScenceNumber
{// use a file(number.txt) to recoder different scencepublic:int scNum;string fileName = "./data/number.txt";ScenceNumber(){ifstream f;f.open(this->fileName);if(f.good()){stringstream cvStr;string tmpStr;getline(f, tmpStr);cvStr << tmpStr;cvStr >> this->scNum;f.close();}else{ofstream f(this->fileName);f << "0" << endl;this->scNum = 0;f.close();}}string getNum(){ofstream f(this->fileName);stringstream cvStr;string tmpStr;this->scNum ++;cvStr << this->scNum;cvStr >> tmpStr;f << tmpStr << endl;f.close();cvStr >> tmpStr;return tmpStr;}
};int main( int argc, char** argv )
{Status result = STATUS_OK;  ScenceNumber ScN;string baseFilePath = "./data";string filePath;VideoFrameRef oniDepthImg;//OpenCV imagecv::Mat cvDepthImg;cv::Mat cvBGRImg;cv::Mat cvFusionImg;cv::namedWindow("depth");cv::namedWindow("image");cv::namedWindow("fusion");char key=0;// initialize OpenNI2result = OpenNI::initialize();CheckOpenNIError( result, "initialize context" );  // open device  Device device;result = device.open( openni::ANY_DEVICE );// create depth stream VideoStream oniDepthStream;result = oniDepthStream.create( device, openni::SENSOR_DEPTH );// set depth video modeVideoMode modeDepth;modeDepth.setResolution( 640, 480 );modeDepth.setFps( 30 );modeDepth.setPixelFormat( PIXEL_FORMAT_DEPTH_1_MM );oniDepthStream.setVideoMode(modeDepth);// start depth streamresult = oniDepthStream.start();//create color streamVideoCapture capture;capture.open(2);capture.set(3, 640); //set the rgb sizecapture.set(4, 480);//set depth and color imge registration modeif( device.isImageRegistrationModeSupported(IMAGE_REGISTRATION_DEPTH_TO_COLOR ) ){device.setImageRegistrationMode( IMAGE_REGISTRATION_DEPTH_TO_COLOR );}long numInSc;numInSc = 0;filePath = baseFilePath + "/count" + ScN.getNum();while( key!=27 ) {// read frameif( oniDepthStream.readFrame( &oniDepthImg ) == STATUS_OK ){capture >> cvBGRImg;cv::Mat cvRawImg16U( oniDepthImg.getHeight(), oniDepthImg.getWidth(), CV_16UC1, (void*)oniDepthImg.getData() );cvRawImg16U.convertTo(cvDepthImg, CV_8U, 255.0/(oniDepthStream.getMaxPixelValue()));cvRawImg16U.convertTo(cvRawImg16U, CV_32F);cv::flip(cvDepthImg, cvDepthImg, 1);cv::flip(cvRawImg16U, cvRawImg16U,1);// convert depth image GRAY to BGRcv::cvtColor(cvDepthImg,cvFusionImg,COLOR_GRAY2BGR);cv::imshow( "depth", cvDepthImg );cv::imshow( "image", cvBGRImg );cv::addWeighted(cvBGRImg,0.5,cvFusionImg,0.5,0,cvFusionImg);cv::imshow( "fusion", cvFusionImg );if (key == 's'){  mkdir(filePath.c_str(), 0777);stringstream cvt;string SNumInSc;cvt << numInSc;cvt >> SNumInSc;cv::imwrite(filePath + "/" + SNumInSc + ".tiff", cvRawImg16U);cv::imwrite(filePath + "/" + SNumInSc + ".png", cvBGRImg);cv::imwrite(filePath + "/" + SNumInSc + ".jpg", cvFusionImg);cout << numInSc << "  saved" << endl;cout << filePath << endl;  numInSc ++;} }key = cv::waitKey(100);}//cv destroycv::destroyWindow("depth");cv::destroyWindow("image");cv::destroyWindow("fusion");//OpenNI2 destroyoniDepthStream.destroy();capture.release();device.close();OpenNI::shutdown();return 0;
}

CMake文件如下：

# cmake needs this line
cmake_minimum_required(VERSION 3.1)# Enable C++11
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED TRUE)# Define project name
project(data_capture)# Find OpenCV, you may need to set OpenCV_DIR variable
# to the absolute path to the directory containing OpenCVConfig.cmake file
# via the command line or GUI
INCLUDE_DIRECTORIES($ENV{OPENNI2_INCLUDE})
link_directories($ENV{OPENNI2_REDIST})
find_package(OpenCV REQUIRED)# If the package has been found, several variables will
# be set, you can find the full list with descriptions
# in the OpenCVConfig.cmake file.
# Print some message showing some of them
message(STATUS "OpenCV library status:")
message(STATUS "    config: ${OpenCV_DIR}")
message(STATUS "    version: ${OpenCV_VERSION}")
message(STATUS "    libraries: ${OpenCV_LIBS}")
message(STATUS "    include path: ${OpenCV_INCLUDE_DIRS}")
message(STATUS "    include path: $ENV{OPENNI2_INCLUDE}")# Declare the executable target built from your sources
add_executable(data_capture test.cpp)# Link your application with OpenCV libraries
target_link_libraries(data_capture LINK_PRIVATE ${OpenCV_LIBS} libOpenNI2.so)

上面程序中device.setImageRegistrationMode(IMAGE_REGISTRATION_DEPTH_TO_COLOR);就是配准操作。

奥比中光 astra 乐视三合一体感摄像头采集深度图彩色图并保存相关推荐

全网最详细 Opencv + OpenNi + 奥比中光(Orbbec) Astra Pro /乐视三合一体感摄像头LeTMC-520 + linux 环境搭建
本文参考 Using Orbbec Astra 3D cameras C++20学习:基于Ubuntu系统编译gcc10.2.0 linux下编译安装opencv生成opencv.pc 摄像头方案 / ...
乐视三合一体感摄像头Astra pro开发记录1（深度图、彩色图及点云简单显示）
在某鱼上淘的乐视三合一体感摄像头,捡漏价九十几块,买来玩玩. 网上已经有一些关于此款摄像头的开发资料. 官方的开发资料:官网链接按官方网站以及其他帖子,下载并安装相机的驱动和SDK,不难配置好相机. ...
乐视三合一体感摄像头Astra pro开发记录2（Qt界面）
本文是在上一篇文章乐视三合一体感摄像头Astra pro开发记录1(深度图.彩色图及点云简单显示)的基础上写的.在开学之前终于把界面的基础功能做完了,估计还有一些bug,也没时间继续调了.开学之后得憋 ...
乐视三合一体感摄像头LeTMC-520
这款体感摄像头其实就是奥比中光摄像头(Orbbec Astra Pro)的乐视版 1. 介绍乐视三合一体感摄像头的各个功能模组分部图,包括两个MIC麦克风,一个红外投影模组,一个面部接近感知模组,一 ...
乐视三合一体感摄像头--基本信息及windows下部分开发
乐视三合一体感摄像头--基本信息及windows下部分开发 Introduction 基本信息 Windows下使用安装驱动使用openNI 使用imageJ 调用RGB图像 Q: 参考资料 In ...
乐视三合一体感摄像头--windows下的开发2
乐视三合一体感摄像头--windows下的开发 Introduction 环境配置过程参考资料后言 Introduction 解决上篇教程乐视三合一体感摄像头–基本信息及windows下部分开发: ...
乐视三合一体感摄像头开发（捡漏）笔记——100块要啥自行车
乐视三合一体感摄像头开发(捡漏)笔记不久前在闲鱼上淘了一个乐视体感摄像头,这是乐视之前做的一款周边硬件,它当时是对标Kinect的,该有的结构一点不差,然而乐视凉凉之后,这个摄像头价格也从八九百降到 ...
Unity开发乐视三合一体感摄像头（1）
熟悉Kinect的同学对体感并不陌生.如果能在Android电视上直接安装自己开发的体感APP就剩去了PC的成本.乐视体感摄像头就是直接应用在Android电视的神器.而且开发起来也不困难.而且乐视 ...
乐视三合一摄像头和kinect_乐视三合一体感摄像头开箱体验
很高兴我能成为第一批乐视三合一体感摄像头使用乐迷,说起这款摄像头可以说是业内良心产品.深度分辨率:1280*1024.工作距离可以达到4米,独创体感识别.骨骼识别和拍照摄像头三合一休,可同时支持6人玩 ...

奥比中光 astra 乐视三合一体感摄像头采集深度图彩色图并保存

目录