yunyang tensorflow-yolov3 Intel Realsense D435 （并发）使用locals()函数批量配置摄像头运行识别程序并画框（代码记录）（代码示例）

文章目录

20191126
20191202-1
20191202-2

20191126

# -*- encoding: utf-8 -*-
"""
@File    : test-使用locals()函数批量配置摄像头运行识别程序并画框.py
@Time    : 2019/11/26 11:20
@Author  : Dontla
@Email   : sxana@qq.com
@Software: PyCharm
"""import cv2
import numpy as np
import tensorflow as tf
import core.utils as utils
from core.config import cfg
from core.yolov3 import YOLOV3
import pyrealsense2 as rsclass YoloTest(object):def __init__(self):# D·C 191111：__C.TEST.INPUT_SIZE = 544self.input_size = cfg.TEST.INPUT_SIZEself.anchor_per_scale = cfg.YOLO.ANCHOR_PER_SCALE# Dontla 191106注释：初始化class.names文件的字典信息属性self.classes = utils.read_class_names(cfg.YOLO.CLASSES)# D·C 191115：类数量属性self.num_classes = len(self.classes)self.anchors = np.array(utils.get_anchors(cfg.YOLO.ANCHORS))# D·C 191111：__C.TEST.SCORE_THRESHOLD = 0.3self.score_threshold = cfg.TEST.SCORE_THRESHOLD# D·C 191120：__C.TEST.IOU_THRESHOLD = 0.45self.iou_threshold = cfg.TEST.IOU_THRESHOLDself.moving_ave_decay = cfg.YOLO.MOVING_AVE_DECAY# D·C 191120：__C.TEST.ANNOT_PATH = "./data/dataset/Dontla/20191023_Artificial_Flower/test.txt"self.annotation_path = cfg.TEST.ANNOT_PATH# D·C 191120：__C.TEST.WEIGHT_FILE = "./checkpoint/f_g_c_weights_files/yolov3_test_loss=15.8845.ckpt-47"self.weight_file = cfg.TEST.WEIGHT_FILE# D·C 191115：可写标记（bool类型值）self.write_image = cfg.TEST.WRITE_IMAGE# D·C 191115：__C.TEST.WRITE_IMAGE_PATH = "./data/detection/"（识别图片画框并标注文本后写入的图片路径）self.write_image_path = cfg.TEST.WRITE_IMAGE_PATH# D·C 191116：TEST.SHOW_LABEL设置为Trueself.show_label = cfg.TEST.SHOW_LABEL# D·C 191120：创建命名空间“input”with tf.name_scope('input'):# D·C 191120：建立变量（创建占位符开辟内存空间）self.input_data = tf.placeholder(dtype=tf.float32, name='input_data')self.trainable = tf.placeholder(dtype=tf.bool, name='trainable')model = YOLOV3(self.input_data, self.trainable)self.pred_sbbox, self.pred_mbbox, self.pred_lbbox = model.pred_sbbox, model.pred_mbbox, model.pred_lbbox# D·C 191120：创建命名空间“指数滑动平均”with tf.name_scope('ema'):ema_obj = tf.train.ExponentialMovingAverage(self.moving_ave_decay)# D·C 191120：在允许软设备放置的会话中启动图形并记录放置决策。（不懂啥意思。。。）allow_soft_placement=True表示允许tf自动选择可用的GPU和CPUself.sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))# D·C 191120：variables_to_restore()用于加载模型计算滑动平均值时将影子变量直接映射到变量本身self.saver = tf.train.Saver(ema_obj.variables_to_restore())# D·C 191120：用于下次训练时恢复模型self.saver.restore(self.sess, self.weight_file)def predict(self, image):# D·C 191107：复制一份图片的镜像，避免对图片直接操作改变图片的内在属性org_image = np.copy(image)# D·C 191107：获取图片尺寸org_h, org_w, _ = org_image.shape# D·C 191108：该函数将源图结合input_size，将其转换成预投喂的方形图像（作者默认544×544，中间为缩小尺寸的源图，上下空区域为灰图）：image_data = utils.image_preprocess(image, [self.input_size, self.input_size])# D·C 191108：打印维度看看：# print(image_data.shape)# (544, 544, 3)# D·C 191108：创建新轴，不懂要创建新轴干嘛？image_data = image_data[np.newaxis, ...]# D·C 191108：打印维度看看：# print(image_data.shape)# (1, 544, 544, 3)# D·C 191110：三个box可能存放了预测框图（可能是N多的框，有用的没用的重叠的都在里面）的信息（但是打印出来的值完全看不懂啊喂？）pred_sbbox, pred_mbbox, pred_lbbox = self.sess.run([self.pred_sbbox, self.pred_mbbox, self.pred_lbbox],feed_dict={self.input_data: image_data,self.trainable: False})# D·C 191110：打印三个box的类型、形状和值看看：# print(type(pred_sbbox))# print(type(pred_mbbox))# print(type(pred_lbbox))# 都是<class 'numpy.ndarray'># print(pred_sbbox.shape)# print(pred_mbbox.shape)# print(pred_lbbox.shape)# (1, 68, 68, 3, 6)# (1, 34, 34, 3, 6)# (1, 17, 17, 3, 6)# print(pred_sbbox)# print(pred_mbbox)# print(pred_lbbox)# D·C 191110：（-1，6）表示不知道有多少行，反正你给我整成6列，然后concatenate又把它们仨给叠起来，最终得到无数个6列数组（后面self.num_classes)个数存放的貌似是这个框属于类的概率）pred_bbox = np.concatenate([np.reshape(pred_sbbox, (-1, 5 + self.num_classes)),np.reshape(pred_mbbox, (-1, 5 + self.num_classes)),np.reshape(pred_lbbox, (-1, 5 + self.num_classes))], axis=0)# D·C 191111：打印pred_bbox和它的维度看看：# print(pred_bbox)# print(pred_bbox.shape)# (18207, 6)# D·C 191111：猜测是第一道过滤，过滤掉score_threshold以下的图片，过滤完之后少了好多：# D·C 191115：bboxes维度为[n,6]，前四列是坐标，第五列是得分，第六列是对应类下标bboxes = utils.postprocess_boxes(pred_bbox, (org_h, org_w), self.input_size, self.score_threshold)# D·C 191111：猜测是第二道过滤，过滤掉iou_threshold以下的图片：bboxes = utils.nms(bboxes, self.iou_threshold)return bboxesdef dontla_evaluate_detect(self):ctx = rs.context()# 判断摄像头是否全部连接cam_num = len(ctx.devices)if cam_num < 2:print('摄像头未全部连接！')else:for i in range(cam_num):locals()['pipeline' + str(i)] = rs.pipeline()locals()['config' + str(i)] = rs.config()locals()['serial' + str(i)] = ctx.devices[i].get_info(rs.camera_info.serial_number)locals()['config' + str(i)].enable_device(locals()['serial' + str(i)])locals()['config' + str(i)].enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)locals()['config' + str(i)].enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)locals()['pipeline' + str(i)].start(locals()['config' + str(i)])# 创建对齐对象（深度对齐颜色）locals()['align' + str(i)] = rs.align(rs.stream.color)try:while True:for i in range(cam_num):locals()['frames' + str(i)] = locals()['pipeline' + str(i)].wait_for_frames()# 获取对齐帧集locals()['aligned_frames' + str(i)] = locals()['align' + str(i)].process(locals()['frames' + str(i)])# 获取对齐后的深度帧和彩色帧locals()['aligned_depth_frame' + str(i)] = locals()['aligned_frames' + str(i)].get_depth_frame()locals()['color_frame' + str(i)] = locals()['aligned_frames' + str(i)].get_color_frame()if not locals()['aligned_depth_frame' + str(i)] or not locals()['color_frame' + str(i)]:continue# 获取颜色帧内参locals()['color_profile' + str(i)] = locals()['color_frame' + str(i)].get_profile()locals()['cvsprofile' + str(i)] = rs.video_stream_profile(locals()['color_profile' + str(i)])locals()['color_intrin' + str(i)] = locals()['cvsprofile' + str(i)].get_intrinsics()locals()['color_intrin_part' + str(i)] = [locals()['color_intrin' + str(i)].ppx,locals()['color_intrin' + str(i)].ppy,locals()['color_intrin' + str(i)].fx,locals()['color_intrin' + str(i)].fy]locals()['color_image' + str(i)] = np.asanyarray(locals()['color_frame' + str(i)].get_data())# D·C 191121：显示帧看看# cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)# cv2.imshow('RealSense', color_frame)# cv2.waitKey(1)locals()['bboxes_pr' + str(i)] = self.predict(locals()['color_image' + str(i)])locals()['image' + str(i)] = utils.draw_bbox(locals()['color_image' + str(i)],locals()['bboxes_pr' + str(i)],locals()['aligned_depth_frame' + str(i)],locals()['color_intrin_part' + str(i)],show_label=self.show_label)# cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)cv2.imshow('window{}'.format(i), locals()['image' + str(i)])cv2.waitKey(1)finally:locals()['pipeline' + str(i)].stop()if __name__ == '__main__':YoloTest().dontla_evaluate_detect()

20191202-1

增加了摄像头初始化机制（hardware_reset）、设备检测机制、程序终止机制等。

# -*- coding: utf-8 -*-
"""
@File    : test-multicam_multithreading.py
@Time    : 2019/11/30 15:18
@Author  : Dontla
@Email   : sxana@qq.com
@Software: PyCharm
"""import cv2
import numpy as np
import tensorflow as tf
import core.utils as utils
from core.config import cfg
from core.yolov3 import YOLOV3
import pyrealsense2 as rs
import time
import sysclass YoloTest(object):def __init__(self):# D·C 191111：__C.TEST.INPUT_SIZE = 544self.input_size = cfg.TEST.INPUT_SIZEself.anchor_per_scale = cfg.YOLO.ANCHOR_PER_SCALE# Dontla 191106注释：初始化class.names文件的字典信息属性self.classes = utils.read_class_names(cfg.YOLO.CLASSES)# D·C 191115：类数量属性self.num_classes = len(self.classes)self.anchors = np.array(utils.get_anchors(cfg.YOLO.ANCHORS))# D·C 191111：__C.TEST.SCORE_THRESHOLD = 0.3self.score_threshold = cfg.TEST.SCORE_THRESHOLD# D·C 191120：__C.TEST.IOU_THRESHOLD = 0.45self.iou_threshold = cfg.TEST.IOU_THRESHOLDself.moving_ave_decay = cfg.YOLO.MOVING_AVE_DECAY# D·C 191120：__C.TEST.ANNOT_PATH = "./data/dataset/Dontla/20191023_Artificial_Flower/test.txt"self.annotation_path = cfg.TEST.ANNOT_PATH# D·C 191120：__C.TEST.WEIGHT_FILE = "./checkpoint/f_g_c_weights_files/yolov3_test_loss=15.8845.ckpt-47"self.weight_file = cfg.TEST.WEIGHT_FILE# D·C 191115：可写标记（bool类型值）self.write_image = cfg.TEST.WRITE_IMAGE# D·C 191115：__C.TEST.WRITE_IMAGE_PATH = "./data/detection/"（识别图片画框并标注文本后写入的图片路径）self.write_image_path = cfg.TEST.WRITE_IMAGE_PATH# D·C 191116：TEST.SHOW_LABEL设置为Trueself.show_label = cfg.TEST.SHOW_LABEL# D·C 191120：创建命名空间“input”with tf.name_scope('input'):# D·C 191120：建立变量（创建占位符开辟内存空间）self.input_data = tf.placeholder(dtype=tf.float32, name='input_data')self.trainable = tf.placeholder(dtype=tf.bool, name='trainable')model = YOLOV3(self.input_data, self.trainable)self.pred_sbbox, self.pred_mbbox, self.pred_lbbox = model.pred_sbbox, model.pred_mbbox, model.pred_lbbox# D·C 191120：创建命名空间“指数滑动平均”with tf.name_scope('ema'):ema_obj = tf.train.ExponentialMovingAverage(self.moving_ave_decay)# D·C 191120：在允许软设备放置的会话中启动图形并记录放置决策。（不懂啥意思。。。）allow_soft_placement=True表示允许tf自动选择可用的GPU和CPUself.sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))# D·C 191120：variables_to_restore()用于加载模型计算滑动平均值时将影子变量直接映射到变量本身self.saver = tf.train.Saver(ema_obj.variables_to_restore())# D·C 191120：用于下次训练时恢复模型self.saver.restore(self.sess, self.weight_file)def predict(self, image):# D·C 191107：复制一份图片的镜像，避免对图片直接操作改变图片的内在属性org_image = np.copy(image)# D·C 191107：获取图片尺寸org_h, org_w, _ = org_image.shape# D·C 191108：该函数将源图结合input_size，将其转换成预投喂的方形图像（作者默认544×544，中间为缩小尺寸的源图，上下空区域为灰图）：image_data = utils.image_preprocess(image, [self.input_size, self.input_size])# D·C 191108：打印维度看看：# print(image_data.shape)# (544, 544, 3)# D·C 191108：创建新轴，不懂要创建新轴干嘛？image_data = image_data[np.newaxis, ...]# D·C 191108：打印维度看看：# print(image_data.shape)# (1, 544, 544, 3)# D·C 191110：三个box可能存放了预测框图（可能是N多的框，有用的没用的重叠的都在里面）的信息（但是打印出来的值完全看不懂啊喂？）pred_sbbox, pred_mbbox, pred_lbbox = self.sess.run([self.pred_sbbox, self.pred_mbbox, self.pred_lbbox],feed_dict={self.input_data: image_data,self.trainable: False})# D·C 191110：打印三个box的类型、形状和值看看：# print(type(pred_sbbox))# print(type(pred_mbbox))# print(type(pred_lbbox))# 都是<class 'numpy.ndarray'># print(pred_sbbox.shape)# print(pred_mbbox.shape)# print(pred_lbbox.shape)# (1, 68, 68, 3, 6)# (1, 34, 34, 3, 6)# (1, 17, 17, 3, 6)# print(pred_sbbox)# print(pred_mbbox)# print(pred_lbbox)# D·C 191110：（-1，6）表示不知道有多少行，反正你给我整成6列，然后concatenate又把它们仨给叠起来，最终得到无数个6列数组（后面self.num_classes)个数存放的貌似是这个框属于类的概率）pred_bbox = np.concatenate([np.reshape(pred_sbbox, (-1, 5 + self.num_classes)),np.reshape(pred_mbbox, (-1, 5 + self.num_classes)),np.reshape(pred_lbbox, (-1, 5 + self.num_classes))], axis=0)# D·C 191111：打印pred_bbox和它的维度看看：# print(pred_bbox)# print(pred_bbox.shape)# (18207, 6)# D·C 191111：猜测是第一道过滤，过滤掉score_threshold以下的图片，过滤完之后少了好多：# D·C 191115：bboxes维度为[n,6]，前四列是坐标，第五列是得分，第六列是对应类下标bboxes = utils.postprocess_boxes(pred_bbox, (org_h, org_w), self.input_size, self.score_threshold)# D·C 191111：猜测是第二道过滤，过滤掉iou_threshold以下的图片：bboxes = utils.nms(bboxes, self.iou_threshold)return bboxesdef dontla_evaluate_detect(self):ctx = rs.context()# devices = ctx.query_devices()# 摄像头个数cam_num = 6# 循环reset摄像头# hardware_reset()后是不是应该延迟一段时间？不延迟就会报错for dev in ctx.query_devices():dev.hardware_reset()while len(ctx.query_devices()) != cam_num:time.sleep(0.5)print('摄像头{}初始化成功'.format(dev.get_info(rs.camera_info.serial_number)))# D·C 191202：猜测摄像头重置后的若干秒内，摄像头是不稳定的，这跟访问ctx.query_devices()时设备丢失是否是同一回事，有待考证！# D·C 191202：除此之外，我还怀疑，访问ctx.query_devices()会对设备连接造成影响，在这里，我们尽量减少访问频率。如果没有影响，那还是要等设备稳定再运行。# 设置睡眠延时倒计时，防止重置失败# sleep_time = 0# for i in range(sleep_time):#     print('倒计时{}'.format(sleep_time - i))#     time.sleep(1)# 循环验证摄像头个数是否为6，如果是则继续向下，并获取实际连接的摄像头个数。否则持续验证（验证次数超过限制则退出程序）。devices = ctx.query_devices()connected_cam_num = len(devices)veri_times = 10while connected_cam_num != cam_num:veri_times -= 1if veri_times == -1:sys.exit()devices = ctx.query_devices()connected_cam_num = len(devices)print('摄像头个数：{}'.format(connected_cam_num))# 打印摄像头序列号和接口号并创建需要显示在窗口上的备注信息字符串列表（窗口名）cam_id = 0serial_list = []for i in devices:cam_id += 1serial_list.append('camera{}; serials number {}; usb port {}'.format(cam_id, i.get_info(rs.camera_info.serial_number),i.get_info(rs.camera_info.usb_type_descriptor)))print('serial number {}：{}；usb port：{}'.format(cam_id, i.get_info(rs.camera_info.serial_number),i.get_info(rs.camera_info.usb_type_descriptor)))# 配置各个摄像头的基本对象for i in range(connected_cam_num):locals()['pipeline' + str(i)] = rs.pipeline()locals()['config' + str(i)] = rs.config()locals()['serial' + str(i)] = ctx.devices[i].get_info(rs.camera_info.serial_number)locals()['config' + str(i)].enable_device(locals()['serial' + str(i)])locals()['config' + str(i)].enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)locals()['config' + str(i)].enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)locals()['pipeline' + str(i)].start(locals()['config' + str(i)])# 创建对齐对象（深度对齐颜色）locals()['align' + str(i)] = rs.align(rs.stream.color)# 运行流并进行识别try:# 设置break标志，方便按下按钮跳出循环退出窗口break2 = Falsewhile True:for i in range(connected_cam_num):locals()['frames' + str(i)] = locals()['pipeline' + str(i)].wait_for_frames()# 获取对齐帧集locals()['aligned_frames' + str(i)] = locals()['align' + str(i)].process(locals()['frames' + str(i)])# 获取对齐后的深度帧和彩色帧locals()['aligned_depth_frame' + str(i)] = locals()['aligned_frames' + str(i)].get_depth_frame()locals()['color_frame' + str(i)] = locals()['aligned_frames' + str(i)].get_color_frame()if not locals()['aligned_depth_frame' + str(i)] or not locals()['color_frame' + str(i)]:continue# 获取颜色帧内参locals()['color_profile' + str(i)] = locals()['color_frame' + str(i)].get_profile()locals()['cvsprofile' + str(i)] = rs.video_stream_profile(locals()['color_profile' + str(i)])locals()['color_intrin' + str(i)] = locals()['cvsprofile' + str(i)].get_intrinsics()locals()['color_intrin_part' + str(i)] = [locals()['color_intrin' + str(i)].ppx,locals()['color_intrin' + str(i)].ppy,locals()['color_intrin' + str(i)].fx,locals()['color_intrin' + str(i)].fy]locals()['color_image' + str(i)] = np.asanyarray(locals()['color_frame' + str(i)].get_data())# D·C 191121：显示帧看看# cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)# cv2.imshow('RealSense', color_frame)# cv2.waitKey(1)locals()['bboxes_pr' + str(i)] = self.predict(locals()['color_image' + str(i)])locals()['image' + str(i)] = utils.draw_bbox(locals()['color_image' + str(i)],locals()['bboxes_pr' + str(i)],locals()['aligned_depth_frame' + str(i)],locals()['color_intrin_part' + str(i)],show_label=self.show_label)# cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)# cv2.imshow('window{}'.format(i), locals()['image' + str(i)])cv2.imshow('{}'.format(serial_list[i]), locals()['image' + str(i)])key = cv2.waitKey(1)# 如果按下ESC，则跳出循环if key == 27:# 貌似直接用return也行# returnbreak2 = Truebreakif break2 == True:breakfinally:# 大概觉得先关闭窗口再停止流比较靠谱# 销毁所有窗口cv2.destroyAllWindows()print('已关闭所有窗口！')# 停止所有流locals()['pipeline' + str(i)].stop()print('正在停止所有流，请等待数秒至程序稳定结束！')if __name__ == '__main__':YoloTest().dontla_evaluate_detect()print('程序已结束！')

20191202-2

增加了连续验证机制，优化了部分结构，并且处理了一部分代码

# -*- coding: utf-8 -*-
"""
@File    : test-multicam_multithreading.py
@Time    : 2019/11/30 15:18
@Author  : Dontla
@Email   : sxana@qq.com
@Software: PyCharm
"""import cv2
import numpy as np
import tensorflow as tf
import core.utils as utils
from core.config import cfg
from core.yolov3 import YOLOV3
import pyrealsense2 as rs
import time
import sysclass YoloTest(object):def __init__(self):# D·C 191111：__C.TEST.INPUT_SIZE = 544self.input_size = cfg.TEST.INPUT_SIZEself.anchor_per_scale = cfg.YOLO.ANCHOR_PER_SCALE# Dontla 191106注释：初始化class.names文件的字典信息属性self.classes = utils.read_class_names(cfg.YOLO.CLASSES)# D·C 191115：类数量属性self.num_classes = len(self.classes)self.anchors = np.array(utils.get_anchors(cfg.YOLO.ANCHORS))# D·C 191111：__C.TEST.SCORE_THRESHOLD = 0.3self.score_threshold = cfg.TEST.SCORE_THRESHOLD# D·C 191120：__C.TEST.IOU_THRESHOLD = 0.45self.iou_threshold = cfg.TEST.IOU_THRESHOLDself.moving_ave_decay = cfg.YOLO.MOVING_AVE_DECAY# D·C 191120：__C.TEST.ANNOT_PATH = "./data/dataset/Dontla/20191023_Artificial_Flower/test.txt"self.annotation_path = cfg.TEST.ANNOT_PATH# D·C 191120：__C.TEST.WEIGHT_FILE = "./checkpoint/f_g_c_weights_files/yolov3_test_loss=15.8845.ckpt-47"self.weight_file = cfg.TEST.WEIGHT_FILE# D·C 191115：可写标记（bool类型值）self.write_image = cfg.TEST.WRITE_IMAGE# D·C 191115：__C.TEST.WRITE_IMAGE_PATH = "./data/detection/"（识别图片画框并标注文本后写入的图片路径）self.write_image_path = cfg.TEST.WRITE_IMAGE_PATH# D·C 191116：TEST.SHOW_LABEL设置为Trueself.show_label = cfg.TEST.SHOW_LABEL# D·C 191120：创建命名空间“input”with tf.name_scope('input'):# D·C 191120：建立变量（创建占位符开辟内存空间）self.input_data = tf.placeholder(dtype=tf.float32, name='input_data')self.trainable = tf.placeholder(dtype=tf.bool, name='trainable')model = YOLOV3(self.input_data, self.trainable)self.pred_sbbox, self.pred_mbbox, self.pred_lbbox = model.pred_sbbox, model.pred_mbbox, model.pred_lbbox# D·C 191120：创建命名空间“指数滑动平均”with tf.name_scope('ema'):ema_obj = tf.train.ExponentialMovingAverage(self.moving_ave_decay)# D·C 191120：在允许软设备放置的会话中启动图形并记录放置决策。（不懂啥意思。。。）allow_soft_placement=True表示允许tf自动选择可用的GPU和CPUself.sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))# D·C 191120：variables_to_restore()用于加载模型计算滑动平均值时将影子变量直接映射到变量本身self.saver = tf.train.Saver(ema_obj.variables_to_restore())# D·C 191120：用于下次训练时恢复模型self.saver.restore(self.sess, self.weight_file)def predict(self, image):# D·C 191107：复制一份图片的镜像，避免对图片直接操作改变图片的内在属性org_image = np.copy(image)# D·C 191107：获取图片尺寸org_h, org_w, _ = org_image.shape# D·C 191108：该函数将源图结合input_size，将其转换成预投喂的方形图像（作者默认544×544，中间为缩小尺寸的源图，上下空区域为灰图）：image_data = utils.image_preprocess(image, [self.input_size, self.input_size])# D·C 191108：打印维度看看：# print(image_data.shape)# (544, 544, 3)# D·C 191108：创建新轴，不懂要创建新轴干嘛？image_data = image_data[np.newaxis, ...]# D·C 191108：打印维度看看：# print(image_data.shape)# (1, 544, 544, 3)# D·C 191110：三个box可能存放了预测框图（可能是N多的框，有用的没用的重叠的都在里面）的信息（但是打印出来的值完全看不懂啊喂？）pred_sbbox, pred_mbbox, pred_lbbox = self.sess.run([self.pred_sbbox, self.pred_mbbox, self.pred_lbbox],feed_dict={self.input_data: image_data,self.trainable: False})# D·C 191110：打印三个box的类型、形状和值看看：# print(type(pred_sbbox))# print(type(pred_mbbox))# print(type(pred_lbbox))# 都是<class 'numpy.ndarray'># print(pred_sbbox.shape)# print(pred_mbbox.shape)# print(pred_lbbox.shape)# (1, 68, 68, 3, 6)# (1, 34, 34, 3, 6)# (1, 17, 17, 3, 6)# print(pred_sbbox)# print(pred_mbbox)# print(pred_lbbox)# D·C 191110：（-1，6）表示不知道有多少行，反正你给我整成6列，然后concatenate又把它们仨给叠起来，最终得到无数个6列数组（后面self.num_classes)个数存放的貌似是这个框属于类的概率）pred_bbox = np.concatenate([np.reshape(pred_sbbox, (-1, 5 + self.num_classes)),np.reshape(pred_mbbox, (-1, 5 + self.num_classes)),np.reshape(pred_lbbox, (-1, 5 + self.num_classes))], axis=0)# D·C 191111：打印pred_bbox和它的维度看看：# print(pred_bbox)# print(pred_bbox.shape)# (18207, 6)# D·C 191111：猜测是第一道过滤，过滤掉score_threshold以下的图片，过滤完之后少了好多：# D·C 191115：bboxes维度为[n,6]，前四列是坐标，第五列是得分，第六列是对应类下标bboxes = utils.postprocess_boxes(pred_bbox, (org_h, org_w), self.input_size, self.score_threshold)# D·C 191111：猜测是第二道过滤，过滤掉iou_threshold以下的图片：bboxes = utils.nms(bboxes, self.iou_threshold)return bboxesdef dontla_evaluate_detect(self):# 摄像头个数（在这里设置所需使用摄像头的总个数）cam_num = 6ctx = rs.context()# 连续验证机制# D·C 1911202：创建最大验证次数max_veri_times；创建连续稳定值continuous_stable_value，用于判断设备重置后是否处于稳定状态max_veri_times = 100continuous_stable_value = 10print('\n', end='')print('开始连续验证，连续验证稳定值：{}，最大验证次数：{}：'.format(continuous_stable_value, max_veri_times))continuous_value = 0veri_times = 0while True:devices = ctx.query_devices()connected_cam_num = len(devices)if connected_cam_num == cam_num:continuous_value += 1if continuous_value == continuous_stable_value:breakelse:continuous_value = 0veri_times += 1if veri_times == max_veri_times:print("检测超时，请检查摄像头连接！")sys.exit()print('摄像头个数：{}'.format(connected_cam_num))# 循环reset摄像头# hardware_reset()后是不是应该延迟一段时间？不延迟就会报错print('\n', end='')print('开始初始化摄像头：')for dev in ctx.query_devices():dev.hardware_reset()while len(ctx.query_devices()) != cam_num:time.sleep(0.5)print('摄像头{}初始化成功'.format(dev.get_info(rs.camera_info.serial_number)))# D·C 191202：猜测摄像头重置后的若干秒内，摄像头是不稳定的，这跟访问ctx.query_devices()时设备丢失是否是同一回事，有待考证！# D·C 191202：除此之外，我还怀疑，访问ctx.query_devices()会对设备连接造成影响，在这里，我们尽量减少访问频率。如果没有影响，那还是要等设备稳定再运行。# 连续验证机制# D·C 1911202：创建最大验证次数max_veri_times；创建连续稳定值continuous_stable_value，用于判断设备重置后是否处于稳定状态print('\n', end='')print('开始连续验证，连续验证稳定值：{}，最大验证次数：{}：'.format(continuous_stable_value, max_veri_times))continuous_value = 0veri_times = 0while True:devices = ctx.query_devices()connected_cam_num = len(devices)if connected_cam_num == cam_num:continuous_value += 1if continuous_value == continuous_stable_value:breakelse:continuous_value = 0veri_times += 1if veri_times == max_veri_times:print("检测超时，请检查摄像头连接！")sys.exit()print('摄像头个数：{}'.format(connected_cam_num))# 打印摄像头序列号和接口号并创建需要显示在窗口上的备注信息字符串列表（窗口名）print('\n', end='')cam_id = 0serial_list = []for i in devices:cam_id += 1serial_list.append('camera{}; serials number {}; usb port {}'.format(cam_id, i.get_info(rs.camera_info.serial_number),i.get_info(rs.camera_info.usb_type_descriptor)))print('serial number {}：{}；usb port：{}'.format(cam_id, i.get_info(rs.camera_info.serial_number),i.get_info(rs.camera_info.usb_type_descriptor)))# 配置各个摄像头的基本对象for i in range(connected_cam_num):# D·C 191203：括号里是否有必要加ctx，加了没加好像没多大区别，但不加它又会提示黄色locals()['pipeline' + str(i)] = rs.pipeline(ctx)locals()['config' + str(i)] = rs.config()locals()['serial' + str(i)] = ctx.devices[i].get_info(rs.camera_info.serial_number)locals()['config' + str(i)].enable_device(locals()['serial' + str(i)])locals()['config' + str(i)].enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)locals()['config' + str(i)].enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)locals()['pipeline' + str(i)].start(locals()['config' + str(i)])# 创建对齐对象（深度对齐颜色）locals()['align' + str(i)] = rs.align(rs.stream.color)# 运行流并进行识别print('\n', end='')print('开始识别：')try:# 设置break标志，方便按下按钮跳出循环退出窗口break2 = Falsewhile True:for i in range(connected_cam_num):locals()['frames' + str(i)] = locals()['pipeline' + str(i)].wait_for_frames()# 获取对齐帧集locals()['aligned_frames' + str(i)] = locals()['align' + str(i)].process(locals()['frames' + str(i)])# 获取对齐后的深度帧和彩色帧locals()['aligned_depth_frame' + str(i)] = locals()['aligned_frames' + str(i)].get_depth_frame()locals()['color_frame' + str(i)] = locals()['aligned_frames' + str(i)].get_color_frame()if not locals()['aligned_depth_frame' + str(i)] or not locals()['color_frame' + str(i)]:continue# 获取颜色帧内参locals()['color_profile' + str(i)] = locals()['color_frame' + str(i)].get_profile()locals()['cvsprofile' + str(i)] = rs.video_stream_profile(locals()['color_profile' + str(i)])locals()['color_intrin' + str(i)] = locals()['cvsprofile' + str(i)].get_intrinsics()locals()['color_intrin_part' + str(i)] = [locals()['color_intrin' + str(i)].ppx,locals()['color_intrin' + str(i)].ppy,locals()['color_intrin' + str(i)].fx,locals()['color_intrin' + str(i)].fy]locals()['color_image' + str(i)] = np.asanyarray(locals()['color_frame' + str(i)].get_data())locals()['bboxes_pr' + str(i)] = self.predict(locals()['color_image' + str(i)])locals()['image' + str(i)] = utils.draw_bbox(locals()['color_image' + str(i)],locals()['bboxes_pr' + str(i)],locals()['aligned_depth_frame' + str(i)],locals()['color_intrin_part' + str(i)],show_label=self.show_label)# D·C 191202：本想创建固定比例的大小可调的窗口，发现无法使用，opencv bug？# cv2.namedWindow('{}'.format(serial_list[i]),#                 flags=cv2.WINDOW_NORMAL | cv2.WINDOW_FREERATIO | cv2.WINDOW_GUI_EXPANDED)cv2.imshow('{}'.format(serial_list[i]), locals()['image' + str(i)])key = cv2.waitKey(1)# 如果按下ESC，则跳出循环if key == 27:# 貌似直接用return也行# returnbreak2 = Truebreakif break2:breakfinally:# 大概觉得先关闭窗口再停止流比较靠谱# 销毁所有窗口cv2.destroyAllWindows()print('\n', end='')print('已关闭所有窗口！')# 停止所有流for i in range(connected_cam_num):locals()['pipeline' + str(i)].stop()print('正在停止所有流，请等待数秒至程序稳定结束！')if __name__ == '__main__':YoloTest().dontla_evaluate_detect()print('程序已结束！')

yunyang tensorflow-yolov3 Intel Realsense D435 （并发）使用locals()函数批量配置摄像头运行识别程序并画框（代码记录）（代码示例）相关推荐

yunyang tensorflow-yolov3 Intel Realsense D435 （并发）调用两个摄像头运行识别程序并画框
只是一个测试,测试在并发运行下,同时开启两个摄像头获取视频流并调用识别函数的运行结果,以后在摄像头多的情况下,肯定不能这样,需要批量创建各种对象. 并发,指的是不在多线程的情况下,每个摄像头的视频流送 ...
Tensorflow yolov3 Intel Realsense D435 双摄像头下测试python多线程（假的多线程）self.predict()函数运行时间（191204）
测试代码: # -*- coding: utf-8 -*- """ @File : test-191204-两个摄像头调用多线程识别.py @Time : 2019/12 ...
Tensorflow yolov3 Intel Realsense D435 单摄像头下各模块识别时间测试
文章目录 1.一次循环总耗时 2.从循环开始到self.predict()之前 3.第一次循环主要耗时函数:wait_for_frames()(其实self.predict()函数在第一次循环更耗时) ...
Tensorflow yolov3 Intel Realsense D435 识别时间测试
CPU:i75820k GPU:GTX1080 Ti 内存:32G 测试1:两摄像头并发耗时:0.1476612091064453秒耗时:0.18199372291564941秒耗时:0.140 ...
Tensorflow yolov3 Intel Realsense D435 多进程multiprocessing线程池pool识别时间测试
单摄像头下: 耗时:3.571178674697876秒耗时:0.0682528018951416秒耗时:0.07838797569274902秒耗时:0.08464503288269043秒 ...
Intel Realsense D435 pyrealsense set_option() rs.option 可配置参数翻译
from pyrealsense2\option.py 精度= option.accuracy apd temperature = option.apd_temperature 最高温度= optio ...
Intel Realsense D435运行报错 RuntimeError: Camera not connected! dev.hardware_reset()函数需加睡眠sleep()
解决方案: 参考:Intel Realsense D435报错 RuntimeError: MFCreateDeviceSource(_device_attrs, &_source) retu ...
tensorflow-yolov3 调试Intel Realsense D435摄像头所遇到的问题（USB自动检测并重置机制）hardware_reset() pyusb libusb devcon
文章目录 191126 191127 191128 191129 尝试第二种方案 Devcon 191130 191126 连接6摄像头运行,开始运行正常,能够正常识别,但不小心线动了一下,视频窗口卡 ...
Tensorfow_yolov3 Intel Realsense D435 图像整合（合并）输入GPU计算耗时测试
以前我们六个摄像头轮询依次输入视频帧给识别函数,识别函数放到GPU计算,640×360下每帧识别耗时为60-80ms,轮一圈下来识别就很慢了,约360-480ms 后来我们讨论这个问题时,我说之前我测 ...

yunyang tensorflow-yolov3 Intel Realsense D435 （并发）使用locals()函数批量配置摄像头运行识别程序并画框（代码记录）（代码示例）

文章目录

20191126

20191202-1

20191202-2

yunyang tensorflow-yolov3 Intel Realsense D435 （并发）使用locals()函数批量配置摄像头运行识别程序并画框（代码记录）（代码示例）相关推荐

最新文章

热门文章