文章目录

1.字符分割识别
2.车牌识别矫正
- 2.1 车牌识别项目安装
- 2.2 车牌矫正的方法
3.移动物检测
- 3.1 帧间差分法
- 3.2 相机捕捉照片
- 3.3 MindVision品牌的相机
- 3.4 无品牌相机，大多数有相机的电脑
4.Caffe_SSD三字码识别
- 4.1 check List
- 4.2 正式安装
5.ckpt转pb文件（fasterrcnn）
6.人脸检测识别
- 6.1 MTCNN（人脸检测）
- 6.2 FaceNet（人脸识别）

1.字符分割识别

import cv2
#1111111111111111、读取图像，并把图像转换为灰度图像并显示
img = cv2.imread("D:/xunlei/2.png")  # 读取图片
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  # 转换了灰度化
cv2.imshow('gray', img_gray)  # 显示图片
cv2.waitKey(0)#1111111111111112、将灰度图像二值化，设定阈值是100
img_thre = img_gray
cv2.threshold(img_gray, 100, 255, cv2.THRESH_BINARY_INV, img_thre)
cv2.imshow('threshold', img_thre)
cv2.waitKey(0)#111111111111111113、保存黑白图片
cv2.imwrite('thre_res.png', img_thre)#11111111111111114、分割字符
white = []  # 记录每一列的白色像素总和
black = []  # ..........黑色.......
height = img_thre.shape[0]
width = img_thre.shape[1]
white_max = 0
black_max = 0
# 计算每一列的黑白色像素总和
for i in range(width):s = 0  # 这一列白色总数t = 0  # 这一列黑色总数for j in range(height):if img_thre[j][i] == 255:s += 1if img_thre[j][i] == 0:t += 1white_max = max(white_max, s)black_max = max(black_max, t)white.append(s)black.append(t)print(s)print(t)
arg = False  # False表示白底黑字；True表示黑底白字
if black_max > white_max:arg = True# 分割图像
def find_end(start_):end_ = start_ + 1for m in range(start_ + 1, width - 1):if (black[m] if arg else white[m]) > (0.95 * black_max if arg else 0.95 * white_max):  # 0.95这个参数请多调整，对应下面的0.05end_ = mbreakreturn end_n = 1
start = 1
end = 2
while n < width - 2:n += 1if (white[n] if arg else black[n]) > (0.05 * white_max if arg else 0.05 * black_max):# 上面这些判断用来辨别是白底黑字还是黑底白字# 0.05这个参数请多调整，对应上面的0.95start = nend = find_end(start)n = endif end - start > 5:cj = img_thre[1:height, start:end]cv2.imshow('caijian', cj)cv2.waitKey(0)

2.车牌识别矫正

2.1 车牌识别项目安装

1.git clone 最新的文档

cd /opt/deploy
git clone  https://github.com/sergiomsilva/alpr-unconstrained.git

2.下载预训练模型

 cd alpr-unconstrained/bash get-networks.sh

3.删除项目自带 Darknet 替换为官方自带darknet

rm -rf darknet
git clone https://github.com/pjreddie/darknet

4.更改官方darknet 支持gpu 并且 make 由于我的cudn 驱动装在了默认位置所以我只需要更改三处即可

cd darknet/
vim Makefile将第1、2行的  已支持GPU
GPU=0
CUDNN=0
修改成：
GPU=1
CUDNN=1将24行的  支持 cudnn
NVCC=nvcc
修改成：
NVCC=/usr/local/cuda/bin/nvcc:wq

5.编译

make all -j6 根据自己核数调整

6.重新进入主目录

cd /opt/alpr-unconstrained
cp -R data/* darknet/data/

7.更改names文件

vim data/ocr/ocr-net.names 0
1
2
3
4
5
6
7
8
9
A
B
C
D
E
F
G
H
I
J
K
L
M
N
P
Q
R
S
T
U
V
W
X
Y
Z
京
沪
津
渝
冀
晋
蒙
辽
吉
黑
苏
浙
皖
闽
赣
鲁
豫
鄂
湘
粤
桂
琼
川
贵
云
藏
陕
甘
青
宁
新

8.修改对应的类训练文件所在的位置

vim data/ocr/ocr-net.dataclasses=66
names=data/ocr/ocr-net.names
train=data/ocr/train.txt
backup=data/ocr/output

9.创建output 目录

mkdir -p data/ocr/output

10.修改CFG网络层的层数和训练参数

cp /opt/deploy/darknet/cfg/yolov3.cfg data/ocr/ocr-net.cfgvim data/ocr/ocr-net.cfg 据自己GPU 和内存来指定 cfg 部分训练的时候将第3、4行的  已支持GPU
batch=64
subdivisions=4[net]
# Testing
# batch=1
# subdivisions=1
# Trainingbatch=64subdivisions=8......[convolutional]
size=1
stride=1
pad=1
filters=33###75activation=linear[yolo]
mask = 6,7,8
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=6###20
num=9
jitter=.3
ignore_thresh = .5
truth_thresh = 1
random=0###1......[convolutional]
size=1
stride=1
pad=1
filters=33###75
activation=linear[yolo]
mask = 3,4,5
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=6###20
num=9
jitter=.3
ignore_thresh = .5
truth_thresh = 1
random=0###1......[convolutional]
size=1
stride=1
pad=1
filters=33###75
activation=linear[yolo]
mask = 0,1,2
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=6###20
num=9
jitter=.3
ignore_thresh = .5
truth_thresh = 1
random=0###1
————————————
filters数目是怎么计算的：3x(classes数目+5)，和聚类数目分布有关，论文中有说明；
比如说我有66类  那么 3* （66+5）=213

2.2 车牌矫正的方法

def findContoursAndDrawBoundingBox(image_rgb):
#部分源代码(我加了点注释) 参考：https://github.com/zeusees/HyperLPRline_upper  = [];line_lower = []; line_experiment = []grouped_rects = []gray_image = cv2.cvtColor(image_rgb,cv2.COLOR_BGR2GRAY)# for k in np.linspace(-1.5, -0.2,10):for k in np.linspace(-50, 0, 15): # thresh_niblack = threshold_niblack(gray_image, window_size=21, k=k)# binary_niblack = gray_image > thresh_niblack# binary_niblack = binary_niblack.astype(np.uint8) * 255# 当一幅图像上的不同部分具有不同亮度时,我们需要采用自适应阈值.此时的阈值是根据图像上的每一个小区域计算与其# 对应的阈值.因此,在同一幅图像上的不同区域采用的是不同的阈值,从而使我们能在亮度不同的情况下得到更好的结果."""Args:- src, 原图像,应该是灰度图-  x, 指当像素值高于(有时是低于)阈值时应该被赋予新的像素值, 255是白色- adaptive_method, CV_ADAPTIVE_THRESH_MEAN_C 或 CV_ADAPTIVE_THRESH_GAUSSIAN_C- threshold_type: 指取阈值类型. CV_THRESH_BINARY, 二进制阈值化. CV_THRESH_BINARY_INV, 反二进制阈值化- block_size: 用来计算阈值的像素邻域大小(块大小):3,5,7,...- param1: 指与方法有关的参数.对方法CV_ADAPTIVE_THRESH_MEAN_C和CV_ADAPTIVE_THRESH_GAUSSIAN_C,它是一个从均值或加权均值提取的常数,尽管它可以是负数。. 对方法 CV_ADAPTIVE_THRESH_MEAN_C，先求出块中的均值，再减掉param1。. 对方法 CV_ADAPTIVE_THRESH_GAUSSIAN_C ，先求出块中的加权和(gaussian)， 再减掉param1。"""binary_niblack = cv2.adaptiveThreshold(gray_image,255,cv2.ADAPTIVE_THRESH_MEAN_C,cv2.THRESH_BINARY,17,k) #邻域大小17是不是太大了??#cv2.imshow("image1",binary_niblack)#cv2.waitKey(0)#imagex, contours, hierarchy = cv2.findContours(binary_niblack.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)contours, hierarchy = cv2.findContours(binary_niblack.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)  # modified by bigzfor contour in contours:#用一个最小的矩形,把找到的形状包起来bdbox = cv2.boundingRect(contour)if (bdbox[3]/float(bdbox[2])>0.7 and bdbox[3]*bdbox[2]>100 and bdbox[3]*bdbox[2]<1200) or (bdbox[3]/float(bdbox[2])>3 and bdbox[3]*bdbox[2]<100):# cv2.rectangle(rgb,(bdbox[0],bdbox[1]),(bdbox[0]+bdbox[2],bdbox[1]+bdbox[3]),(255,0,0),1)line_upper.append([bdbox[0],bdbox[1]])line_lower.append([bdbox[0]+bdbox[2],bdbox[1]+bdbox[3]])line_experiment.append([bdbox[0],bdbox[1]])line_experiment.append([bdbox[0]+bdbox[2],bdbox[1]+bdbox[3]])# grouped_rects.append(bdbox)"""想为图像周围建一个边使用訪函数,这经常在卷积运算或0填充时被用到Args:- src: 输入图像- top,bottom,left,right 对应边界的像素数目- borderType: 要添加哪种类型的边界- BORDER_CONSTANT #边缘填充用固定像素值,比如填充黑边,就用0,白边255- BORDER_REPLICATE #用原始图像相应的边缘的像素去做填充,看起来有一种把图像边缘"拉糊了"的效果"""rgb = cv2.copyMakeBorder(image_rgb,30,30,0,0,cv2.BORDER_REPLICATE)leftyA, rightyA = fitLine_ransac(np.array(line_lower),3)rows,cols = rgb.shape[:2]# rgb = cv2.line(rgb, (cols - 1, rightyA), (0, leftyA), (0, 0, 255), 1,cv2.LINE_AA)leftyB, rightyB = fitLine_ransac(np.array(line_upper),-3)rows,cols = rgb.shape[:2]# rgb = cv2.line(rgb, (cols - 1, rightyB), (0, leftyB), (0,255, 0), 1,cv2.LINE_AA)pts_map1  = np.float32([[cols - 1, rightyA], [0, leftyA],[cols - 1, rightyB], [0, leftyB]])pts_map2 = np.float32([[136,36],[0,36],[136,0],[0,0]])mat = cv2.getPerspectiveTransform(pts_map1,pts_map2)image = cv2.warpPerspective(rgb,mat,(136,36),flags=cv2.INTER_CUBIC)#校正角度#cv2.imshow("校正前",image)#cv2.waitKey(0)image,M = deskew.fastDeskew(image)#cv2.imshow("校正后",image)#cv2.waitKey(0)return image

3.移动物检测

3.1 帧间差分法

1.使用opencv展示图像

import cv2
def cv2_display(image_ndarray):windowName = 'display'cv2.imshow(windowName, image_ndarray)# 按Esc键或者q键可以退出循环pressKey = cv2.waitKey(0)if 27 == pressKey or ord('q') == pressKey:cv2.destroyAllWindows()

2.加载2张图片文件为图像数据

image_ndarray_1 = cv2.imread('../resources/1.jpg')
image_ndarray_2 = cv2.imread('../resources/2.jpg')

2.1 展示原始图像数据

# 按Esc键或者q键可以退出cv2显示窗口
cv2_display(image_ndarray_1)
cv2_display(image_ndarray_2)

3.图像处理

def get_processedImage(image_ndarray):# 对拍摄图像进行图像处理，先转灰度图，再进行高斯滤波。image_ndarray_1 = cv2.cvtColor(image_ndarray, cv2.COLOR_BGR2GRAY)# 用高斯滤波对图像处理，避免亮度、震动等参数微小变化影响效果filter_size = 7image_ndarray_2 = cv2.GaussianBlur(image_ndarray_1, (filter_size, filter_size), 0)return image_ndarray_2

image_ndarray_1_2 = get_processedImage(image_ndarray_1)
image_ndarray_2_2 = get_processedImage(image_ndarray_2)

3.1 展示处理后的图像数据

cv2_display(image_ndarray_1_2) # 展示处理后的图像数据
cv2_display(image_ndarray_2_2)

4.图像相减

absdiff_ndarray = cv2.absdiff(image_ndarray_1_2, image_ndarray_2_2)
cv2_display(absdiff_ndarray) # 展示相减后的图像数据

5.图像二值化

result_1 = cv2.threshold(absdiff_ndarray, 25, 255, cv2.THRESH_BINARY)
type(result_1) # 输出tuple
len(result_1) # 输出2
type(result_1[0]) # 输出float
result_1[0]  # 输出25.0
type(result_1[1])  # 输出numpy.ndarray
result_1[1].shape  # 输出(960, 1280)
cv2_display(result_1[1])
threshhold_ndarray = result_1[1]

5.1 显示二值化后的图像

cv2_display(threshhold_ndarray)

6.获取轮廓列表，并做响应操作

contour_list = cv2.findContours(threshhold_ndarray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[0]

import datetime image_ndarray_3 = image_ndarray_2.copy()
for contour in contour_list:# 对于较小矩形区域，选择忽略if cv2.contourArea(contour) < 2000:continueelse:x1, y1, w, h = cv2.boundingRect(contour)x2, y2 = x1 + w, y1 + hleftTop_coordinate = x1, y1rightBottom_coordinate = x2, y2bgr_color = (0, 0, 255)thickness = 2cv2.rectangle(image_ndarray_3, leftTop_coordinate, rightBottom_coordinate, bgr_color, thickness)text = "Find motion object! x=%d, y=%d" %(x1, y1)print(text)
cv2.putText(image_ndarray_3, text, (10, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, bgr_color, thickness)
time_string = datetime.datetime.now().strftime("%A %d %B %Y %I:%M:%S%p")
_ = cv2.putText(image_ndarray_3, time_string, (10, 100), cv2.FONT_HERSHEY_SIMPLEX, 1, bgr_color, thickness)

6.1 根据轮廓绘制方框后，显示图像

cv2_display(image_ndarray_3)

3.2 相机捕捉照片

# 导入常用的库
import os
import cv2
# 导入MindVision品牌的相机库mvsdk
import mvsdk# 相机持续拍摄并保存照片
def get_capturedImage(imageFilePath=None):if not imageFilePath:imageFilePath = '../resources/temp.jpg'device_list = mvsdk.CameraEnumerateDevice()if len(device_list) == 0:return Nonedevice_info = device_list[0]cameraIndex = mvsdk.CameraInit(device_info, -1, -1)capability = mvsdk.CameraGetCapability(cameraIndex)mvsdk.CameraSetTriggerMode(cameraIndex, 0)# 加载相机配置文件configFilePath = '../resources/camera.Config'assert os.path.exists(configFilePath), 'please check if exists %s'%configFilePathmvsdk.CameraReadParameterFromFile(cameraIndex, configFilePath)# 获取相机拍摄照片的预处理mvsdk.CameraPlay(cameraIndex)FrameBufferSize = capability.sResolutionRange.iWidthMax * capability.sResolutionRange.iHeightMax * 3FrameBuffer_address = mvsdk.CameraAlignMalloc(FrameBufferSize, 16)RawData, FrameHead = mvsdk.CameraGetImageBuffer(cameraIndex, 2000)mvsdk.CameraImageProcess(cameraIndex, RawData, FrameBuffer_address, FrameHead)mvsdk.CameraReleaseImageBuffer(cameraIndex, RawData)# 把文件路径转换为绝对文件路径imageFilePath_1 = os.path.abspath(imageFilePath)status = mvsdk.CameraSaveImage(cameraIndex, imageFilePath_1, FrameBuffer_address, FrameHead, mvsdk.FILE_JPG, 100)if status != mvsdk.CAMERA_STATUS_SUCCESS:print('ID为%d的相机拍摄并保存照片失败!!!')is_successful = Falseelse:print('ID为%d的相机保存照片至路径：%s' %(cameraIndex, imageFilePath_1))is_successful = True# 关闭相机、释放帧缓存mvsdk.CameraUnInit(cameraIndex)mvsdk.CameraAlignFree(FrameBuffer_address)

3.3 MindVision品牌的相机

import os
import cv2
# 导入MindVision品牌的相机库mvsdk
import mvsdk# 相机持续拍摄并保存照片
# 如果没有使用MindVision品牌相机，需要修改此函数内容
def get_capturedImage(cameraIndex):imageFilePath = '../resources/temp.jpg'capability = mvsdk.CameraGetCapability(cameraIndex)mvsdk.CameraSetTriggerMode(cameraIndex, 0)# 加载相机配置文件configFilePath = '../resources/camera.Config'assert os.path.exists(configFilePath), 'please check if exists %s'%configFilePathmvsdk.CameraReadParameterFromFile(cameraIndex, configFilePath)# 获取相机拍摄照片的预处理mvsdk.CameraPlay(cameraIndex)frameBufferSize = capability.sResolutionRange.iWidthMax * capability.sResolutionRange.iHeightMax * 3frameBufferAddress = mvsdk.CameraAlignMalloc(frameBufferSize, 16)rawData, frameHead = mvsdk.CameraGetImageBuffer(cameraIndex, 2000)mvsdk.CameraImageProcess(cameraIndex, rawData, frameBufferAddress, frameHead)mvsdk.CameraReleaseImageBuffer(cameraIndex, rawData)# 把文件路径转换为绝对文件路径imageFilePath_1 = os.path.abspath(imageFilePath)status = mvsdk.CameraSaveImage(cameraIndex, imageFilePath_1, frameBufferAddress, frameHead, mvsdk.FILE_JPG, 100)if status != mvsdk.CAMERA_STATUS_SUCCESS:print('ID为%d的相机拍摄并保存照片失败!!!')is_successful = Falseelse:is_successful = Trueimage_ndarray = cv2.imread(imageFilePath) if is_successful else Nonereturn image_ndarray# 对拍摄图像进行图像处理，先转灰度图，再进行高斯滤波。
def get_processedImage(image_ndarray):image_ndarray_1 = cv2.cvtColor(image_ndarray, cv2.COLOR_BGR2GRAY)# 用高斯滤波对图像处理，避免亮度、震动等参数微小变化影响效果filter_size = 15image_ndarray_2 = cv2.GaussianBlur(image_ndarray_1, (filter_size, filter_size), 0)return image_ndarray_2# 获取表示当前时间的字符串
import time
def get_timeString():now_timestamp = time.time()now_structTime = time.localtime(now_timestamp)timeString_pattern = '%Y %m %d %H:%M:%S'now_timeString = time.strftime(timeString_pattern, now_structTime)return now_timeString# 根据两张图片的不同，在第2张图上绘制不同位置的方框、日期时间
def get_drawedDetectedImage(first_image_ndarray, second_image_ndarray):if second_image_ndarray is None or first_image_ndarray is None:return Nonefirst_image_ndarray_2 = get_processedImage(first_image_ndarray)second_image_ndarray_2 = get_processedImage(second_image_ndarray)# cv2.absdiff表示计算2个图像差值的绝对值absdiff_ndarray = cv2.absdiff(first_image_ndarray_2, second_image_ndarray_2)# cv2.threshold表示设定阈值做图像二值化threshold_ndarray = cv2.threshold(absdiff_ndarray, 25, 255, cv2.THRESH_BINARY)[1]# cv2.dilate表示图像膨胀dilate_ndarray = cv2.dilate(threshold_ndarray, None, iterations=2)# cv2.findContours表示找出图像中的轮廓contour_list = cv2.findContours(threshold_ndarray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[0]copy_image_ndarray = second_image_ndarray.copy()for contour in contour_list:if cv2.contourArea(contour) < 2000:continueelse:x1, y1, w, h = cv2.boundingRect(contour)x2, y2 = x1 + w, y1 + hleftTop_coordinate = x1, y1rightBottom_coordinate = x2, y2bgr_color = (0, 0, 255)thickness = 2cv2.rectangle(copy_image_ndarray, leftTop_coordinate, rightBottom_coordinate, bgr_color, thickness)time_string = get_timeString()text = '在时刻%s 发现运动物体! x=%d, y=%d' %(time_string, x1, y1)print(text)time_string = get_timeString()bgr_color = (0, 0, 255)thickness = 2cv2.putText(copy_image_ndarray, time_string, (10,50), cv2.FONT_HERSHEY_SIMPLEX, 1, bgr_color, thickness)return copy_image_ndarray# 使用cv2库展示图片
def show_image(image_ndarray):  windowName = 'display'cv2.imshow(windowName, image_ndarray)# 主函数
from sys import exit
if __name__ == '__main__':# 获取相机设备的信息device_list = mvsdk.CameraEnumerateDevice()if len(device_list) == 0:print('没有连接MindVision品牌的相机设备')exit()device_info = device_list[0]cameraIndex = mvsdk.CameraInit(device_info, -1, -1)# 开始用相机监控first_image_ndarray = Nonewhile True:second_image_ndarray = get_capturedImage(cameraIndex)drawed_image_ndarray = get_drawedDetectedImage(first_image_ndarray, second_image_ndarray)if drawed_image_ndarray is not None:show_image(drawed_image_ndarray)# 在展示图片后，等待1秒，接收按键pressKey = cv2.waitKey(1)# 按Esc键或者q键可以退出循环if 27 == pressKey or ord('q') == pressKey:cv2.destroyAllWindows()  break# 随着时间推移，当前帧作为下一帧的前一帧first_image_ndarray = second_image_ndarray# 关闭相机mvsdk.CameraUnInit(cameraIndex)

3.4 无品牌相机，大多数有相机的电脑

# 导入常用的库
import cv2
import time
import os# 对拍摄图像进行图像处理，先转灰度图，再进行高斯滤波。
def get_processedImage(image_ndarray):image_ndarray_1 = cv2.cvtColor(image_ndarray, cv2.COLOR_BGR2GRAY)# 用高斯滤波对图像处理，避免亮度、震动等参数微小变化影响效果filter_size = 21image_ndarray_2 = cv2.GaussianBlur(image_ndarray_1, (filter_size, filter_size), 0)return image_ndarray_2# 获取表示当前时间的字符串
import time
def get_timeString():now_timestamp = time.time()now_structTime = time.localtime(now_timestamp)timeString_pattern = '%Y %m %d %H:%M:%S'now_timeString = time.strftime(timeString_pattern, now_structTime)return now_timeString# 根据两张图片的不同，在第2张图上绘制不同位置的方框、日期时间
def get_drawedDetectedImage(first_image_ndarray, second_image_ndarray):if second_image_ndarray is None or first_image_ndarray is None:return Nonefirst_image_ndarray_2 = get_processedImage(first_image_ndarray)second_image_ndarray_2 = get_processedImage(second_image_ndarray)# cv2.absdiff表示计算2个图像差值的绝对值absdiff_ndarray = cv2.absdiff(first_image_ndarray_2, second_image_ndarray_2)# cv2.threshold表示设定阈值做图像二值化threshold_ndarray = cv2.threshold(absdiff_ndarray, 25, 255, cv2.THRESH_BINARY)[1]# cv2.dilate表示图像膨胀dilate_ndarray = cv2.dilate(threshold_ndarray, None, iterations=2)# cv2.findContours表示找出图像中的轮廓contour_list = cv2.findContours(threshold_ndarray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[0]copy_image_ndarray = second_image_ndarray.copy()height, width, _ = copy_image_ndarray.shapecontour_minArea = int(height * width * 0.001)for contour in contour_list:if cv2.contourArea(contour) < contour_minArea:continueelse:x1, y1, w, h = cv2.boundingRect(contour)x2, y2 = x1 + w, y1 + hleftTop_coordinate = x1, y1rightBottom_coordinate = x2, y2bgr_color = (0, 0, 255)thickness = 2cv2.rectangle(copy_image_ndarray, leftTop_coordinate, rightBottom_coordinate, bgr_color, thickness)time_string = get_timeString()text = '在时刻%s 发现移动物体! x=%d, y=%d' %(time_string, x1, y1)print(text)time_string = get_timeString()bgr_color = (0, 0, 255)thickness = 2cv2.putText(copy_image_ndarray, time_string, (10,50), cv2.FONT_HERSHEY_SIMPLEX, 1, bgr_color, thickness)return copy_image_ndarray# 主函数
from sys import exit
if __name__ == '__main__':cameraIndex = 0# 实例化视频流对象camera = cv2.VideoCapture(cameraIndex)is_successful, first_image_ndarray = camera.read()if not is_successful:print("相机未成功连接，可能原因：1.相机不支持cv2库直接调用;2.如果有多个相机，设置正确的cameraIndex")exit()while True:is_successful, second_image_ndarray = camera.read()windowName = 'cv2_display'drawed_image_ndarray = get_drawedDetectedImage(first_image_ndarray, second_image_ndarray)cv2.imshow(windowName, drawed_image_ndarray)# 在展示图片后，等待1秒，接收按键pressKey = cv2.waitKey(1)# 按Esc键或者q键可以退出循环if 27 == pressKey or ord('q') == pressKey:cv2.destroyAllWindows()  break# 随着时间推移，当前帧作为下一帧的前一帧first_image_ndarray = second_image_ndarray   # 关闭相机camera.release()

如下Opencv4的DNN模块

如下Net，String都是类。训练用深度学习框架（关心各个层的权重），推理用opencv（只需要知道输入输出，并解析输出获得结果）

4.Caffe_SSD三字码识别

4.1 check List

1.检查 CUDA nvcc -V环境是否安装正常如果不正常则去安装 NVIDIA ，CUDA ，CUDNN （版本搭配）

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

2.sudo ldconfig 检查是否有软连接没有生效

4.2 正式安装

1.依赖解决：

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compilersudo apt-get install --no-install-recommends libboost-all-devsudo apt-get install libopenblas-dev liblapack-dev libatlas-base-devsudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-devsudo apt-get install git cmake build-essential

从源代码编译Opencv，进入官网 : http://opencv.org/releases.html , 下载 3.x系列解压到你要安装的位置，命令行进入已解压的文件夹

mkdir build # 创建编译的文件目录cd buildcmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..make -j8  #编译  注意自己的核数

在执行 make -j8 命令编译到 92% 时可能会出现以下错误，是由于opecv3.1与cuda8.0不兼容导致的。解决办法：修改 /opencv-3.1.0/modules/cudalegacy/src/graphcuts.cpp 文件内容，如图：

编译成功后安装：sudo make install
安装完成后通过查看 opencv 版本验证是否安装成功：pkg-config --modversion opencv

2.安装caffe-SSD

git clone https://github.com/weiliu89/caffe.git
cd caffe
git checkout ssd

再次检查依赖

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install libatlas-base-dev python-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

3.修改Makefile.config文件,复制一份的原因是编译 caffe 时需要的是 Makefile.config 文件，而Makefile.config.example 只是caffe 给出的配置文件例子，不能用来编译 caffe。

cp Makefile.config.example Makefile.config
sudo vim Makefile.config

应用 cudnn

将第5行的 # 取消
#USE_CUDNN := 1
修改成：
USE_CUDNN := 1

应用 opencv 版本

将第21行的 # 取消
#OPENCV_VERSION := 3
修改为：
OPENCV_VERSION := 3

使用 python 接口

将第89行的 # 取消
#WITH_PYTHON_LAYER := 1
修改为
WITH_PYTHON_LAYER := 1

修改 python 路径

将 92/93行的 代码修改如下
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
修改为：
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

注释Python2 切换python3，代码修改如下否则会编译Python.h and numpy/arrayobject.h. 出错

将 67/68行的   实际行数稍微有些出路可能 PYTHON_INCLUDE := /usr/include/python2.7 \/usr/lib/python2.7/dist-packages/numpy/core/include
修改为： # PYTHON_INCLUDE := /usr/include/python2.7 \#                 /usr/lib/python2.7/dist-packages/numpy/core/include 将77/78 行的 注释解除  并更新  为Python3.6  实际行数稍微有些出路可能 # PYTHON_LIBRARIES := boost_python3 python3.5m# PYTHON_INCLUDE := /usr/include/python3.5m \#                 /usr/lib/python3.5/dist-packages/numpy/core/includePYTHON_LIBRARIES := boost_python3 python3.6mPYTHON_INCLUDE := /usr/include/python3.6m \/usr/lib/python3.6/dist-packages/numpy/core/include

如果最后提示不支持compute_20,就把这句删掉，最后效果是
nvcc fatal : Unsupported gpu architecture ‘compute_20’
Makefile:588: recipe for target ‘.build_release/cuda/src/caffe/solvers/sgd_solver.o’ failed
make: *** [.build_release/cuda/src/caffe/solvers/sgd_solver.o] Error 1
make : ** * Waiting for unfinished jobs…

建议显卡直接改成如下:
CUDA_ARCH := -gencode arch=compute_30,code=sm_30
-gencode arch=compute_35,code=sm_35
-gencode arch=compute_50,code=sm_50
-gencode arch=compute_52,code=sm_52
-gencode arch=compute_60,code=sm_60
-gencode arch=compute_61,code=sm_61
-gencode arch=compute_61,code=compute_61

然后修改caffe 目录下的 Makefile 文件：注意不是Makefile.config文件

sudo vim Makefile
将第409行 替换为如下
NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
替换为：
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
将：181行替换为如下
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
改为：
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial将：265行 替换如下（我的环境太新除了这个问题【可选项】当你出现了
.build_release/lib/libcaffe.so: undefined reference to `boost::re_detail_106501::put_mem_block(void*)'）LIBRARIES += boost_thread stdc++
改为：
LIBRARIES += boost_thread stdc++ boost_regex

4.下载python 环境依赖包　:
去到caffe根目录中的python目录中运行 …/caffe/python
5.设置系统python环境在末尾添加环境变量

vim ~/.bashrc
export PYTHONPATH="/opt/build/caffe/python"   # 此处为caffe 的rootdir 目录
source ~/.bashrc

6.编译caffe保存开始编译，在 caffe 目录下执行如果出错建议修改完毕使用 make clean 继续Try

make all -j32  代表几核并行编译 请与自己电脑量力而行 后续将不再重复声明哇
make test -j32
make pycaffe
make runtest　-j32

“/ usr / bin / ld：在Caffe编译中找不到-lopenblas”错误
即使在克隆OpenBlas之后包括基本包，并且将在14.04和16中链接相应的库。
apt install liblapack-dev liblapack3 libopenblas-base libopenblas-dev
apt install liblapack-dev liblapack3 libopenblas-base libopenblas-dev
到此安装caffe 结束
7.更改caffe 源码: https://blog.csdn.net/sinat_14916279/article/details/56489601

安装参考：https://blog.csdn.net/lukaslong/article/details/81390276

5.ckpt转pb文件（fasterrcnn）

import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python import pywrap_tensorflowdef freeze_graph(input_checkpoint,output_graph):#指定输出的节点名称,该节点名称必须是原模型中存在的节点。直接用最后输出的节点，可以在tensorboard中查找到，tensorboard只能在linux中使用output_node_names = "SCORE/resnet_v1_101_5/cls_prob/cls_prob/scores,SCORE/resnet_v1_101_5/bbox_pred/BiasAdd/bbox_pred/scores,SCORE/resnet_v1_101_5/cls_pred/cls_pred/scores"saver = tf.train.import_meta_graph(input_checkpoint + '.meta', clear_devices=True) #通过 import_meta_graph 导入模型中的图----1graph = tf.get_default_graph() #获得默认的图input_graph_def = graph.as_graph_def() #返回一个序列化的图代表当前的图with tf.Session() as sess:saver.restore(sess, input_checkpoint) #通过 saver.restore 从模型中恢复图中各个变量的数据----2output_graph_def = graph_util.convert_variables_to_constants(  #通过 graph_util.convert_variables_to_constants 将模型持久化----3sess=sess,input_graph_def=input_graph_def, #等于:sess.graph_defoutput_node_names=output_node_names.split(","))  #如果有多个输出节点，以逗号隔开with tf.gfile.GFile(output_graph, "wb") as f: #保存模型f.write(output_graph_def.SerializeToString()) #序列化输出print("%d ops in the final graph." % len(output_graph_def.node)) #得到当前图有几个操作节点input_checkpoint='./checkpoints/res101_faster_rcnn_iter_70000.ckpt'
out_pb_path='./checkpoints/frozen_model.pb'reader = pywrap_tensorflow.NewCheckpointReader(input_checkpoint)
var_to_shape_map = reader.get_variable_to_shape_map()
for key in var_to_shape_map:  # Print tensor name and valuesprint("tensor_name: ", key)#print(reader.get_tensor(key))freeze_graph(input_checkpoint, out_pb_path)

6.人脸检测识别

6.1 MTCNN（人脸检测）

训练集：http://mmlab.ie.cuhk.edu.hk/archive/CNN_FacePoint.htm

6.2 FaceNet（人脸识别）

CASIA-WebFace数据集：链接：https://pan.baidu.com/s/1dYHmuTOShnXBE0xUfvdWxw 提取码：7bap
训练模型：链接：https://pan.baidu.com/s/1QaMS46EWB80EYLQ8p8a4kw 提取码：cpfa

.dat文件链接：https://pan.baidu.com/s/1TPdqB0afWnIDh4T7d704Fg 提取码：jxit

【Python4】字符分割识别，车牌识别矫正，移动物检测，Caffe_SSD三字码识别，ckpt文件转pb文件，人脸检测与识别相关推荐

java车牌识别字符分割_车牌识别LPR（六）-- 字符分割
第六篇:字符分割在知道了车牌字符的规律之后,可以根据车牌的特点对字符进行分割.一般最容易想到的方法就是根据车牌投影.像素统计特征对车牌图像进行字符分割的方法.是一种最常用的.最基本的.最简单的车牌字 ...
21.失真/低高通/振铃效应/旁瓣泄漏效应/频域滤波/图像深度/频带/线性滤波源码分析 -- OpenCV从零开始到图像（人脸 + 物体）识别系列
本文作者:小嗷微信公众号:aoxiaoji 吹比QQ群:736854977 简书链接:https://www.jianshu.com/u/45da1fbce7d0 本文你会找到以下问题的答案: 失真 ...
python编程胡牌将是什么意思_OpenCV+Python识别车牌和字符分割的实现
本篇文章主要基于python语言和OpenCV库(cv2)进行车牌区域识别和字符分割,开篇之前针对在python中安装opencv的环境这里不做介绍,可以自行安装配置! 车牌号检测需要大致分为四个部分 ...
python车牌字符分割_OpenCV+Python识别车牌和字符分割的实现
本篇文章主要基于python语言和OpenCV库(cv2)进行车牌区域识别和字符分割,开篇之前针对在python中安装opencv的环境这里不做介绍,可以自行安装配置! 车牌号检测需要大致分为四个部分 ...
OpenCV+Python识别车牌和字符分割
本篇文章主要基于python语言和OpenCV库(cv2)进行车牌区域识别和字符分割,开篇之前针对在python中安装opencv的环境这里不做介绍,可以自行安装配置! 车牌号检测需要大致分为四个部分 ...
【车牌识别】基于matlab GUI阈值分割车牌识别（带面板）【含Matlab源码 721期】
⛄一.车牌识别简介 1 车牌图像处理车牌图像处理主要有五个组成部分:图像灰度化.图像二值化.图像边缘检测.图像形态学运算和图像滤波处理.它是车牌识别系统中最根本且最基础的操作,车牌图像处理的好坏情况 ...
【MATLAB图像处理实用案例详解（3）】—— 基于阈值分割的车牌定位识别
目录一.背景意义二.理论基础 2.1 车牌区域分割: 2.2 车牌定位及裁剪三.算法流程 3.1 车牌图像处理 3.1.1 图像灰度化 3.1.2 图像二值化 3.1.3 图像边缘检测 3.1. ...
利用MTCNN和FaceNet实现人脸检测和人脸识别 | CSDN博文精选
作者 | pan_jinquan 来源 | CSDN博文精选 (*点击阅读原文,查看作者更多文章) 人脸检测和人脸识别技术算是目前人工智能方面应用最成熟的技术了.本博客将利用MTCNN和FaceNet ...
teleport 组件的作用_人脸识别综述! 覆盖人脸检测，预处理和特征表示三大核心组件！...
The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances 人脸识别是计算机视觉社区中最基础和历史悠久的 ...

【Python4】字符分割识别，车牌识别矫正，移动物检测，Caffe_SSD三字码识别，ckpt文件转pb文件，人脸检测与识别