win10环境下基于face_recognition搭建自己的人脸识别软件

在win10环境下安装face_recognition，了解face_recognition中api的使用，如人脸截取、人脸矫正、人脸特征提取、人脸关键点提取、人脸mask获取、人脸特征比对等功能。最后构建自己的人脸数据库，使用knn实现人脸识别软件。

1、安装face_recognition

face_recognition库依赖于dlib库，安装dlib库则需要安装cmake，故此安装命令分别为：

pip3 install cmake

特别说明

在安装cmake后，可能会由于python环境路径没有添加到环境变量中，故此需要将程序输出的路径添加到系统环境变量path中

安装好cmake后即可安装dlib和face_recognition

pip3 install dlib
pip3 install face_recognition

2、api使用

2.1、获取图像中人脸的位置

关键函数：face_recognition.face_locations(img)
传入img图像（whc格式的np数组），返回图像中多个人脸的坐标，坐标格式为y0, x1, y1, x0

from skimage import io
import face_recognition
img=io.imread('D:\深度学习\人脸识别模型\database\zhou1-rui4-fa1.jpg')
face_locations = face_recognition.face_locations(img)
print(face_locations)

输出为

[(81, 373, 236, 218)]

封装为函数测试

import os
import face_recognition
from skimage import io,transform
from PIL import Image, ImageDraw,ImageFont
import matplotlib.pyplot as plt
import matplotlib.patches as patches
##----------------------------
#1、定位图像中的人脸
def dect_crop_face(image):  '''  在一张包含人脸的图片中圈出来人脸  '''  fig,ax = plt.subplots(1,figsize=(8,12))ax.imshow(image)face_locations = face_recognition.face_locations(image)  heads=[]for one in face_locations:     y0, x1, y1, x0=one  rect = patches.Rectangle((x0,y0),x1-x0,y1-y0,linewidth=2,edgecolor='r',facecolor='none')ax.add_patch(rect)x1, y1 =x1+25,y1+25cropped = image[y0:y1,x0:x1]if True:fig,ax2 = plt.subplots(1)#plt.title('dect_face crop')ax2.imshow(cropped)heads.append(cropped)#plt.figure(figsize=(8,12))plt.show()return face_locations,heads
face_locations,heads=dect_crop_face(img)

测试效果为

将识别出的多个人脸保存为图像

将识别出的多个人脸裁剪出来，并缩放到指定大小（200, 200），然后保存为图像

#2、裁剪出识别到的人脸
#扩大裁剪出人脸的范围
def scale_rect(y0, x1, y1, x0,width=None,height=None,rate=0):if rate>0:sx,sy=x1-x0,y1-y0#x0=0 if int(x0-sx*rate)<0 else int(x0-sx*rate)#y0=0 if int(y0-sy*rate)<0 else int(y0-sy*rate)x1=width if int(x1+sx*rate)>width else int(x1+sx*rate)y1=height if int(y1+sy*rate)>height else int(y1+sy*rate)return y0, x1, y1, x0
def crop_face(image,face_locations=None,show=None):  '''  图片中人脸截图保存  '''  if not face_locations:face_locations = face_recognition.face_locations(image)  #(top, right, bottom, left)  head=[]for i in range(len(face_locations)):  y0, x1, y1, x0 = face_locations[i]  y0, x1, y1, x0 =scale_rect(y0, x1, y1, x0,width=image.shape[1],height=image.shape[0])cropped = image[y0:y1,x0:x1] # (left, upper, right, lower)  左上角  右下角  head.append(cropped)cropped=transform.resize(cropped, (200, 200))if show:plt.figure(figsize=(7,7))#plt.title('face crop')plt.imshow(cropped)plt.show()io.imsave('face_tmp.png',cropped)return head

2.2 提取人脸特征进行比对

提取人脸特征，具体流程为：先定位人脸，然后编码
通过face_encoding函数可以将图像中的多个人脸编码为多个128维的向量

#3、将图片中的每张人脸编码成一个128维长度的向量
def face_encoding(image):face_locations = face_recognition.face_locations(image)  #(top, right, bottom, left)   face_encodings = face_recognition.face_encodings(image, face_locations)  #将单个人脸数据转化为一个128维的向量  return face_encodings

人脸比对，具体思路为，根据人脸特征向量的距离判断其相似性，如果距离小于阈值则判断为相似。判断人脸特征距离的函数为face_distance，具体封装使用如下所示

#4、对比两张脸是否一致 image1：要对比的图像，image2：unknow的图像
def face_feature_contrast(known_encoding,unknown_encoding,rate=0.6):  '''  给定两个人脸特征，判断是否是同一个人  '''  #face_recognition.compare_facesrate = face_recognition.face_distance([known_encoding], unknown_encoding)  boolres=True if rate<0.6 else Falsereturn 1-rate,boolres
def face_contrast(image1,image2,rate=0.6):  '''  给定两张图片，判断是否是同一个人  '''  known_encoding = face_encoding(image1)unknown_encoding = face_encoding(image2) return face_feature_contrast(known_encoding,unknown_encoding,rate=0.6)

2.3 脸部关键点识别、标注

通过face_recognition.face_landmarks(image) 函数获取人脸关键点坐标，函数返回一个json数组，数组中元素的keys包含[‘chin’, ‘left_eyebrow’, ‘right_eyebrow’, ‘nose_bridge’, ‘nose_tip’, ‘left_eye’, ‘right_eye’, ‘top_lip’, ‘bottom_lip’]

def face_key_point(image,show=False):  face_landmarks_list = face_recognition.face_landmarks(image)  #print("I found {} face(s) in this photograph.".format(len(face_landmarks_list)))# plt.figure(figsize=(8,12))pil_image = Image.fromarray(image)  draw = ImageDraw.Draw(pil_image)  font = ImageFont.truetype("consola.ttf", 12, encoding="unic")#设置字体for face_landmarks in face_landmarks_list:  point_nums=0for facial_feature in face_landmarks.keys():  #draw.line(face_landmarks[facial_feature], width=1,fill='blue') for point in face_landmarks[facial_feature]:#draw.point(point,fill='blue') x,y=pointr=1draw.ellipse((x-r, y-r, x+r, y+r), 'red', 'red') point_nums+=1#draw.text(point, str(point_nums), 'red',font)print("The {} feature: {}".format(facial_feature, face_landmarks[facial_feature])) if show:plt.imshow(pil_image)plt.show()return pil_image,face_landmarks
pil_image,face_landmarks=face_key_point(img,show=True)

The chin feature: [(218, 140), (222, 159), (225, 176), (231, 194), (239, 209), (254, 223), (272, 234), (293, 241), (315, 242), (334, 235), (347, 221), (357, 203), (364, 185), (366, 166), (364, 148), (362, 130), (360, 113)] The left_eyebrow feature: [(240, 131), (247, 119), (260, 113), (274, 111), (288, 114)] The right_eyebrow feature: [(306, 111), (317, 104), (330, 100), (342, 103), (350, 111)] The nose_bridge feature: [(299, 125), (302, 138), (306, 150), (309, 163)] The nose_tip feature: [(291, 170), (300, 172), (310, 173), (317, 169), (324, 164)] The left_eye feature: [(256, 137), (263, 131), (271, 130), (279, 132), (272, 134), (264, 136)] The right_eye feature: [(316, 125), (323, 120), (331, 119), (339, 121), (333, 123), (324, 124)] The top_lip feature: [(270, 189), (285, 184), (299, 183), (310, 183), (319, 180), (331, 177), (342, 176), (339, 178), (320, 183), (310, 187), (300, 186), (273, 189)] The bottom_lip feature: [(342, 176), (333, 193), (322, 202), (312, 205), (301, 206), (286, 202), (270, 189), (273, 189), (300, 199), (310, 199), (320, 196), (339, 178)] ![在这里插入图片描述](https://img-blog.csdnimg.cn/20470b6b42644b089f2ad81323eb29cc.png)

2.4 获取人脸的mask

使用dlib加载模型，使用模型detector预测出人脸的位置，然后使用predictor 预测出每一个人脸的mask

import face_recognition_models
import dlib
import cv2
import numpy as np
#获取原始人脸特征点
def get_landmarks(image):predictor_68_path = face_recognition_models.pose_predictor_model_location()#detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor(predictor_68_path)img_gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)rects = detector(img_gray, 0)for i in range(len(rects)):landmarks = np.matrix([[p.x, p.y] for p in predictor(image, rects[i]).parts()])return landmarks
#根据特征点提取人脸的掩码
def get_image_hull_mask(image,show=False, ie_polys=None):image_shape=image.shapeimage_landmarks=get_landmarks(image)# get the mask of the imageif image_landmarks.shape[0] != 68:raise Exception('get_image_hull_mask works only with 68 landmarks')int_lmrks = np.array(image_landmarks, dtype=np.int)#hull_mask = np.zeros(image_shape[0:2]+(1,), dtype=np.float32)hull_mask = np.full(image_shape[0:2] + (1,), 0, dtype=np.float32)cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[0:9],int_lmrks[17:18]))), (1,))cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[8:17],int_lmrks[26:27]))), (1,))cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[17:20],int_lmrks[8:9]))), (1,))cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[24:27],int_lmrks[8:9]))), (1,))cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[19:25],int_lmrks[8:9],))), (1,))cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[17:22],int_lmrks[27:28],int_lmrks[31:36],int_lmrks[8:9]))), (1,))cv2.fillConvexPoly(hull_mask, cv2.convexHull(np.concatenate((int_lmrks[22:27],int_lmrks[27:28],int_lmrks[31:36],int_lmrks[8:9]))), (1,))# nosecv2.fillConvexPoly(hull_mask, cv2.convexHull(int_lmrks[27:36]), (1,))image[hull_mask[:,:,0]==0]=0if ie_polys is not None:ie_polys.overlay_mask(hull_mask)plt.imshow(image)return hull_mask
hull_mask=get_image_hull_mask(img,True)

根据提取出的人脸进行化妆

#6、根据第5部分脸部关键点识别进行自动化妆
def face_makeup(image,show=False):  face_landmarks_list = face_recognition.face_landmarks(image)  pil_image = Image.fromarray(image)  for face_landmarks in face_landmarks_list:  demo = ImageDraw.Draw(pil_image, 'RGBA')  demo.polygon(face_landmarks['left_eyebrow'], fill=(68, 54, 39, 128))  demo.polygon(face_landmarks['right_eyebrow'], fill=(68, 54, 39, 128)) demo.line(face_landmarks['left_eyebrow'], fill=(68, 54, 39, 150), width=2)  demo.line(face_landmarks['right_eyebrow'], fill=(68, 54, 39, 150), width=2)demo.polygon(face_landmarks['top_lip'], fill=(150, 0, 0, 128))  demo.polygon(face_landmarks['bottom_lip'], fill=(150, 0, 0, 128))  demo.line(face_landmarks['top_lip'], fill=(150, 0, 0, 64), width=2)  demo.line(face_landmarks['bottom_lip'], fill=(150, 0, 0, 64), width=2)  demo.polygon(face_landmarks['left_eye'], fill=(255, 255, 255, 30))  demo.polygon(face_landmarks['right_eye'], fill=(255, 255, 255, 30))  demo.line(face_landmarks['left_eye'] + [face_landmarks['left_eye'][0]], fill=(0, 0, 0, 110), width=2)  demo.line(face_landmarks['right_eye'] + [face_landmarks['right_eye'][0]], fill=(0, 0, 0, 110), width=2) if show:plt.imshow(image)plt.imshow(pil_image)plt.show()return pil_image
pil_image=face_makeup(img,True)

2.5 裁剪人脸并矫正

由于通过face_recognition.face_locations获取的人脸存在一定倾斜，故使用dlib.get_face_chips获取矫正后的人脸切片。

import dlib
import face_recognition_models
from skimage import io
import matplotlib.pyplot as plt
predictor_5_path = face_recognition_models.pose_predictor_five_point_model_location()#
predictor_68_path = face_recognition_models.pose_predictor_model_location()#
print(predictor_68_path)
detector = dlib.get_frontal_face_detector()
sp = dlib.shape_predictor(predictor_68_path)img=io.imread('D:\深度学习\人脸识别模型\database\zhou1-rui4-fa1.jpg')
plt.imshow(img)
plt.show()
if True:# 检测脸部dets = detector(img, 1)print("Number of faces detected: {}".format(len(dets)))    num_faces = len(dets)# 查找脸部位置faces = dlib.full_object_detections()for detection in dets:faces.append(sp(img, detection))    images = dlib.get_face_chips(img, faces, size=320)for image in images:plt.imshow(image)plt.show()#image = dlib.get_face_chip(img, faces[0], size=320)

3、搭建自己的人脸识别系统

基于face_recognition的人脸特征提取，和knn分类算法，我们可以搭建自己的人脸识别系统。这里采用各类明星的公开数据集进行测试

3.1 数据集准备

目标采集到以下明星的照片，存入database目录，作为标准人脸照片。可以针对每一个明星采集多个样本，那则需要将样本数据存入同一个文件夹下。这里每个样本数据中只允许存在一个人脸。

针对上述人脸基础数据，再次采集第二张图片作为测试数据，数据存入database-test目录。

3.2 构建人脸数据集

遍历传入的路径，判断是单个样本还是多个样本，并提取每一个样本的第一个人脸（如果存在多个人脸，则跳过该样本），将人脸图像存入faces数组，人脸特征存入encodings 数组，人脸标签存入labels 数组（这里采用文件名或者文件夹名作为人脸的标签），将存在多个人脸的数据存入unknown数组。其中train_dir 存储的是人脸的文件名，也就是姓名。


from matplotlib import pyplot as plt
import face_recognition
from sklearn import svm
import os
import numpy as np
def img_encoding(path):faces=[]encodings = []labels = []# Training directorytrain_dir = os.listdir(path)unknown=[]# Loop through each person in the training directoryfor person in train_dir:if os.path.isdir(path + person):#目录文件夹，多样本模式pix = os.listdir(path + person)for person_img in pix:# Get the face encodings for the face in each image fileface = face_recognition.load_image_file(path + person + "/" + person_img)face_bounding_boxes = face_recognition.face_locations(face)#If training image contains exactly one faceif len(face_bounding_boxes) == 1:faces.append(face)face_enc = face_recognition.face_encodings(face)[0]encodings.append(face_enc)labels.append(train_dir.index(person))else:print(person+"/"+person_img+" was skipped and can't be used for training")unknown.append(person + "/" + person_img)else:#文件样本单一模式face = face_recognition.load_image_file(path + person)face_bounding_boxes = face_recognition.face_locations(face)if len(face_bounding_boxes) == 1:faces.append(face)face_enc = face_recognition.face_encodings(face)[0]encodings.append(face_enc)labels.append(train_dir.index(person))else:print(person+"/"+person_img+" was skipped and can't be used for training")unknown.append(person + "/" + person_img)print('data of % had load!'%path)return np.array(train_dir),np.array(faces),np.array(encodings),np.array(labels),np.array(unknown)

3.3 测试人脸数据

先调用img_encoding函数加载出人脸数据（filenams、人脸图像数据、人脸特征编码、人脸标签数据和unknown数据），然后训练knn，最后使用knn来进行人脸分类。

def myimshows(imgs, titles=False, fname="test.jpg", size=6):lens = len(imgs)fig = plt.figure(figsize=(size * lens,size))if titles == False:titles="0123456789"for i in range(1, lens + 1):cols = 100 + lens * 10 + iplt.xticks(())plt.yticks(())plt.subplot(cols)if len(imgs[i - 1].shape) == 2:plt.imshow(imgs[i - 1], cmap='Reds')else:plt.imshow(imgs[i - 1])plt.title(titles[i - 1])plt.xticks(())plt.yticks(())#plt.savefig(fname, bbox_inches='tight')plt.show()# Create and train the SVC classifier
#clf = svm.SVC(gamma='auto')
#clf.fit(encodings,names)
train_dir,faces,encodings,names,unknown=img_encoding('D:\\深度学习\\人脸识别模型\\database\\')
knn_clf = neighbors.KNeighborsClassifier(n_neighbors=int(round(math.sqrt(encodings[0].shape[0]))-2), algorithm='ball_tree', weights='distance')
knn_clf.fit(encodings, names)test_dir='D:\\深度学习\\人脸识别模型\\databse-test\\'
test_list=os.listdir(test_dir)
correct=0
#遍历图片
for file in test_list:test_image = face_recognition.load_image_file(test_dir+file)face_locations = face_recognition.face_locations(test_image)no = len(face_locations)print(file," recognition:=> ")#遍历人脸for i in range(no):test_image_enc = face_recognition.face_encodings(test_image)[i]cls_p = knn_clf.predict([test_image_enc])[0]#以文件名判断是否预测正确if file.split('.')[0]==train_dir[cls_p].split('.')[0]:correct+=1else:print("---recognition:error---")print(train_dir[cls_p])myimshows([test_image,faces[cls_p]],['test image','predict result'])
print(correct/len(test_list))
print("------------end-----------")

上述程序遍历了databse-test下所有的图片文件，并截取出图像中的每一个人脸图片使用knn预测其类型，然后调用myimshows进行绘图展示