前言

生物特征识别（Biometrics）是利用机器学习算法对生物个体，尤其是人类本身的某些特性进行分析和识别的技术。其不仅限于利用人体所固有的指纹、掌纹、面部、虹膜等生理特征，也包括使用步态、打字习惯等个人行为进行身份鉴定。近年来广义的定义也将表情等图片属性归为软生物特征识别（Soft biometrics）一类。生物特征识别应用的领域主要是门禁系统、考勤系统等。

一、人脸识别步骤

如图所示为本人使用XMind ZEN软件绘制的人脸注册、识别流程图，从图中可以清晰的看出人脸识别的步骤。其中，最主要的模块分为3步：人脸检测，人脸对齐，人脸特征提取。
1）人脸检测（face detect）：在图像中首先定位出人脸的位置，然后裁剪(crop)出包含人脸位置的矩形框，一般还会进行填充、缩放到指定尺寸，还可能会对人脸图像进行标准化normalize；
2）人脸对齐（face alignment）：提取人脸关键点坐标，然后使用放射变化或相似变换等进行人脸对齐变换。（人脸对齐的目标就是把所有的人脸图片统一到一个固定的正脸姿态和大小，从而提高模型对人脸姿态变化的鲁棒性。）
3）人脸特征提取（extract feature）:主要使用深度学习等方法提取人脸的特征。

二、SeetaFace2基本介绍

SeetaFace2.0是中科视拓开源的第二代跨平台C++商业人脸识别库，关于其介绍可以参考以下链接，本文将不再赘述。

1、官方Github源码：
2、其他博客介绍

三、seetaFace2人脸注册、识别代码详解

主要参考serach工程中的example.cpp

3.1 人脸注册

3.1.1 人脸检测

通过example.cpp line42中 engine.Register( image )跳转到FaceDetectorPrivate.cpp中Detect函数，该函数输入一张包含人脸的图片，输出SeetaFaceInfoArray格式的包含人脸框（x,y,w,h）和人脸置信度得分score的数据，以及一幅图像中人脸的个数，并将SeetaFaceInfoArray格式的以上数据存入vector中。

SeetaFaceInfoArray FaceDetectorPrivate::Detect( const SeetaImageData &image )
{SeetaFaceInfoArray  ret;ret.size = 0;ret.data = nullptr;Impl *p = ( Impl * )impl_; //??if( !p->IsLegalImage( image ) ){return ret;}// sclae imageseeta::Image img = image;float scale = 1;seeta::Image scaled_img = ScaleImage( img, p->width_limit_, p->height_limit_, &scale );img = scaled_img;img = seeta::color( img );int pad_h = std::min( int( p->max_pad_ratio * img.height() ), p->max_pad_h );int pad_w = std::min( int( p->max_pad_ratio * img.width() ), p->max_pad_w );SeetaImageData img_pad;img_pad.width = img.width() + 2 * pad_w;img_pad.height = img.height() + 2 * pad_h;img_pad.channels = img.channels();img_pad.data = new uint8_t[img_pad.channels * img_pad.height * img_pad.width];p->PadImage( img.data(), img.width(), img.height(), img.channels(), img_pad.data, pad_w, pad_h );auto local_min_face_size = std::max( 12, int( p->min_face_ * scale ) );auto local_max_face_size = p->max_face_;if( local_max_face_size > 0 ) local_max_face_size = std::max( 12, int( p->max_face_ * scale ) );std::vector<Rect> winList;winList = p->SlidingWindow( img, img_pad, p->net_[0], p->class_threshold_[0], local_min_face_size, local_max_face_size );winList = p->NMS( winList, true, p->nms_threshold_[0] );// std::cout << "Stage1 result: " << winList.size() << std::endl;winList = p->RunNet( img_pad, p->net_[1], p->class_threshold_[1], 24, winList );winList = p->NMS( winList, true, p->nms_threshold_[1] );// std::cout << "Stage2 result: " << winList.size() << std::endl;winList = p->RunNet( img_pad, p->net_[2], p->class_threshold_[2], 48, winList );winList = p->NMS( winList, false, p->nms_threshold_[2] );// std::cout << "Stage3 result: " << winList.size() << std::endl;// scale resultfor( auto &info : winList ){info.x -= pad_w;info.y -= pad_h;info.x = int( info.x / scale );info.y = int( info.y / scale );info.width = int( info.width / scale );info.height = int( info.height / scale );}std::vector<Rect> &preList = p->preList_;if( p->stable_ ){for( size_t i = 0; i < winList.size(); i++ ){for( size_t j = 0; j < preList.size(); j++ ){if( p->IoU( winList[i], preList[j] ) > 0.85 )winList[i] = preList[j];elseif( p->IoU( winList[i], preList[j] ) > 0.6 ){winList[i].x = ( winList[i].x + preList[j].x ) / 2;winList[i].y = ( winList[i].y + preList[j].y ) / 2;winList[i].width = ( winList[i].width + preList[j].width ) / 2;winList[i].height = ( winList[i].height + preList[j].height ) / 2;}}}preList = winList;}delete[] img_pad.data;m_pre_faces.clear();m_pre_faces = p->TransWindow( image, image, winList );ret.size = int(m_pre_faces.size());ret.data = m_pre_faces.data();return ret;
}

3.1.2 人脸对齐

返回到Register通过FaceEngin.h中line66的return this->Register( image, faces[0] );跳转到FaceLandmarkerPrivate.cppline135 的PointDetectLandmarks函数中，即为提取人脸特征标志点的函数实现


bool FaceLandmarkerPrivate::PointDetectLandmarks( const SeetaImageData &src_img, const SeetaRect &face_info, SeetaPointF *landmarks, int *masks ) const
{// bounding boxdouble width = face_info.width - 1, height = face_info.height - 1;double min_x = face_info.x, max_x = face_info.x + width;double min_y = face_info.y, max_y = face_info.y + height;// move bounding boxmin_x += width * x_move_;max_x += width * x_move_;min_y += height * y_move_;max_y += height * y_move_;//make the bounding box squaredouble center_x = ( min_x + max_x ) / 2.0, center_y = ( min_y + max_y ) / 2.0;double r = ( ( width > height ) ? width : height ) / 2.0;min_x = center_x - r;max_x = center_x + r;min_y = center_y - r;max_y = center_y + r;width = max_x - min_x;height = max_y - min_y;// expandmin_x = round( min_x - width * expand_size_ );min_y = round( min_y - height * expand_size_ );max_x = round( max_x + width * expand_size_ );max_y = round( max_y + height * expand_size_ );SeetaImageData dst_img;dst_img.width = int( max_x ) - int( min_x ) + 1;dst_img.height = int( max_y ) - int( min_y ) + 1;dst_img.channels = src_img.channels;std::unique_ptr<uint8_t[]> dst_img_data( new uint8_t[( int( max_x ) - int( min_x ) + 1 ) * ( int( max_y ) - int( min_y ) + 1 ) * src_img.channels] );dst_img.data = dst_img_data.get();CropFace( src_img.data, src_img.width, src_img.height, src_img.channels,dst_img.data, int( min_x ), int( min_y ), int( max_x ), int( max_y ) );bool flag = PredictLandmark( dst_img, landmarks, masks );for( int i = 0; i < landmark_num_; i++ ){landmarks[i].x += min_x;landmarks[i].y += min_y;}return flag;
}

3.1.3 人脸特征提取

主要的实现在FaceRecognizerPrivate.cppline636的ExtractFeature函数中.

bool FaceRecognizerPrivate::ExtractFeature( const std::vector<SeetaImageData> &faces, float *feats, bool normalization )
{if( !recognizer->net ) return false;if( faces.empty() ) return true;int number = int( faces.size() );int channels = GetCropChannels();int height = GetCropHeight();int width = GetCropWidth();auto single_image_size = channels * height * width;std::unique_ptr<unsigned char[]> data_point_char( new unsigned char[number * single_image_size] );for( int i = 0; i < number; ++i ){if( faces[i].channels == channels &&faces[i].height == height &&faces[i].width == width ){CopyData( &data_point_char[i * single_image_size], faces[i].data, single_image_size );continue;}if( recognizer->method == "resize" ){seeta::Image face( faces[i].data, faces[i].width, faces[i].height, faces[i].channels );seeta::Image fixed = seeta::resize( face, seeta::Size( GetCropWidth(), GetCropHeight() ) );CopyData( &data_point_char[i * single_image_size], fixed.data(), single_image_size );}else{seeta::Image face( faces[i].data, faces[i].width, faces[i].height, faces[i].channels );seeta::Rect rect( ( GetCropWidth() - faces[i].width ) / 2, ( GetCropHeight() - faces[i].height ) / 2, GetCropWidth(), GetCropHeight() );seeta::Image fixed = seeta::crop_resize( face, rect, seeta::Size( GetCropWidth(), GetCropHeight() ) );CopyData( &data_point_char[i * single_image_size], fixed.data(), single_image_size );}}return LocalExtractFeature(number, width, height, channels, data_point_char.get(),recognizer->net, GetMaxBatch(), recognizer->header.blob_name.c_str(), GetFeatureSize(),feats,normalization,recognizer->sqrt_times );
}

3.2 人脸识别

人脸检测、人脸对齐和人脸特征提取与人脸注册内容相同，不再赘述；人脸图像质量评估较为简单，在evaluate函数中；

   float QualityAssessor::evaluate(const SeetaImageData &image, const SeetaRect &face,const SeetaPointF *points) const {// std::cout << "=============================" << std::endl;float clarity;if (check_lightness(image, face)&& check_face_size(face)&& check_pose(image, face, points)&& check_clarity(image, face, clarity)) {return clarity;} else {return 0;}}

人脸特征比对模块在FaceRecognizerPrivate.cppline461的CalcSimilarity函数中。

float FaceRecognizerPrivate::CalcSimilarity( const float *fc1, const float *fc2, long dim )
{if( dim <= 0 ) dim = GetFeatureSize();double dot = 0;double norm1 = 0;double norm2 = 0;for( size_t i = 0; i < dim; ++i ){dot += fc1[i] * fc2[i];norm1 += fc1[i] * fc1[i];norm2 += fc2[i] * fc2[i];}double similar = dot / ( sqrt( norm1 * norm2 ) + 1e-5 );return recognizer->trans( float( similar ) );
}

附：seetaface2代码的三个接口

FaceDetector：人脸检测器会对输入的彩色（三通道）或灰度（单通道）图像进行人脸检测，并返回所
有检测到的人脸位置信息。
FaceLandmarks：人脸特征点检测器根据输入的彩色图片和人脸位置，对人脸特征点进行检测，并返回对
应人脸的特定数量的特征点坐标信息。
FaceRecognizer：人脸识别器根据输入的彩色图片和人脸特征点，对输入的人脸提取特征值，根据提取的
特征值进行两张人脸相似度的比较。

人脸识别SeetaFace2原理与代码详解相关推荐

DeepLearning tutorial（1）Softmax回归原理简介+代码详解
FROM: http://blog.csdn.net/u012162613/article/details/43157801 DeepLearning tutorial(1)Softmax回归原理简介 ...
DeepLearning tutorial（3）MLP多层感知机原理简介+代码详解
FROM:http://blog.csdn.net/u012162613/article/details/43221829 @author:wepon @blog:http://blog.csdn.n ...
DeepLearning tutorial（4）CNN卷积神经网络原理简介+代码详解
FROM: http://blog.csdn.net/u012162613/article/details/43225445 DeepLearning tutorial(4)CNN卷积神经网络原理简介 ...
Pytorch|YOWO原理及代码详解(二)
Pytorch|YOWO原理及代码详解(二) 本博客上接,Pytorch|YOWO原理及代码详解(一),阅前可看. 1.正式训练 if opt.evaluate:logging('evaluating ...
batchnorm原理及代码详解
转载自:http://www.ishenping.com/ArtInfo/156473.html batchnorm原理及代码详解原博文原微信推文见到原作者的这篇微信小文整理得很详尽.故在csd ...
Pytorch | yolov3原理及代码详解（二）
阅前可看: Pytorch | yolov3原理及代码详解(一) https://blog.csdn.net/qq_24739717/article/details/92399359 分析代码: ht ...
【OpenCV/C++】KNN算法识别数字的实现原理与代码详解
KNN算法识别数字一.KNN原理 1.1 KNN原理介绍 1.2 KNN的关键参数二.KNN算法识别手写数字 2.1 训练过程代码详解 2.2 预测分类的实现过程三.KNN算法识别印刷数字 2. ...
BilSTM 实体识别_NLP-入门实体命名识别（NER）+Bilstm-CRF模型原理Pytorch代码详解——最全攻略
最近在系统地接触学习NER,但是发现这方面的小帖子还比较零散.所以我把学习的记录放出来给大家作参考,其中汇聚了很多其他博主的知识,在本文中也放出了他们的原链.希望能够以这篇文章为载体,帮助其他跟我一样 ...
BilSTM 实体识别_NLP入门实体命名识别（NER）+BilstmCRF模型原理Pytorch代码详解——最全攻略...
来自 | 知乎作者 | seven链接 | https://zhuanlan.zhihu.com/p/79552594编辑 | 机器学习算法与自然语言处理公众号本文仅作学术分享,如有侵权,请联系 ...

人脸识别SeetaFace2原理与代码详解