OpenCV进阶（8）性别和年龄识别

在本教程中，我们将讨论深度学习应用于人脸的一个有趣应用。我们将估计年龄并从单个图像中找出人的性别。我们将简要讨论本文的主要思想，并提供有关如何在 OpenCV 中使用该模型的分步说明。我们将使用 OpenCV 学习性别和年龄分类。

1. 使用 CNN 进行性别和年龄分类

作者使用了一个非常简单的卷积神经网络架构，类似于CaffeNet和AlexNet。该网络使用3个卷积层，2个完全连接层和一个最终输出层。下面给出了这些层的详细信息。

Conv1：第一个卷积层有96 个，核大小为 7的卷积核。
Conv2：第二个卷积层有256 个，核大小为 5的卷积核。
Conv3：第三个卷积层有 384 个，核大小为 3的卷积核。
两个全连接层各有 512 个节点。
他们使用 Adience 数据集来训练模型。

1.1性别预测

们将性别预测定义为一个分类问题。性别预测网络中的输出层是 softmax 类型，有 2 个节点，表示“男”和“女”两类。

1.2年龄预测

理想情况下，年龄预测应该作为回归问题来处理，因为我们期望输出一个实数。然而，使用回归准确估计年龄具有挑战性。即使是人类也无法通过观察一个人来准确预测年龄。但是，我们知道他们是 20 多岁还是 30 多岁。由于这个原因，明智的做法是将这个问题定义为一个分类问题，我们尝试估计这个人所处的年龄组。例如，0-2 岁范围内的年龄是一个类别，4-6 岁是另一个类别类等。

Adience 数据集有 8 个类别，分为以下年龄组 [(0 – 2), (4 – 6), (8 – 12), (15 – 20), (25 – 32), (38 – 43), ( 48 – 53), (60 – 100)]。因此，年龄预测网络在最后的 softmax 层中有 8 个节点，表示提到的年龄范围。

应该记住，从单一图像预测年龄不是一个很容易解决的问题，因为感知的年龄取决于很多因素，相同年龄的人在世界各地可能看起来非常不同。而且，人们非常努力地隐藏他们的真实年龄!

2. 代码教程

代码可以分为四部分：

1.检测人脸
2.检测性别
3.检测年龄
4.显示输出

2.1 代码展示

让我们看看在 OpenCV 中使用 DNN 模块进行性别和年龄预测的代码。

链接：https://pan.baidu.com/s/1eVd4kEczt4diGApc6BbQFQ
提取码：123a

（1）Python

# Usage
# python AgeGender.py --input sample1.jpg# 导入所需模块
import cv2 as cv
import math
import time
import argparsedef getFaceBox(net, frame, conf_threshold=0.15):frameOpencvDnn = frame.copy()frameHeight = frameOpencvDnn.shape[0]frameWidth = frameOpencvDnn.shape[1]blob = cv.dnn.blobFromImage(frameOpencvDnn, 1.0, (300, 300), [104, 117, 123], True, False)net.setInput(blob)detections = net.forward()bboxes = []for i in range(detections.shape[2]):confidence = detections[0, 0, i, 2]if confidence > conf_threshold:x1 = int(detections[0, 0, i, 3] * frameWidth)y1 = int(detections[0, 0, i, 4] * frameHeight)x2 = int(detections[0, 0, i, 5] * frameWidth)y2 = int(detections[0, 0, i, 6] * frameHeight)bboxes.append([x1, y1, x2, y2])cv.rectangle(frameOpencvDnn, (x1, y1), (x2, y2), (0, 255, 0), int(round(frameHeight/150)), 8)return frameOpencvDnn, bboxesparser = argparse.ArgumentParser(description='Use this script to run age and gender recognition using OpenCV.')
parser.add_argument('--input', help='Path to input image or video file. Skip this argument to capture frames from a camera.', default="people.jpg")
parser.add_argument("--device", default="cpu", help="Device to inference on")args = parser.parse_args()args = parser.parse_args()faceProto = "opencv_face_detector.pbtxt"
faceModel = "opencv_face_detector_uint8.pb"ageProto = "age_deploy.prototxt"
ageModel = "age_net.caffemodel"genderProto = "gender_deploy.prototxt"
genderModel = "gender_net.caffemodel"MODEL_MEAN_VALUES = (78.4263377603, 87.7689143744, 114.895847746)
ageList = ['(0-2)', '(4-6)', '(8-12)', '(15-20)', '(25-32)', '(38-43)', '(48-53)', '(60-100)']
genderList = ['Male', 'Female']# 加载网络
ageNet = cv.dnn.readNet(ageModel, ageProto)
genderNet = cv.dnn.readNet(genderModel, genderProto)
faceNet = cv.dnn.readNet(faceModel, faceProto)if args.device == "cpu":ageNet.setPreferableBackend(cv.dnn.DNN_TARGET_CPU)genderNet.setPreferableBackend(cv.dnn.DNN_TARGET_CPU)faceNet.setPreferableBackend(cv.dnn.DNN_TARGET_CPU)print("Using CPU device")
elif args.device == "gpu":ageNet.setPreferableBackend(cv.dnn.DNN_BACKEND_CUDA)ageNet.setPreferableTarget(cv.dnn.DNN_TARGET_CUDA)genderNet.setPreferableBackend(cv.dnn.DNN_BACKEND_CUDA)genderNet.setPreferableTarget(cv.dnn.DNN_TARGET_CUDA)genderNet.setPreferableBackend(cv.dnn.DNN_BACKEND_CUDA)genderNet.setPreferableTarget(cv.dnn.DNN_TARGET_CUDA)print("Using GPU device")# 打开视频文件或图像文件或相机流
cap = cv.VideoCapture(args.input if args.input else 0)
padding = 20
while cv.waitKey(1) < 0:# 读取帧图像t = time.time()hasFrame, frame = cap.read()if not hasFrame:cv.waitKey()breakframeFace, bboxes = getFaceBox(faceNet, frame)if not bboxes:print("No face Detected, Checking next frame")continuefor bbox in bboxes:# print(bbox)face = frame[max(0,bbox[1]-padding):min(bbox[3]+padding,frame.shape[0]-1),max(0,bbox[0]-padding):min(bbox[2]+padding, frame.shape[1]-1)]blob = cv.dnn.blobFromImage(face, 1.0, (227, 227), MODEL_MEAN_VALUES, swapRB=False)genderNet.setInput(blob)genderPreds = genderNet.forward()gender = genderList[genderPreds[0].argmax()]# print("Gender Output : {}".format(genderPreds))print("Gender : {}, conf = {:.3f}".format(gender, genderPreds[0].max()))ageNet.setInput(blob)agePreds = ageNet.forward()age = ageList[agePreds[0].argmax()]print("Age Output : {}".format(agePreds))print("Age : {}, conf = {:.3f}".format(age, agePreds[0].max()))label = "{},{}".format(gender, age)cv.putText(frameFace, label, (bbox[0], bbox[1]-10), cv.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2, cv.LINE_AA)cv.imshow("Age Gender Demo", frameFace)# cv.imwrite("age-gender-out-{}".format(args.input),frameFace)print("time : {:.3f}".format(time.time() - t))# cmake -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=~/opencv_gpu -DINSTALL_PYTHON_EXAMPLES=OFF -DINSTALL_C_EXAMPLES=OFF -DOPENCV_ENABLE_NONFREE=ON -DOPENCV_EXTRA_MODULES_PATH=~/cv2_gpu/opencv_contrib/modules -DPYTHON_EXECUTABLE=~/env/bin/python3 -DBUILD_EXAMPLES=ON -DWITH_CUDA=ON -DWITH_CUDNN=ON -DOPENCV_DNN_CUDA=ON  -DENABLE_FAST_MATH=ON -DCUDA_FAST_MATH=ON  -DWITH_CUBLAS=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.2 -DOpenCL_LIBRARY=/usr/local/cuda-10.2/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-10.2/include/ ..

（2）C++

// Usage
//./AgeGender sample1.jpg
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/dnn.hpp>
#include <tuple>
#include <iostream>
#include <opencv2/opencv.hpp>
#include <iterator>
using namespace cv;
using namespace cv::dnn;
using namespace std;tuple<Mat, vector<vector<int>>> getFaceBox(Net net, Mat &frame, double conf_threshold)
{Mat frameOpenCVDNN = frame.clone();int frameHeight = frameOpenCVDNN.rows;int frameWidth = frameOpenCVDNN.cols;double inScaleFactor = 1.0;Size size = Size(300, 300);// std::vector<int> meanVal = {104, 117, 123};Scalar meanVal = Scalar(104, 117, 123);cv::Mat inputBlob;inputBlob = cv::dnn::blobFromImage(frameOpenCVDNN, inScaleFactor, size, meanVal, true, false);net.setInput(inputBlob, "data");cv::Mat detection = net.forward("detection_out");cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());vector<vector<int>> bboxes;for(int i = 0; i < detectionMat.rows; i++){float confidence = detectionMat.at<float>(i, 2);if(confidence > conf_threshold){int x1 = static_cast<int>(detectionMat.at<float>(i, 3) * frameWidth);int y1 = static_cast<int>(detectionMat.at<float>(i, 4) * frameHeight);int x2 = static_cast<int>(detectionMat.at<float>(i, 5) * frameWidth);int y2 = static_cast<int>(detectionMat.at<float>(i, 6) * frameHeight);vector<int> box = {x1, y1, x2, y2};bboxes.push_back(box);cv::rectangle(frameOpenCVDNN, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(0, 255, 0),2, 4);}}return make_tuple(frameOpenCVDNN, bboxes);
}int main(int argc, char** argv)
{string faceProto = "opencv_face_detector.pbtxt";string faceModel = "opencv_face_detector_uint8.pb";string ageProto = "age_deploy.prototxt";string ageModel = "age_net.caffemodel";string genderProto = "gender_deploy.prototxt";string genderModel = "gender_net.caffemodel";Scalar MODEL_MEAN_VALUES = Scalar(78.4263377603, 87.7689143744, 114.895847746);vector<string> ageList = {"(0-2)", "(4-6)", "(8-12)", "(15-20)", "(25-32)","(38-43)", "(48-53)", "(60-100)"};vector<string> genderList = {"Male", "Female"};cout << "USAGE : ./AgeGender <videoFile> " << endl;cout << "USAGE : ./AgeGender <device> " << endl;cout << "USAGE : ./AgeGender <videoFile> <device>" << endl;string device = "cpu";string videoFile = "0";// 从命令行获取参数if (argc == 2){   if((string)argv[1] == "gpu")device = "gpu";else if((string)argv[1] == "cpu")device = "cpu";else videoFile = argv[1];}else if (argc == 3){videoFile = argv[1];if((string)argv[2] == "gpu")device = "gpu";}// 加载网络模型Net ageNet = readNet(ageModel, ageProto);Net genderNet = readNet(genderModel, genderProto);Net faceNet = readNet(faceModel, faceProto);if (device == "cpu"){cout << "Using CPU device" << endl;ageNet.setPreferableBackend(DNN_TARGET_CPU);genderNet.setPreferableBackend(DNN_TARGET_CPU);faceNet.setPreferableBackend(DNN_TARGET_CPU);}else if (device == "gpu"){cout << "Using GPU device" << endl;ageNet.setPreferableBackend(DNN_BACKEND_CUDA);ageNet.setPreferableTarget(DNN_TARGET_CUDA);genderNet.setPreferableBackend(DNN_BACKEND_CUDA);genderNet.setPreferableTarget(DNN_TARGET_CUDA);faceNet.setPreferableBackend(DNN_BACKEND_CUDA);faceNet.setPreferableTarget(DNN_TARGET_CUDA);}VideoCapture cap;if (videoFile.length() > 1)cap.open(videoFile);elsecap.open(0);int padding = 20;while(waitKey(1) < 0) {// read frameMat frame;cap.read(frame);if (frame.empty()){waitKey();break;}vector<vector<int>> bboxes;Mat frameFace;tie(frameFace, bboxes) = getFaceBox(faceNet, frame, 0.7);if(bboxes.size() == 0) {cout << "No face detected, checking next frame." << endl;continue;}for (auto it = begin(bboxes); it != end(bboxes); ++it) {Rect rec(it->at(0) - padding, it->at(1) - padding, it->at(2) - it->at(0) + 2*padding, it->at(3) - it->at(1) + 2*padding);Mat face = frame(rec); // take the ROI of box on the frameMat blob;blob = blobFromImage(face, 1, Size(227, 227), MODEL_MEAN_VALUES, false);genderNet.setInput(blob);// string gender_preds;vector<float> genderPreds = genderNet.forward();// 在这里打印性别// 找到最大元素索引// 距离函数在 C++ 中执行 argmax() 工作int max_index_gender = std::distance(genderPreds.begin(), max_element(genderPreds.begin(), genderPreds.end()));string gender = genderList[max_index_gender];cout << "Gender: " << gender << endl;/* // 如果您想遍历gender_preds 向量，请取消注释for(auto it=begin(gender_preds); it != end(gender_preds); ++it) {cout << *it << endl;}*/ageNet.setInput(blob);vector<float> agePreds = ageNet.forward();/* // 如果要遍历 age_preds，请取消注释下面的代码* vectorcout << "PRINTING AGE_PREDS" << endl;for(auto it = age_preds.begin(); it != age_preds.end(); ++it) {cout << *it << endl;}*/// 在 age_preds 向量中找到最大值索引int max_indice_age = std::distance(agePreds.begin(), max_element(agePreds.begin(), agePreds.end()));string age = ageList[max_indice_age];cout << "Age: " << age << endl;string label = gender + ", " + age; // labelcv::putText(frameFace, label, Point(it->at(0), it->at(1) -15), cv::FONT_HERSHEY_SIMPLEX, 0.9, Scalar(0, 255, 255), 2, cv::LINE_AA);imshow("Frame", frameFace);imwrite("out.jpg",frameFace);}}
}

2.2代码解析

2.2.1人脸检测

我们将使用 DNN 人脸检测器进行人脸检测。该模型只有 2.7MB，即使在 CPU 上也非常快。人脸检测是使用函数 getFaceBox 完成的，如下所示。

tuple<Mat, vector<vector<int>>> getFaceBox(Net net, Mat &frame, double conf_threshold)
{Mat frameOpenCVDNN = frame.clone();int frameHeight = frameOpenCVDNN.rows;int frameWidth = frameOpenCVDNN.cols;double inScaleFactor = 1.0;Size size = Size(300, 300);// std::vector<int> meanVal = {104, 117, 123};Scalar meanVal = Scalar(104, 117, 123);cv::Mat inputBlob;cv::dnn::blobFromImage(frameOpenCVDNN, inputBlob, inScaleFactor, size, meanVal, true, false);net.setInput(inputBlob, "data");cv::Mat detection = net.forward("detection_out");cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());vector<vector<int>> bboxes;for(int i = 0; i < detectionMat.rows; i++){float confidence = detectionMat.at<float>(i, 2);if(confidence > conf_threshold){int x1 = static_cast<int>(detectionMat.at<float>(i, 3) * frameWidth);int y1 = static_cast<int>(detectionMat.at<float>(i, 4) * frameHeight);int x2 = static_cast<int>(detectionMat.at<float>(i, 5) * frameWidth);int y2 = static_cast<int>(detectionMat.at<float>(i, 6) * frameHeight);vector<int> box = {x1, y1, x2, y2};bboxes.push_back(box);cv::rectangle(frameOpenCVDNN, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(0, 255, 0),2, 4);}}return make_tuple(frameOpenCVDNN, bboxes);
}

def getFaceBox(net, frame, conf_threshold=0.7):frameOpencvDnn = frame.copy()frameHeight = frameOpencvDnn.shape[0]frameWidth = frameOpencvDnn.shape[1]blob = cv.dnn.blobFromImage(frameOpencvDnn, 1.0, (300, 300), [104, 117, 123], True, False)net.setInput(blob)detections = net.forward()bboxes = []for i in range(detections.shape[2]):confidence = detections[0, 0, i, 2]if confidence > conf_threshold:x1 = int(detections[0, 0, i, 3] * frameWidth)y1 = int(detections[0, 0, i, 4] * frameHeight)x2 = int(detections[0, 0, i, 5] * frameWidth)y2 = int(detections[0, 0, i, 6] * frameHeight)bboxes.append([x1, y1, x2, y2])cv.rectangle(frameOpencvDnn, (x1, y1), (x2, y2), (0, 255, 0), int(round(frameHeight/150)), 8)return frameOpencvDnn, bboxes

2.2.2预测性别

我们将性别网络加载到内存中，并将检测到的人脸通过网络。前向传递给出了两个类的概率或置信度。我们取两个输出的最大值并将其用作最终的性别预测。

string genderProto = "gender_deploy.prototxt";
string genderModel = "gender_net.caffemodel";
Net genderNet = readNet(genderModel, genderProto);vector<string> genderList = {"Male", "Female"};blob = blobFromImage(face, 1, Size(227, 227), MODEL_MEAN_VALUES, false);
genderNet.setInput(blob);
// string gender_preds;
vector<float> genderPreds = genderNet.forward();
// 在这里打印性别
// 找到最大元素索引
// distance函数在c++中执行argmax()函数
int max_index_gender = std::distance(genderPreds.begin(), max_element(genderPreds.begin(), genderPreds.end()));
string gender = genderList[max_index_gender];

genderProto = "gender_deploy.prototxt"
genderModel = "gender_net.caffemodel"
ageNet = cv.dnn.readNet(ageModel, ageProto)genderList = ['Male', 'Female']blob = cv.dnn.blobFromImage(face, 1, (227, 227), MODEL_MEAN_VALUES, swapRB=False)
genderNet.setInput(blob)
genderPreds = genderNet.forward()
gender = genderList[genderPreds[0].argmax()]
print("Gender Output : {}".format(genderPreds))
print("Gender : {}".format(gender))

2.2.3预测年龄

我们加载年龄网络并使用前向传递来获得输出。由于网络架构类似于性别网络，我们可以从所有输出中取最大值以获得预测的年龄组。

string ageProto = "age_deploy.prototxt";
string ageModel = "age_net.caffemodel";
Net ageNet = readNet(ageModel, ageProto);vector<string> ageList = {"(0-2)", "(4-6)", "(8-12)", "(15-20)", "(25-32)", "(38-43)", "(48-53)", "(60-100)"};ageNet.setInput(blob);
vector<float> agePreds = ageNet.forward();
int max_indice_age = distance(agePreds.begin(), max_element(agePreds.begin(), agePreds.end()));
string age = ageList[max_indice_age];

ageProto = "age_deploy.prototxt"
ageModel = "age_net.caffemodel"
ageNet = cv.dnn.readNet(ageModel, ageProto)ageList = ['(0 - 2)', '(4 - 6)', '(8 - 12)', '(15 - 20)', '(25 - 32)', '(38 - 43)', '(48 - 53)', '(60 - 100)']ageNet.setInput(blob)
agePreds = ageNet.forward()
age = ageList[agePreds[0].argmax()]
print("Gender Output : {}".format(agePreds))
print("Gender : {}".format(age))

2.3显示输出

我们将在输入图像上显示网络的输出，并使用 imshow 函数显示它们。

string label = gender + ", " + age; // label
cv::putText(frameFace, label, Point(it->at(0), it->at(1) -20), cv::FONT_HERSHEY_SIMPLEX, 0.9, Scalar(0, 255, 255), 2, cv::LINE_AA);
imshow("Frame", frameFace);

label = "{}, {}".format(gender, age)
cv.putText(frameFace, label, (bbox[0], bbox[1]-20), cv.FONT_HERSHEY_SIMPLEX, 0.8, (255, 0, 0), 3, cv.LINE_AA)
cv.imshow("Age Gender Demo", frameFace)

2.4结果

3.结论

尽管性别预测网络表现良好，但年龄预测网络没有达到我们的预期。我们试图在论文中找到答案，并找到以下年龄预测模型的混淆矩阵。

从上表可以得出以下意见：

0-2、4-6、8-13 和 25-32 年龄组的预测准确度相对较高。（见对角线元素）
输出严重偏向于年龄组 25-32（参见属于年龄组 25-32 的行）。这意味着网络很容易混淆 15 到 43 岁之间的年龄。因此，即使实际年龄在 15-20 或 38-43 之间，预测年龄也很有可能是 25- 32.这在结果部分也很明显。

除此之外，我们观察到如果我们在检测到的人脸周围使用填充，模型的准确性会提高。这可能是因为训练时的输入是标准的人脸图像，而不是我们在人脸检测后得到的裁剪得很近的人脸。

我们还在进行预测之前分析了人脸对齐的使用，发现某些示例的预测有所改善，但与此同时，某些示例的预测变得更糟。如果您主要使用非正面的面孔，则使用对齐可能是个好主意。

参考目录

https://learnopencv.com/age-gender-classification-using-opencv-deep-learning-c-python/