idea自动捕获

by Rishav Agarwal

通过里沙夫·阿加瓦尔

Smilefie：如何通过检测微笑来自动捕获自拍 (Smilefie: how you can auto-capture selfies by detecting a smile)

Ten second takeaway: use Python and OpenCV to create an app that automatically captures a selfie on detecting a smile. Now let’s get into it. :)

十秒钟讲解：使用Python和OpenCV创建一个可在检测到微笑时自动捕获自拍照的应用程序。现在让我们开始吧。 :)

I came across this advertisement for Oppo — the phone automatically captures a selfie when the beautiful actress smiles at the camera. This seemed like a pretty easy challenge given the wonderful dlib library from Python.

我碰到了有关Oppo的广告 –当美丽的女演员对着镜头微笑时，手机会自动捕获自拍照。鉴于Python出色的dlib库，这似乎是一个非常容易的挑战。

In this post, I’ll talk about how you can create a similar app that captures a selfie from a webcam on detecting a smile. All in ~50 lines of code.

在这篇文章中，我将讨论如何创建类似的应用程序，该应用程序在检测到微笑时可以从网络摄像头捕获自拍照。 全部在〜50行代码中 。

Craft.io概述 (Process Overview)

Use the facial landmark detector in dlib to get the mouth coordinates使用dlib中的面部界标检测器获取嘴部坐标
Set up a smile threshold, using a mouth aspect ratio (MAR)使用嘴高宽比(MAR)设置微笑阈值
Access the webcam to setup a live stream访问网络摄像头以设置实时流
Capture the image拍摄影像
Save the image保存图像
Close the cam feed关闭凸轮供纸器

需要图书馆 (Libraries required)

Numpy: Used for fast matrix calculations and manipulations.

numpy：用于快速矩阵计算和处理。
dlib: Library containing the facial landmarks.

dlib ：包含面部标志的库。
Cv2: The Open CV library used for image manipulation and saving.

Cv2 ：用于图像处理和保存的Open CV库。
Scipy.spatial : Used to calculate the Euclidean distance between facial points.

Scipy.spatial ：用于计算面部之间的欧几里得距离。
Imutils: Library to access video stream.

Imutils ：用于访问视频流的库。

All libraries can be installed using pip, except dlib. For dlib we have to install CMake and boost. Here is how to install them on macOS using brew.

除 dlib 外，所有库都可以使用pip安装。对于dlib，我们必须安装CMake和boost 。这是使用brew在macOS上安装它们的方法。

If you don’t have brew, here’s how to install Homebrew.

如果您没有Brew，请按照以下步骤安装Homebrew 。

安装CMake (Install CMake)

brew install cmake

安装提升 (Install boost)

brew install boostbrew install boost-python --with-python3

The second command makes sure that boost is usable with Python 3.

第二个命令确保boost可以在Python 3中使用 。

安装dlib (Install dlib)

After this, we can install dlib using

之后，我们可以使用以下命令安装dlib

pip install dlib

Tip: I like to use Anaconda, a virtual environment for each separate project. Here is a great blog on the whys and hows of the conda environment.

提示：我喜欢对每个单独的项目使用虚拟环境Anaconda 。这是一个关于conda环境的原因和方式的很棒的博客。

导入库 (Importing libraries)

from scipy.spatial import distance as distfrom imutils.video import VideoStream, FPSfrom imutils import face_utilsimport imutilsimport numpy as npimport timeimport dlibimport cv2

面部界标探测器 (Facial landmark detector)

The facial landmark detector is an API implemented inside dlib. It produces 68 x- y-coordinates that map to specific facial structures.

面部标志检测器是在dlib中实现的API 。它产生68个x- y坐标 ，映射到特定的面部结构。

This can be visualised as:

可以将其可视化为：

We will focus on the mouth which can be accessed through point range [49,…, 68]. There are twenty coordinates.

我们将重点介绍可通过点范围[49，…，68]访问的嘴。有二十个坐标。

Using dlib, we can get these features using the following code:

使用dlib，我们可以使用以下代码获得这些功能：

shape_predictor= “../shape_predictor_68_face_landmarks.dat”detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor(shape_predictor)(mStart, mEnd) = face_utils.FACIAL_LANDMARKS_IDXS[“mouth”]

(mStart, mEnd) gets us the first and last coordinates for the mouth.

(mStart, mEnd)获取我们嘴的第一个和最后一个坐标。

You can download the pre-trained landmark file here or just email me and I’ll send it to you. Remember to extract it.

您可以在此处下载经过预先训练的地标文件，也可以给我发送电子邮件，我会将其发送给您。记住要提取它。

微笑功能 (The smile function)

The image below shows only the twenty mouth coordinates:

下图仅显示了二十个嘴坐标：

I created a mouth aspect ratio (MAR) inspired by two articles on blink detection. These are Real-Time Eye Blink Detection using Facial Landmarks. and Eye blink detection with OpenCV, Python, and dlib. The second article expands on the first. Both discuss an aspect ratio, in this case for the eyes (EAR):

我创建了一张受两篇关于眨眼检测的文章启发的嘴高宽比(MAR)。这些是使用面部地标的实时眼睛眨眼检测。和使用OpenCV，Python和dlib进行眨眼检测。第二篇文章是第一篇文章的扩展。两者都讨论了长宽比，在这种情况下是针对眼睛(EAR)的：

The formula for the EAR is:

EAR的公式是：

D = distance between p1 and p4

D = p1和p4之间的距离

L= average of distance between p2 and p6; p3 and p5

L = p2和p6之间的平均距离； p3和p5

EAR= L/D

In our case, MAR is defined simply as the relationship of the points shown below

在我们的案例中，MAR的定义简单为如下所示的点之间的关系

We compute the distance between p49 and p55 as D, and we average the distances between:

我们将p49和p55之间的距离计算为D，并平均以下距离：

p51 and p59

p51和p59

p52 and p58

p52和p58

p53 and p57

p53和p57

Let’s call it L, using the same naming structure:

我们使用相同的命名结构，将其命名为L：

MAR = L/D

Here is the function to calculate the MAR.

这是计算MAR的函数。

def smile(mouth): A = dist.euclidean(mouth[3], mouth[9]) B = dist.euclidean(mouth[2], mouth[10]) C = dist.euclidean(mouth[4], mouth[8]) L = (A+B+C)/3 D = dist.euclidean(mouth[0], mouth[6]) mar=L/D return mar

Tip: When we splice the array the point 49 becomes first element of the array (0) and all the other indices are adjusted accordingly:

提示：拼接数组时，点49成为数组(0)的第一个元素，所有其他索引也进行相应调整：

Smiling with the mouth closed increases the distance between p49 and p55 and decreases the distance between the top and bottom points. So, L will decrease and D will increase.

闭上嘴微笑会增加p49和p55之间的距离，并减小最高点和最低点之间的距离。因此，L将减少而D将增加。

Smiling with mouth open leads to D decreasing and L increasing.

张开嘴微笑会导致D减小和L增大。

See how the MAR changes when I change mouth shapes:

查看当我改变嘴形时MAR的变化：

Based on this, I set a smile to be a MAR of <.3 or &gt;.38. I could have taken just D as D will always increase when one is smiling. But D will not be same for all, as people have different mouth shapes.

基于此，我将MAR设置为<.3或＆ lt; .38。我本来可以选择D，因为当一个人微笑时D总是会增加。但是D并非所有人都一样，因为人们的嘴形不同。

These are crude estimates and may include other emotions like “awe”. To overcome this, you can create a more advanced model. You could take more facial features into account, or simply train a CV-based emotions classifier.

这些是粗略的估计，可能包括“敬畏”之类的其他情绪。为了克服这个问题，您可以创建一个更高级的模型。您可以考虑更多的面部特征，或者只是训练基于简历的情绪分类器。

Now that we have a smile function, we can implement the video capture.

现在我们有了一个微笑功能，我们可以实现视频捕获了。

视频截取 (Video capture)

访问网络摄像头 (Access the webcam)

We can access the webcam through the imutils library using the following command. cv2.namedWindow creates a new window:

我们可以使用以下命令通过imutils库访问网络摄像头。 cv2.namedWindow创建一个新窗口：

vs = VideoStream(src=0).start()fileStream = Falsetime.sleep(1.0)cv2.namedWindow('frame',cv2.WINDOW_NORMAL)

人脸检测 (Face detection)

Now we come to the main loop where the magic happens. First we capture a single frame and convert it to grayscale for easy computation. We use this to detect the face. cv2.convexHull(mouth) detects the mouth outline and cv2.drawContours draws a green outline around it.

现在我们进入魔术发生的主循环。首先，我们捕获单个帧并将其转换为灰度以便于计算。我们用它来检测人脸。 cv2.convexHull(mouth) 检测到嘴部轮廓，然后cv2.drawContours在其周围绘制绿色轮廓。

while True: frame = vs.read() frame = imutils.resize(frame, width=450) gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) rects = detector(gray, 0) for rect in rects:  shape = predictor(gray, rect)  shape = face_utils.shape_to_np(shape)  mouth= shape[mStart:mEnd]  mar= smile(mouth)  mouthHull = cv2.convexHull(mouth)  cv2.drawContours(frame, [mouthHull], -1, (0, 255, 0), 1)

Tip: this setup can detect multiple smiles in a single frame.

提示：此设置可以在一帧中检测多个笑容。

自动捕捉 (Auto-capture)

Next we set the auto-capture condition:

接下来，我们设置自动捕获条件：

if mar <= .3 or mar > .38 : COUNTER += 1 else:  if COUNTER >= 15:   TOTAL += 1   frame = vs.read()   time.sleep(0.3)   img_name = “opencv_frame_{}.png”.format(TOTAL)   cv2.imwrite(img_name, frame)   print(“{} written!”.format(img_name))   cv2.destroyWindow(“test”)  COUNTER = 0

Here, I consider a smile to be “selfie worthy” if the person holds it for half a second, or 30 frames.

在这里，如果有人将微笑保持半秒钟或30帧，我认为微笑是“值得自拍照的”。

We check if the MAR is < .3 or &gt; .38 for at least 15 frames and then save the 16th frame. The file is saved to the same folder as the code with name “opencv_frame_<counter>.png”.

我们检查MAR是否<.3或＆g t; .38至少15帧，然后保存第16帧。该文件与名称为“ opencv_frame_ <counter> .png”的代码保存在同一文件夹中。

I have added a few time.sleep functions to smooth out the experience. Phones usually get around these hardware issues by using tricks like animations or loading screens.

我增加了一些时间time.sleep 功能以平滑体验。手机通常会通过使用动画或加载屏幕等技巧来解决这些硬件问题。

Tip: This part is inside the while loop.

提示：此部分位于while循环内。

We also print the MAR on the frame with the cv2.putText function:

我们还使用cv2.putText函数将MAR打印在框架上：

cv2.putText(frame, “MAR: {}”.format(mar), (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

Tip: My Mac has a 30 fps camera, so I used 30 as the number of frames. You can change this accordingly. An easier way is to find the fps is to use the fps function in imutils.

提示：我的Mac具有30 fps的相机，所以我使用30帧数。您可以相应地更改它。查找fps的一种简单方法是在imutils中使用fps功能。

退出视频流 (Quit video streaming)

Finally, put a quit command that stops the video streaming when the “q” key is pressed. This is achieved by adding:

最后，输入退出命令，当按下“ q”键时停止视频流。这可以通过添加以下内容来实现：

key2 = cv2.waitKey(1) & 0xFF if key2 == ord(‘q’): break

Lastly, we destroy the window using

最后，我们使用

cv2.destroyAllWindows()vs.stop()

and we are done!

我们完成了！

The entire code in action:

运行中的整个代码：

You can find the entire code on my GitHub.

您可以在我的GitHub上找到完整的代码。

This was a basic application of the amazing dlib library. From here, you can go on to create things like your own snapchat filters, high-tech home surveillance systems, or even a post-Orwellian happiness detector.

这是惊人的dlib库的基本应用程序。在这里，您可以继续创建自己的snapchat过滤器，高科技家庭监视系统，甚至是后奥威尔式的幸福探测器。

Tweet at me in case you end up doing any more cool things with this or find a better smile detector. Another cool idea is to do some post processing to the captured image (as in the advertisement) to make the picture prettier.

如果您最终用它做更多更酷的事情或找到更好的微笑探测器，请向我发送推文。另一个很酷的想法是对捕获的图像(如广告中)进行一些后期处理，以使图片更漂亮。

Thanks for reading. If you liked what you read, clap, and follow me. It would mean a lot and encourage me to write more. Let’s connect on Twitter and Linkedin as well :)

谢谢阅读。如果您喜欢阅读的内容，请鼓掌并关注我。这将意味着很多，并鼓励我写更多内容。让我们同时在Twitter和Linkedin上连接:)

翻译自: https://www.freecodecamp.org/news/smilfie-auto-capture-selfies-by-detecting-a-smile-using-opencv-and-python-8c5cfb6ec197/