People often try to look like a celebrity they like the most. They always seem to love it when they get compliments like…

人们经常试图看起来像他们最喜欢的名人。 当他们得到诸如……的称赞时，他们似乎总是喜欢它。

“Hey, you look just like that guy from the Interstellar movie, OMG, what’s his name…??? Yes, MATTHEW MCCONAUGHEY!!”

“嘿，你看起来就像星际电影中的那个家伙，OMG，他叫什么名字？？？ 是的，MATTHEW MCCONAUGHEY！”

Meanwhile…

与此同时…

Maybe your friends were just being polite to make you feel good, but is there a way to objectively identify which celebrity you resemble the most?

也许您的朋友只是彬彬有礼，使您感觉良好，但是有没有一种方法可以客观地确定您最喜欢哪个名人？

In this post, I’ll use a Siamese Neural Network approach to calculate a similarity percentage(0–100%) between two images by comparing facial features.

在本文中，我将使用暹罗神经网络方法通过比较面部特征来计算两幅图像之间的相似度百分比(0–100％)。

介绍 (INTRODUCTION)

Comparing images for similarities has far and wide-reaching applications. Some use cases for matching images can be found in the field of healthcare, where radiology reports could be compared with stock images of some medical condition in order to assist doctors in the diagnosis of diseases.

比较图像的相似性具有广泛的应用。 可以在医疗保健领域找到一些匹配图像的用例，可以将放射线报告与某些医疗状况的库存图像进行比较，以帮助医生诊断疾病。

In retail where an image of a product might be searched on an e-commerce website for price and availability details and in general search applications for looking up an image in a database of images. Such a task is far from trivial as images of similar objects might appear different due to differences in camera equipment, lighting conditions, image orientation, color, and resolution.

在零售中，可以在电子商务网站上搜索产品图像以获取价格和可用性详细信息，并且通常在搜索应用程序中用于在图像数据库中查找图像。 这样的任务绝非易事，因为由于摄像设备，照明条件，图像方向，颜色和分辨率的差异，相似物体的图像可能看起来有所不同。

There have been many algorithms that have been proposed for matching images, such as SIFT, Pyramid Match among others. Recent advancements in Convolutional Neural Networks (CNNs) have enabled direct comparison of images without the use of handcrafted features that were earlier used to compare images.

已经提出了许多用于匹配图像的算法，例如SIFT，金字塔匹配。 卷积神经网络(CNN)的最新进展使图像的直接比较成为可能，而无需使用以前用于比较图像的手工功能。

Siamese networks are typically used in this domain, they are a special type of neural network architecture. Instead of a model learning to classify its inputs, the Siamese networks learn to differentiate between two inputs. It learns the similarity between them.

暹罗网络通常用于此领域，它们是神经网络架构的一种特殊类型。 暹罗网络不是在学习模型来对其输入进行分类，而是在两个输入之间进行区分。 它了解它们之间的相似性。

数据集 (DATASET)

The dataset used for this exercise is taken from FaceScrub, which is a face dataset built by detecting faces in images returned from searches for public figures on the Internet, followed by automatically discarding those not belonging to each queried person. It comprises a total of over 100,000 face images of male and female 530 celebrities, with about 200 images per person. The name and gender annotation and the pixel values of the faces of celebrities inside the images are also included in this dataset.

此练习使用的数据集来自FaceScrub ， FaceScrub是通过检测Internet搜索公共人物返回的图像中的人脸，然后自动丢弃不属于每个被查询者的人脸而构建的人脸数据集。 它总共包含100,000幅530位男性和女性名人的面部图像，每人约200张图像。 图像中名人的名字和性别注释以及面Kong的像素值也包含在此数据集中。

数据预处理 (DATA PREPROCESSING)

• Discard all black and white images, and keep on the colored images, for consistency.丢弃所有黑白图像，并保留彩色图像，以保持一致性。
• For the sake of simplicity (and because of the limited computation power I had), only 20 images per celebrity were used for training.为了简单起见(并且由于我的计算能力有限)，每个名人仅使用20张图像进行训练。
• Crop out the faces of celebrities from the images.从图像中裁剪出名人的脸。
• Resize all the images to a 150X150 resolution (RGB images) for a consistent input size.将所有图像调整为150X150分辨率(RGB图像)，以获得一致的输入尺寸。
• Randomly applied a contrast limited adaptive histogram equalization algorithm to the images to improve the contrast.将对比度受限的自适应直方图均衡算法随机应用于图像以提高对比度。
• Added additional augmented images on the fly while training the network to reduce overfitting on the training set. The images were rotated randomly from -30 to 30 degrees and flipped.在训练网络时动态添加其他增强图像，以减少训练集的过度拟合。 图像从-30度随机旋转到30度并翻转。

模型— 暹罗神经网络 (MODEL — SIAMESE NEURAL NETWORK)

A Siamese neural network (sometimes called a twin neural network) is an artificial neural network that uses the same weights while working in tandem on two different input vectors to compute comparable output vectors (Wiki).

暹罗神经网络(有时称为双神经网络)是一种人工神经网络，它在两个不同的输入向量上串联工作时使用相同的权重，以计算可比较的输出向量( Wiki )。

This diagram illustrates how a Siamese neural network works. The two sister networks are identical and share the hyperparameters and weights. Each sister network of the siamese network is fed with a different image and the neural network is trained using triplet loss or contrastive loss. The loss is calculated using the ground truth, i.e. whether the images are similar or not.

该图说明了暹罗神经网络的工作方式。 两个姐妹网络是相同的，并且共享超参数和权重。 暹罗网络的每个姐妹网络都收到不同的图像，并且使用三重态损失或对比损失来训练神经网络。 使用地面真实性(即图像是否相似)来计算损失。

The Sister network used here is a SqueezeNet which is an 18-layer Neural network consisting of Convolution, Normalization, Pooling, and ReLU activation layers. In addition to that, it also consists of 8 Fire Nodes as illustrated below.

这里使用的姐妹网络是SqueezeNet ，它是一个18层的神经网络，由卷积，规范化，池化和ReLU激活层组成。 除此之外，它还包括8个火节点，如下图所示。

Finally, since the objective of this solution is not to classify but to differentiate between images, a Contrastive Loss Function is used that distinguishes between the input pair of feature vectors. The following is the formula for contrastive loss.

最后，由于此解决方案的目的不是要分类而是要在图像之间进行区分，因此使用了对比损失函数来区分输入的特征向量对。 以下是对比损失的公式。

Here is what the final Siamese model training looks like

这是最终的暹罗模型训练的样子

In the training example above, two celebrity images, Bruce Lee and Jackie Chan, are fed individually to the sister networks. The two output vectors at the end of the Sister networks are then used to compute the contrastive loss, based on the ground truth that these two images are not from the same person. This loss value is then backpropagated to both the sister networks to update their weights equally. Similarly, two images of the same actor will also be fed to the model to help it understand the similarities along with the dissimilarities.

在上面的训练示例中，两个名人图像，李小龙和成龙分别喂给了姐妹网络。 然后，基于这两个图像不是来自同一个人的事实，将Sister网络末端的两个输出矢量用于计算对比损失。 然后，将该损耗值反向传播到两个姐妹网络，以平等地更新其权重。 类似地，同一演员的两个图像也将被馈送到模型，以帮助其理解相似性和相似性。

输出值 (OUTPUT)

After uploading a new image, the model returns a list of probability scores calculated against other celebrity images, indicating how similar the two images are. Here is a sample output of celebrity look-alikes for Matt Damon.

上载新图像后，该模型将返回针对其他名人图像计算出的概率分数列表，指示这两个图像有多相似。 这是Matt Damon的名人相像的样本输出。

The model can also be used to compare the similarity between two images as shown below…

该模型还可用于比较两个图像之间的相似性，如下所示……

有问题吗？ (GOT QUESTIONS?)

If you have any questions/feedback about the post, please contact me on Linkedin here

如果您有任何疑问/该讯息有关的反馈，请在LinkedIn与我联系点击这里