论文名字:AN AUTOMATIC FRAMEWORK FOR EXAMPLE-BASED VIRTUAL MAKEUP

源自英特尔中国研究所 :http://tech.hqew.com/news_373212

论文地址:https://ieeexplore.ieee.org/document/6738660

现存方法需要很多用户输入来调整面部基准点(landmarks)且不能移除用户脸上瑕疵.此文章提出的方法能从很多个妆容里选.用现存算法来检测面部基准点(face landmarks),并用皮肤颜色基于聚类之高斯混合模型(Gaussian Mixture model) 的分割来调整基准点 .接着.皮肤颜色区域被分为三层,妆容用不同方法迁移到不同层..

索引关键词: virtual makeup, face landmark detection

有关的研究,

[1] is an example-based makeup transfer algorithm.  But it deeply relies on “before-and-after” pairs, which is taken in the same position, expression and lighting environment.  In the pre-processing steps, it needs many user inputs to select special areas, such as eye brows, moles,
and freckles. [2] improves [1] with texture separation and blending example makeup with input face image.  But it also needs users to adjust 88 face landmarks.  The makeup procedure in our framework is similar to [2], but we improve it with automatic landmark adjustment.In additional research on virtual makeup is on skin color enhancement, such as [5][6].   [5] also uses “before-andafter” pairs as examples for foundation synthesis.  [6] utilizes a physical model to extract hemoglobin and melanin components.  It also need user to select skin area which includes spots or moles.  Furthermore,
[4] is a makeup suggestion system to optimize the best makeup for user with a professional “before-and-after” pair dataset. However, since the limited dataset, the cosmetic effects are also limited. There are also some studies on face beautification.
[3] is an interesting study to improve face attractiveness with structure adjustment, which adjusts the input face structure with training lots of attractive faces.
Other studies improve face attractiveness from textures [7][8][9], which remove flaws, moles and wrinkles from faces.   Some commercial softwares, such as Taaz [10] and Modi Face [11], provide users with virtual makeup on face photos by simulating the effects of specific cosmetics.But the manual adjustment of face landmark still limits the user experience in this kind of software usages.

这篇提出的方法能解决两个难题:自动检测face landmark,example和输入图间的妆容合成

First, we utilize state-of-the-art ASM face landmark detection algorithm to locate key points in the face, and automatically adjust skin areas with Gaussian mixture models (GMM). Second, we use bilateral filtering to skin areas to remove flaws, wrinkles, etc.. At last, we use similar makeup technique in [2] to transfer the makeup.

文章剩下部分是这样的. 第二部分:we introduce the face landmark detection method and GMM-based auto-adjustment of skin area. Skin smoothing and makeup transfer methods are introduced in the following section 3 and 4. In section 5, some results are presented. In the last section, we draw a conclusion.

第二部分: DIGITAL MAKEUP FRAMEWORK

第一步,对输入图自动面部landmark对齐,第二部,skin smoothing based on skin area mask is generated.第三步,smoothed 输入图和提供妆容的样品图间的面部对齐.第四部,分层(layer decomposition),输入图和样品图都分解为3层:脸部结构层,皮肤细节层,,颜色层.不同层运用不同迁移方法来达到更好效果.第五步,所有处理完的层合并.架构如下图

第二部分的每步解析:

2.1.面部自动对齐

Active Shape Model是面部基准检测常用方法,但可能会因光照条件变化和颜色差异不能准确获得基准点,需要用户手动调整.在作者看来,很多妆容迁移算法都是只处理皮肤区域,而不处理眼睛那些,所以作者用了高斯混合模型(GMM)去调整(refine)皮肤区域
GMM需要在所有区域中采样,作者运用了[9]中的取样方法,运用66个面部特征点来选择皮肤区域
在取样完成后,作者利用GMM[11]来给像素颜色分布建模,
接着用Bayesian分割算法来给GMM统计完的人像分为皮肤和非皮肤区域.是多尺度随机区域(MSRF)和[12]中描述的estimator联合起来使用的
分割完以后.我们得到一个更准确的皮肤和非皮肤区域的分界线.然后再用nearest-neighborhood point来精化分界线.
下巴上的点我们不会改,因为有些人是双下巴.
眉毛上的点也不会变,因为有人有刘海.

/*所以,面部检测是用Active Shape Model这个方法,然后再用高斯混合模型(GMM)去调整面部皮肤区域.具体是用[9]这篇论文里的来取样,运用66个面部特征点来选择皮肤区域,然后再用高斯混合模型(GMM)来给人像的像素颜色建模,高斯模型详见[11]

接着再用Bayesian分割算法来给以上处理好的人像分割成皮肤和非皮肤(眼睛,眉毛那些),具体为利用多尺度随机区域(MSRF)和sequential maximum a posteriori (SMAP)estimator,详见[12],然后就分界得很清楚了,但是还要用nearest-neighborhood point来精化分界线,其中下巴和眉毛都不会变

[9]An Algorithm for Automatic Skin Smoothing in Digital Portraits

[11] C. A. Bouman, “CLUSTER: an unsupervised algorithm for modeling Gaussian mixtures,” https://engineering.pudue.edu/~bouman/software/cluster/manual.p df.

[12] C.A. Bouman and M. Shapiro, “A multiscale random field model for Bayesian image segmentation,” IEEE Trans. Image Processing, vol. 3, no. 2, pp.162-177, March 1994.

2.2.磨皮

Image filtering technology 可以去除瑕疵,但是也会损失细节.我们折中选了 Bilateral filter [11] 来磨皮.

With the skin mask obtained in 2.1 we use a poison image editing method [13] to mosaic input and filtered images together.

2.3.Digital makeup

这个步骤里,样本和输入见先特征对齐,然后图片会被分为三层,皮肤结构层,皮肤细节层和颜色层.最终颜色是在不同层中分别迁移并合并的.而面部特征对齐是通过薄板样条插值(Thin Plate Spline)来把样品变形到输入的,变形控制点是2.1里的结果

分层细节:

The input image I and the warped example image E are
decomposed into color and lightness layer by
converting them to CIE L*a*b* color space.
The L* channel is considered as lightness layer
and a* and b* the color layer.  The lightness
layer is decomposed into face structure and skindetail layers.  While several recent computationalphotography techniques can perform image decomposition,
the weighted least squares (WLS) [14] is one of the
state-ofthe-art methods.  It was originally proposedfor general edge-preserving image decomposition.

3.结果

美妆样品是从专业化妆手册:[15]里找的,输入图是数据集[16]里找的;

因为磨皮在妆容迁移前就做了,比起其他软件,我们的方法对皮肤瑕疵解决的更好,比如鱼尾纹,

参考文献

11. REFERENCES [1] W.S. Tong, C.K. Tang, M.S. Brown and Y.Q. Xu, “ExampleBased Cosmetic Transfer,” 15th Pacific Conference on Computer Graphics and Applications, 2007. [2] D. Guo and T. Sim, “Digital Face Makeup by Example”, In Proc.  CVPR, 2009. [3] T, Leyvand, D. Cohen-Or, G. Dror, and D. Lischinski.  “Datadriven enhancement of facial attractiveness”.  ACM Trans. Graphics, 27(3):1-9, 2008. [4] K. Schebaum, T. Ritschel, M. Hullin, T. Thormahlen, V.Blanz, H.P. Seidel, “Computer-Suggested Facial Makeup”,  In Proc. EUROGRAPHICS, 2011, Volum 30(2011). [5] N. Ojima, K. Yoshiba, O.Osanai, and S. Akasaki.  “Image synthesis of cosmetic applied skin based on  optical properties of foundation layers”, In Proc. ICIS, pages 467-468, 1999. [6] N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, “Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin”,  ACM Trans. Graphics, 22(3):770-779, 2003. [7] M. Brand and P. Pletscher.  “A confitional random field for automatic photo editing”.  In Proc. CVPR, 2008. [8] G. Guo, “Digital Anti-Aging in Face Images”,  In Proc. ICCV, 2011. [9] C. Lee, M.T. Schramm, M. Boutin, J.P. Allebach,  “An Algorithm for Automatic Skin Smoothing in Digital Portraits”,  In Proc. ICIP, 2009. [10] S. Milborrow and F. Nicolls.  “locating facial features with an extended active shape model”,  In Proc. ECCV, 2008. [11] C. A. Bouman, “CLUSTER: an unsupervised algorithm for modeling Gaussian mixtures,” https://engineering.pudue.edu/~bouman/software/cluster/manual.p df. [12] C.A. Bouman and M. Shapiro, “A multiscale random field model for Bayesian image segmentation,” IEEE Trans. Image Processing, vol. 3, no. 2, pp.162-177, March 1994. [13] F.Bookstein.  Principal warps: Thin-plate splines and the decomposition of deformations.  IEEE Trans. Pattern Analysis and Machine Intelligence, 11(6):567-585, 1989. [14] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski.  “Edgepreserving decompositions for multi-scale tone and detail manipulation”,  ACM Trans. Graphics, 27(3):1-10, 2008. [15] F.Nars. Makeup your Mind. PowerHouse Books, 2004. [16] “Caltech faces 1999 (front),” http://www.vision.caltech.edu/html-files/archive.html.
3210

读美妆论文AN AUTOMATIC FRAMEWORK FOR EXAMPLE-BASED VIRTUAL MAKEUP相关推荐

人像美妆---妆容迁移算法研究(Makeup transfer)
对于人像美妆算法,现在的美妆相机.玩美彩妆之类的app已经做的比较成熟了,但是具体算法,基本网络上是杳无可查,今天本人介绍一种自动的人像美妆算法----(Makeup Transfer)妆容迁移妆容 ...
java基于ssm开发的美妆商城化妆品电商系统有论文
简介 Java基于ssm(可以转springboot项目哦)开发的美妆商城系统,主要是卖化妆品的系统,用户可以浏览商品,加入购物车,下单,在个人中心管理自己的订单.管理员可以管理自己的商品,发布商品, ...
Springboot美妆网站的设计与实现毕业设计-附源码211539
摘要近年来,随着移动互联网的快速发展,电子商务越来越受到网民们的欢迎,电子商务对国家经济的发展也起着越来越重要的作用.简单的流程.便捷可靠的支付方式.快捷畅通的物流快递.安全的信息保护都使得电子商 ...
Springboot+美妆网站的设计与实现毕业设计-附源码211539
Springboot美妆网站的设计与实现摘要近年来,随着移动互联网的快速发展,电子商务越来越受到网民们的欢迎,电子商务对国家经济的发展也起着越来越重要的作用.简单的流程.便捷可靠的支付方式.快捷 ...
当人工智能变成美妆博主……
全文共1874字,预计学习时长6分钟来源:Pexels 2017年,美容业市场估值5324.3亿美元,且这个数字正沿正轨快速上升.根据同一份报告,到2023年,预计市值将达8056.1亿美元.此行业 ...
图像美妆算法---自动祛斑算法研究
目前,市面上有不少的人脸美妆软件,大多数也都有自动去除皮肤斑点痘痘的功能,网上对于手动祛斑的算法已有了相关实现,但是,自动祛斑的算法却少之又少,今天,在这里我简单讲一下这方面的开发经验. 对于自动祛斑 ...
近期有哪些值得读的推荐系统论文？来看看这份私人阅读清单
在碎片化阅读充斥眼球的时代,越来越少的人会去关注每篇论文背后的探索和思考.在这个栏目里,你会快速 get 每篇精选论文的亮点和痛点,时刻紧跟 AI 前沿成果. 本期「本周值得读」关注的是「推荐系统」领 ...
2021数模美赛论文提交具体规则和注意事项汇总！
针对美赛论文提交与竞赛规则,工作室为小伙伴们整理了如下注意事项,请小伙伴们仔细阅读,提前做好准备~ 目录一.竞赛开始前二.竞赛期间三.竞赛结束后一.竞赛开始前 1. 题目查看竞赛问题将在20 ...
【报告分享】2020美妆行业短视频营销报告.pdf（附下载链接）
今天分享的报告是微播易于2020年2月发布的<2020美妆行业短视频营销报告.pdf>,报告包含三大部分:1.美妆行业短视频营销现状:2.美妆类自媒体发展趋势:3.美妆行业短视频营销机会: ...

读美妆论文AN AUTOMATIC FRAMEWORK FOR EXAMPLE-BASED VIRTUAL MAKEUP