一种端到端的人脸对齐方法

doi:10.19678/j.issn.1000-3428.0059225

摘要/Abstract

摘要： 现有的人脸对齐方法多数是非端到端的，中间过程需要大量的人工干预，导致人脸关键点检测的稳定性较差。为此，提出一种端到端的基于深度学习的人脸对齐方法。基于MobileNets系列网络的子模块，使用类VGG结构的方式进行搭建，将整张图片作为输入，采用基于深度可分离卷积模块进行特征提取，并运用改进的倒残差结构避免网络训练过程的梯度消失，减少特征损失。在此基础上将眼间距离作为正规化方法，在300W人脸数据集上进行测试，结果表明，与CDM、DRMF等方法相比，该方法在保证较优精度的同时，具有良好的实时性。

关键词: 人脸对齐, 人脸特征点, 特征提取, 深度可分离卷积, 倒残差结构

Abstract: Most of the existing face alignment methods are not end-to-end, and require frequent manual intervention, which leads to a reduction in their stability.To address the problem, an end-to-end face alignment method based on deep learning is proposed.The network required by this method is constructed based on the sub-modules of the MobileNet series in a structure similar to VGG.Taking the entire image as the input, the depth-wise separable convolution module is used for feature extraction, and the method employs an improved inverted residual structure to avoid the disappearance of gradients in the network training process while reducing the loss of features.The distance between eyes is taken as the basis for normalization.The designed network is tested on the 300W face dataset and compared with CDM, DRMF methods. The experimental results show that the proposed algorithm displays excellent accuracy and real-time performance.

Key words: face alignment, facial landmark, feature extraction, depth-wise separable convolution, inverted residual structure

中图分类号:

TP391

康智慧, 王全玉, 王战军. 一种端到端的人脸对齐方法[J]. 计算机工程, 2021, 47(10): 207-213.

KANG Zhihui, WANG Quanyu, WANG Zhanjun. An End-to-End Face Alignment Method[J]. Computer Engineering, 2021, 47(10): 207-213.

http://www.ecice06.com/CN/Y2021/V47/I10/207

图/表 11

20211016174341

20211016174344

20211016174348

20211016174353

20211016174357

20211016174401

20211016174404

20211016174407

20211016174411

20211016174415

20211016174419

参考文献

[1] COOTES T F, TAYLOR C J, COOPER D H, et al.Active shape models-their training and application[J].Computer Vision and Image Understanding, 1995, 61(1):38-59.
[2] COOTES T F, EDWARDS G J, TAYLOR C J, et al.Active appearance models[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6):681-685.
[3] HOWARD A G, ZHU M, CHEN B, et al.MobileNets:efficient convolutional neural networks for mobile vision applications[EB/OL].[2020-07-10].https://arxiv.org/pdf/1704.04861.pdf.
[4] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2020-07-10].https://arxiv.org/abs/1409.1556.
[5] SARAGIH J, GOECKE R.A nonlinear discriminative approach to AAM fitting[C]//Proceedings of the 11th IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2007:1-8.
[6] TZIMIROPOULOS G, PANTIC M.Optimization problems for fast AAM fitting in-the-wild[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2013:593-600.
[7] SUN Y, WANG X, TANG X, et al.Deep convolutional network cascade for facial point detection[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:3476-3483.
[8] ZHOU E, FAN H, CAO Z, et al.Extensive facial landmark localization with coarse-to-fine convolutional network cascade[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:386-391.
[9] ZHANG K, ZHANG Z, LI Z, et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[10] KOWALSKI M, NARUNIEC J, TRZCINSKI T P, et al.Deep alignment network:a convolutional neural network for robust face alignment[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:2034-2043.
[11] SANDER M, HOWARD A, ZHU M, et al.MobileNetV2:inverted residuals and linear bottlenecks[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:4510-4520.
[12] HU J, SHEN L, ALBANIE S, et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 57(10):181-196.
[13] JESORSKY O, KIRCHBERG K J, FRISCHHOLZ R W.Robust face detection using the Hausdorff distance[C]//Proceedings of IEEE AVBPAʼ01.Washington D.C., USA:IEEE Press, 2001:90-95.
[14] BELHUMEUR P N, JACOBS D W, KRIEGMAN D J, et al.Localizing parts of faces using a consensus of exemplars[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12):2930-2940.
[15] KOSTINGER M, WOHLHART P, ROTH P M, et al.Annotated facial landmarks in the wild:a large-scale, real-world database for facial landmark localization[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2011:2144-2151.
[16] BELHUMEUR P N, JACOBS D W, KRIEGMAN D J, et al.Localizing parts of faces using a consensus of exemplars[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2011:545-552.
[17] ZHU X, RAMANAN D.Face detection, pose estimation, and landmark localization in the wild[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2012:2879-2886.
[18] SAGONAS C, TZIMIROPOULOS G, ZAFEIRIOU S, et al.A semi-automatic methodology for facial landmark annotation[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:896-903.
[19] XIONG X, LA TORRE F D.Supervised descent method and its applications to face alignment[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:532-539.
[20] ASTHANA A, ZAFEIRIOU S, CHENG S, et al.Robust discriminative response map fitting with constrained local models[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:3444-3451.
[21] ZHANG J, SHAN S G, KAN M N, et al.Coarse-to-fine auto-encoder networks for real-time face alignment[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2014:1-16.
[22] CAO X, WEI Y, WEN F, et al.Face alignment by explicit shape regression[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2012:2887-2894.
[23] XIONG X, LA TORRE F D.Supervised descent method and its applications to face alignment[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:532-539.
[24] ZHU S, LI C, LOT C C, et al.Face alignment by coarse-to-fine shape searching[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:4998-5006.
[25] ZHANG Z, LUO P, LOY C C, et al.Facial landmark detection by deep multi-task learning[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2014:94-108.
[26] TIRGEORGIS G, SNAPE P, NICOLAOU M A, et al.Mnemonic descent method:a recurrent process applied for end-to-end face alignment[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:4177-4187.

选择文件类型/文献管理软件名称

选择包含的内容