基于循环生成对抗网络的含遮挡人脸识别

doi:10.19678/j.issn.1000-3428.0061124

摘要/Abstract

摘要： 人脸图像的遮挡会严重影响人脸识别准确率，当前处理带遮挡人脸识别的方法主要有丢弃法和修复法两种。丢弃法因忽略或丢弃大量遮挡区域的有效特征易造成识别准确率不高，而当前大多数修复法需要原图的相关信息，限制了其应用。针对现有含遮挡人脸识别方法存在的问题，提出一种基于循环生成对抗网络的人脸识别算法，通过利用2对生成器和判别器进行循环训练，实现遮挡人脸图像的盲修复，此过程不需要除遮挡区域外的原图信息。在此基础上，采用ResNet-50网络对修复后人脸进行识别，该网络通过跳跃连接消除深层卷积神经网络中的网络退化问题，能够降低网络训练的难度，且不会增加额外的参数和计算量。针对盲修复后人脸特征存在类内差异大和类间差异小的特性，引入一种能够量化类间距离的分类损失函数RegularFace作为识别网络损失函数。实验结果表明，与DCGAN+CNN算法相比，所提算法对不同遮挡类型和遮挡区域的人脸图像识别准确率均有所提高，当线性遮挡面积为40%时，所提算法的识别准确率提高了14.4个百分点。

关键词: 含遮挡人脸, 盲修复, 人脸识别, 循环生成对抗网络, RegularFace损失函数

Abstract: Face image occlusion has an important impact on the accuracy of face recognition.Currently, two primary methods are used to process face recognition with occlusion:discard and repair.The discarding method ignores (discards) the effective features of several occluded areas, which easily leads to a low recognition accuracy.However, most of the current restoration methods require a known original image with related information for restoration, which limits its application.Aiming at the problems existing in the existing face recognition methods with occlusion, this paper proposes a face recognition method based on cyclic Generative Adversarial Networks(GAN).This method first uses two pairs of generators and discriminators to cyclically train to realize the blind restoration of the occluded face image.This process does not require information of the original image, except the occluded area.On this basis, a ResNet-50 network recognizes the repaired face after, eliminating the network degradation problem in deep convolutional neural networks using jump connections, which can reduce the difficulty of network training without additional parameters and computation.Considering the large and small inter-class differences in face features after blind reparation, a classification loss function that can quantify the distance between classes is introduced as the recognition network loss function.Compared with the DCGAN+CNN algorithm, the results show that, the recognition accuracy of the proposed algorithm improved for face images with different occlusion types and occlusion areas, and, when the linear occlusion area is 40%, the recognition accuracy of the proposed algorithm improved by 14.4 percentage points.

Key words: face with occlusion, blind restoration, face recognition, cyclic Generative Adversarial Network(GAN), RegularFace loss function

中图分类号:

TP391

徐润昊, 程吉祥, 李志丹, 付小龙. 基于循环生成对抗网络的含遮挡人脸识别[J]. 计算机工程, 2022, 48(5): 289-296,305.

XU Runhao, CHENG Jixiang, LI Zhidan, FU Xiaolong. Face Recognition with Occlusion Based on Cyclic Generative Adversarial Networks[J]. Computer Engineering, 2022, 48(5): 289-296,305.

http://www.ecice06.com/CN/Y2022/V48/I5/289

图/表 15

20220806175107

20220806175111

20220806175115

20220806175118

20220806175122

20220806175126

20220806175205

20220806175209

20220806175213

20220806175217

20220806175222

20220806175226

20220806175230

20220806175233

20220806175237

参考文献

[1] MASI I, WU Y, HASSNER T, et al.Deep face recognition:a survey[C]//Proceedings of the 31st Conference on Graphics, Patterns and Images.Washington D.C., USA:IEEE Press, 2018:471-478.
[2] CHENG Y, JIANG B, JIA K B.A deep structure for facial expression recognition under partial occlusion[C]//Proceedings of the 10th International Conference on Intelligent Information Hiding and Multimedia Signal Processing.Washington D.C., USA:IEEE Press, 2014:211-214.
[3] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Washington D.C., USA:IEEE Press, 2016:770-778.
[4] ZHAO K, XU J Y, CHENG M M.RegularFace:deep face recognition via exclusive regularization[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:1136-1144.
[5] CHEN Y, LIU S.Deep partial occlusion facial expression recognition via improved CNN[C]//Proceedings of the Advances in Visual Computing.Berlin, Germany:Springer, 2020:451-462.
[6] ZHANG T P, TANG Y Y, FANG B, et al.Face recognition under varying illumination using gradientfaces[J].IEEE Transactions on Image Processing, 2009, 18(11):2599-2606.
[7] ZHANG W C, SHAN S G, WEN G, et al.Local gabor binary pattern histogram sequence:a novel non-statistical model for face representation and recognition[C]//Proceedings of the 10th IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2005:786-791.
[8] 李小薪, 梁荣华.有遮挡人脸识别综述:从子空间回归到深度学习[J].计算机学报, 2018, 41(1):177-207. LI X X, LIANG R H.A review for face recognition with occlusion:from subspace regression to deep learning[J].Chinese Journal of Computers, 2018, 41(1):177-207.(in Chinese)
[9] CEN F, WANG G H.Dictionary representation of deep features for occlusion-robust face recognition[J].IEEE Access, 2019, 7:26595-26605.
[10] LI Y, LIU S, YANG J, et al.Generative face completion[EB/OL].[2021-02-10].https://www.researchgate.net/publication/316272513_Generative_Face_Completion.
[11] 张雪菲, 程乐超, 白升利, 等.基于变分自编码器的人脸图像修复[J].计算机辅助设计与图形学学报, 2020, 32(3):401-409. ZHANG X F, CHENG L C, BAI S L, et al.Face image inpainting via variational autoencoder[J].Journal of Computer-Aided Design & Computer Graphics, 2020, 32(3):401-409.(in Chinese)
[12] 曹真, 杨云, 齐勇, 等.基于多损失约束与注意力块的图像修复方法[J].陕西科技大学学报, 2020, 38(3):158-165. CAO Z, YANG Y, QI Y, et al.Image inpainting method based on multi-loss constraint and attention block[J].Journal of Shaanxi University of Science & Technology, 2020, 38(3):158-165.(in Chinese)
[13] ZHANG S, HE R, SUN Z N, et al.DeMeshNet:blind face inpainting for deep MeshFace verification[J].IEEE Transactions on Information Forensics and Security, 2018, 13(3):637-647.
[14] 陈俊周, 王娟, 龚勋.基于级联生成对抗网络的人脸图像修复[J].电子科技大学学报, 2019, 48(6):910-917. CHEN J Z, WANG J, GONG X.Face image inpainting using cascaded generative adversarial networks[J].Journal of University of Electronic Science and Technology of China, 2019, 48(6):910-917.(in Chinese)
[15] 曾阳艳, 叶柏龙.基于支持向量机的人脸识别系统的研究[J].计算机工程与应用, 2008, 44(15):182-184. ZENG Y Y, YE B L.Face recognition based on support vector machines[J].Computer Engineering and Applications, 2008, 44(15):182-184.(in Chinese)
[16] 冯小荣, 惠康华, 柳振东.基于卷积特征和贝叶斯分类器的人脸识别[J].智能系统学报, 2018, 13(5):769-775. FENG X R, HUI K H, LIU Z D.Face recognition based on convolution feature and Bayes classifier[J].CAAI Transactions on Intelligent Systems, 2018, 13(5):769-775.(in Chinese)
[17] KRIZHEVSKY A, SUTSKEVER I, HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM, 2017, 60(6):84-90.
[18] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2014:2672-2680.
[19] LATA K, DAVE M, NISHANTH K N.Image-to-image translation using generative adversarial network[C]//Proceedings of the 3rd International Conference on Electronics, Communication and Aerospace Technology.Washington D.C., USA:IEEE Press, 2019:186-189.
[20] WEN Y D, ZHANG K P, LI Z F, et al.A discriminative feature learning approach for deep face recognition[C]//Proceedings of Conference on Computer Vision.Berlin, Germany:Springer, 2016:499-515.
[21] MIKOLOV T, KOMBRINK S, BURGET L, et al.Extensions of recurrent neural network language model[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing.Washington D.C., USA:IEEE Press, 2011:5528-5531.
[22] LIU W Y, WEN Y D, YU Z D, et al.SphereFace:deep hypersphere embedding for face recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:6738-6746.
[23] WANG H, WANG Y T, ZHOU Z, et al.CosFace:large margin cosine loss for deep face recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:5265-5274.
[24] DENG J K, GUO J, XUE N N, et al.ArcFace:additive angular margin loss for deep face recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:4685-4694.
[25] YI D, LEI Z, LIAO S C, et al.Learning face representation from scratch[EB/OL].[2021-02-10].https://arxiv.org/abs/1411.7923v1.
[26] ELMAHMUDI A, UGAIL H.Deep face recognition using imperfect facial data[J].Future Generation Computer Systems, 2019, 99:213-225.

选择文件类型/文献管理软件名称

选择包含的内容