基于深度特征抑制的遮挡人脸识别网络

doi:10.19678/j.issn.1000-3428.0068289

摘要/Abstract

摘要：

人脸识别技术是公安侦查中人证核验的关键技术之一。尽管现有算法在无遮挡人脸识别上都能达到较高的识别精度, 但当人脸被遮挡时, 使得有效的人脸特征丢失, 导致识别精度大幅下降。针对上述问题, 提出一种基于深度特征抑制的遮挡人脸识别网络, 通过遮挡人脸自适应地生成特征掩码, 利用特征掩码抑制深层特征图中因遮挡损坏的特征, 最后根据抑制后的特征完成人脸识别。为了提升抑制后特征的辨别力, 在训练环节通过孪生网络结构将遮挡人脸与对应无遮挡人脸的深度特征进行度量学习。同时利用不同层次的特征信息, 构建特征金字塔网络(FPN)和自适应特征融合模块对人脸的多尺度特征信息进行提取, 对其中包含特征信息较多的特征层赋予更大的融合权重, 从而增强特征的表征能力。实验结果表明, 该方法具有较好的鲁棒性, 其中在LFW数据集和LFW口罩遮挡数据集上的准确率分别达到了99.50%和98.42%, 在AR数据集4个实验设置上的准确率分别达到了100%、100%、99.86%和99.02%, 优于目前的主流算法。

关键词: 人脸识别, 遮挡人脸识别, 自适应特征融合, 特征掩码, 度量学习

Abstract:

Facial recognition technology is a key technology for verifying personal evidence in public security investigations. Although existing algorithms can achieve high recognition accuracy in unobstructed face recognition, effective facial features are lost when a face is occluded, resulting in a significant decrease in recognition accuracy. Hence, an occluded face recognition network based on deep feature suppression is proposed to address these issues. The network adaptively generates feature masks based on occluded faces, suppresses features damaged by occlusion in deep feature maps through feature masks, and uses the suppressed features to complete face recognition. To improve the discrimination of suppressed features, a twin network structure is used in the training phase to measure and learn the depth features of the occluded and corresponding unobstructed faces. Simultaneously, to fully utilize different levels of feature information, a Feature Pyramid Network (FPN) and an adaptive feature fusion module are constructed to extract multiscale feature information from faces. The feature layers containing more feature information are assigned greater fusion weights, thereby enhancing the representation abilities of the features. The experimental results show that the proposed method has good robustness, with accuracy rates of 99.50% and 98.42% for the LFW and LFW mask occlusions in the dataset, respectively, and 100%, 100%, 99.86%, and 99.02% for the four experimental settings in the AR dataset, respectively, surpassing those of current mainstream algorithms.

Key words: face recognition, occluded face recognition, adaptive feature integration, feature mask, metric learning

王富平, 刘鸿玮, 张锲石, 段冠庄. 基于深度特征抑制的遮挡人脸识别网络[J]. 计算机工程, 2024, 50(8): 259-269.

Fuping WANG, Hongwei LIU, Qieshi ZHANG, Guanzhuang DUAN. Occluded Face Recognition Network Based on Deep Feature Suppression[J]. Computer Engineering, 2024, 50(8): 259-269.

https://www.ecice06.com/CN/Y2024/V50/I8/259

图/表 20

图1 模型的总体框架

Fig.1 Overall framework of the model

图2 LResNet50E-IR网络结构

Fig.2 LResNet50E-IR network structure

图3 特征金字塔

Fig.3 Feature pyramid

图4 自适应特征融合

Fig.4 Adaptive feature integration

图5 掩码生成模块

Fig.5 Mask generation module

图6 CASIA-WebFace-OCC数据集部分样本

Fig.6 Partial samples of CASIA-WebFace-OCC dataset

图7 LFW数据集部分样本

Fig.7 Partial samples of LFW dataset

图8 LFW围巾遮挡数据集部分样本

Fig.8 Partial samples of LFW scarf occluded dataset

图9 LFW中心遮挡数据集部分样本

Fig.9 Partial samples of LFW central occluded dataset

图10 LFW口罩遮挡数据集部分样本

Fig.10 Partial samples of LFW mask occluded dataset

图11 LFW模糊处理的部分样本

Fig.11 Partial samples of LFW fuzzy processing

图12 AR Face Dataset部分样本

Fig.12 Partial samples of AR Face Dataset

参考文献 31

1	WRIGHT J, YANG A Y, GANESH A, et al. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 210- 227. doi: 10.1109/TPAMI.2008.79
2	WAGNER A, WRIGHT J, GANESH A, et al. Towards a practical face recognition system: robust registration and illumination by sparse representation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 597-604.
3	DENG W H, HU J N, GUO J. Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1864- 1870. doi: 10.1109/TPAMI.2012.30
4	唐娴, 黄军伟. 低秩鲁棒性主成分分析的遮挡人脸识别. 南京理工大学学报, 2017, 41(4): 460- 465. URL
	TANG X, HUANG J W. Occlusion face recognition based on robust principal component analysis and low rank. Journal of Nanjing University of Science and Technology, 2017, 41(4): 460- 465. URL
5	王丽娟, 李可爱, 郝志峰, 等. 基于低秩表示的鲁棒回归模型. 计算机工程, 2020, 46(1): 74-79, 86. URL
	WANG L J, LI K A, HAO Z F, et al. Robust regression model based on low rank representation. Computer Engineering, 2020, 46(1): 74-79, 86. URL
6	ZHOU Z H, WAGNER A, MOBAHI H, et al. Face recognition with contiguous occlusion using Markov random fields[C]//Proceedings of the 12th IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2009: 1050-1057.
7	SUN Y, WANG X, TANG X. Deep learning face representation by joint identification-verification[EB/OL]. [2023-07-20]. https://arxiv.org/abs/1406.4773v1.
8	CHAN T H, JIA K, GAO S, et al. PCANet: a simple deep learning baseline for image classification?. IEEE Transactions on Image Processing, 2015, 24(12): 5017- 5032. doi: 10.1109/TIP.2015.2475625
9	LIU W Y, WEN Y D, YU Z D, et al. SphereFace: deep hypersphere embedding for face recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 212-220.
10	WANG H, WANG Y T, ZHOU Z, et al. CosFace: large margin cosine loss for deep face recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 5265-5274.
11	DENG J K, GUO J, XUE N N, et al. ArcFace: additive angular margin loss for deep face recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 4690-4699.
12	SONG L X, GONG D H, LI Z F, et al. Occlusion robust face recognition based on mask learning with pairwise differential Siamese network[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 773-782.
13	QIU H B, GONG D H, LI Z F, et al. End2End occluded face recognition by masking corrupted features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10): 6939- 6952. doi: 10.1109/TPAMI.2021.3098962
14	徐润昊, 程吉祥, 李志丹, 等. 基于循环生成对抗网络的含遮挡人脸识别. 计算机工程, 2022, 48(5): 289-296, 305. URL
	XU R H, CHENG J X, LI Z D, et al. Face recognition with occlusion based on cyclic generative adversarial networks. Computer Engineering, 2022, 48(5): 289-296, 305. URL
15	CHEN S, LIU Y, GAO X, et al. MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices[M]. Berlin, Germany: Springer, 2018: 428-438.
16	HAO S Z, CHEN C F, CHEN Z F, et al. A unified framework for masked and mask-free face recognition via feature rectification[C]//Proceedings of IEEE International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2022: 726-730.
17	YUAN G, ZHENG H C, DONG J Y. MSML: enhancing occlusion-robustness by multi-scale segmentation-based mask learning for face recognition. Artificial Intelligence, 2022, 36(3): 3197- 3205.
18	KIM M, JAIN A K, LIU X M. AdaFace: quality adaptive margin for face recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 18750-18759.
19	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 2117-2125.
20	HUANG G B, MATTAR M A, BERG T L, et al. Labeled faces in the wild: a database for studying face recognition in unconstrained environments[EB/OL]. [023-07-20]. https://vis-www.cs.umass.edu/lfw/lfw.pdf.
21	MARTINEZ A R, BENAVENTE R. The AR face database. Computer Vision, 1998, 24, 318- 323.
22	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6): 84- 90. doi: 10.1145/3065386
23	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [023-07-20]. https://arxiv.org/pdf/1409.1556v6.pdf.
24	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
25	YI D, LEI Z, LIAO S, et al. Learning face representation from scratch[EB/OL]. [023-07-20]. https://arxiv.org/pdf/1411.7923.pdf.
26	ALANSARI M, HAY O A, JAVED S, et al. GhostFaceNets: lightweight face recognition model from cheap operations. IEEE Access, 2023, 11, 35429- 35446.
27	WENG R, LU J, TAN Y P. Robust point set matching for partial face recognition. IEEE Transactions Image Process, 2016, 25(3): 1163- 1176.
28	YANG J, LUO L, QIAN J J, et al. Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(1): 156- 171.
29	WAN W T, CHEN J S. Occlusion robust face recognition based on mask learning[C]//Proceedings of 2017 IEEE International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2017: 3795-3799.
30	CHEN W P, GAO Y S. Recognizing partially occluded faces from a single sample per class using string-based matching[M]. Berlin, Germany: Springer, 2010.
31	MCLAUGHLIN N, JI-MING, CROOKES D. Largest matching areas for illumination and occlusion robust face recognition. IEEE Transactions on Cybernetics, 2017, 47(3): 796- 808.

[1]	吴冠荣, 李元祥, 王艺霖, 陆雨寒, 陈秀华. 基于对比学习的小样本金属表面损伤分类[J]. 计算机工程, 2024, 50(3): 36-43.
[2]	李倩, 向海昀, 张玉婷, 甘昀, 廖浩德. 结合高斯滤波与MASK的G-MASK人脸对抗攻击[J]. 计算机工程, 2024, 50(2): 308-316.
[3]	石进, 徐杨, 曹斌. 基于自适应三线性池化网络的细粒度图像分类[J]. 计算机工程, 2023, 49(5): 239-246,254.
[4]	郝阿香, 贾郭军. 结合注意力与批特征擦除的行人重识别模型[J]. 计算机工程, 2022, 48(7): 270-276,306.
[5]	徐润昊, 程吉祥, 李志丹, 付小龙. 基于循环生成对抗网络的含遮挡人脸识别[J]. 计算机工程, 2022, 48(5): 289-296,305.
[6]	吴鹏翔, 李凡长. 一种基于群等变卷积的度量元学习算法[J]. 计算机工程, 2022, 48(2): 72-78.
[7]	叶萌, 杨娟, 汪荣贵, 薛丽霞, 李懂. 基于特征聚合网络的小样本学习方法[J]. 计算机工程, 2021, 47(3): 77-82.
[8]	尚桠朝, 孟令军. 基于多特征和尺度估计的KCF_MTSA算法[J]. 计算机工程, 2021, 47(3): 102-108,116.
[9]	黄义妨, 魏丹丹, 武淼, 李慧斌, 郭勐. 面向不同传感器与复杂场景的人脸识别系统防伪方法综述[J]. 计算机工程, 2021, 47(12): 1-18.
[10]	柯鹏飞, 蔡茂国, 吴涛. 基于改进卷积神经网络与集成学习的人脸识别算法[J]. 计算机工程, 2020, 46(2): 262-267,273.
[11]	王丽娟, 李可爱, 郝志峰, 蔡瑞初, 尹明. 基于低秩表示的鲁棒回归模型[J]. 计算机工程, 2020, 46(1): 74-79,86.
[12]	马钰锡,谭励,董旭,于重重. 面向VTM的交互式活体检测算法[J]. 计算机工程, 2019, 45(3): 256-261.
[13]	张裕平, 龚晓峰, 雒瑞森. 基于稀疏化双向二维主成分分析的人脸识别[J]. 计算机工程, 2019, 45(12): 232-236.
[14]	魏明俊,许道云,徐梦珂. 于均方差度量分块的自动加权稀疏表示算法[J]. 计算机工程, 2017, 43(5): 174-178,184.
[15]	廖周宇,王钰婷,谢晓兰,刘建明. 基于粒子群优化的支持向量机人脸识别[J]. 计算机工程, 2017, 43(12): 248-254.

选择文件类型/文献管理软件名称

选择包含的内容