基于多层次自注意力网络的人脸特征点检测

doi:10.19678/j.issn.1000-3428.0066927

摘要/Abstract

摘要：

人脸特征点检测是人脸图像处理的关键步骤之一，常用检测方法是基于深度神经网络的坐标回归方法，具有处理速度快的优点，但是用于回归的高层次网络特征丢失空间结构信息，且缺乏细粒度表征能力，导致检测精度降低。针对该问题，提出一种基于多层次自注意力网络的人脸关键点检测算法。为提取更具有细粒度表征能力的图像语义特征，构建基于自注意力机制的多层次特征融合模块，实现高层次高语义信息特征和低层次高空间信息特征的跨层次特征融合。在此基础上，设计一种多任务学习人脸特征点检测定位与人脸姿态角估计的训练方式，优化网络对人脸整体朝向姿态的估计，以提升特征点检测的准确性。在人脸特征点主流数据集300W和WFLW上的实验结果表明，与SAAT、AnchorFace等方法相比，该方法有效提升网络的检测精度，标准平均误差指标分别为3.23%和4.55%，相较于基线模型降低0.37和0.59个百分点，在WFLW数据集上错误率指标为3.56%，相较于基线模型降低了2.86个百分点，能够提取更具鲁棒性和细粒度的表达特征。

关键词: 人脸特征点检测, 卷积神经网络, 自注意力机制, 特征融合, 多任务学习, 深度学习

Abstract:

Facial landmark detection, a key step in facial image processing, is commonly performed using the coordinate regression method based on deep neural networks, which has the advantage of fast processing speed. However, the high-level network features used for regression lose spatial structural information and lack fine-grained representation ability, leading to a decrease in detection accuracy. Therefore, a facial landmark detection algorithm based on multi-level self-attention network is proposed to address this issue. To extract image semantic features with finer granularity representation ability, a multi-level feature fusion module based on self-attention mechanism is constructed to achieve cross-level fusion of high-level semantic and low-level spatial information features. On this basis, a training method for multi-task learning of facial landmark detection and localization, as well as facial pose angle estimation, is designed to optimize the network estimation of the overall orientation and pose of the face, thereby improving the accuracy of landmark detection. The experimental results on mainstream facial landmark datasets 300W and WFLW show that compared with methods such as SAAT and AnchorFace, the proposed method effectively improves network detection accuracy, achieving a standard average error of 3.23% and 4.55%, respectively, which are 0.37 and 0.59 percentage points lower than the baseline model. The error rate indicator on the WFLW dataset is 3.56%, which is 2.86 percentage points lower than the baseline model, demonstrating that the proposed method can extract more robust and fine-grained expression features.

Key words: facial landmark detection, Convolutional Neural Network(CNN), self-attention mechanism, feature fusion, multi-task learning, deep learning

徐浩宸, 刘满华. 基于多层次自注意力网络的人脸特征点检测[J]. 计算机工程, 2024, 50(2): 239-246.

Haochen XU, Manhua LIU. Facial Landmark Detection Based on Hierarchical Self-Attention Network[J]. Computer Engineering, 2024, 50(2): 239-246.

https://www.ecice06.com/CN/Y2024/V50/I2/239

图/表 9

图1 基于多层次自注意力网络的人脸特征点检测算法流程

Fig.1 Procedure of facial landmark detection algorithm based on hierarchical self-attention network

图2 人脸特征点示意图

Fig.2 Schematic diagram of facial landmark

图3 多层次自注意力网络结构

Fig.3 Structure of hierarchical self-attention network

图4 主干网络结构

Fig.4 Structure of backbone network

图5 不同算法的人脸特征点检测结果可视化对比

Fig.5 Visual results comparison of facial landmark detection using different algorithms

参考文献 27

1	于明, 钟元想, 王岩. 人脸微表情分析方法综述. 计算机工程, 2023, 49(2): 1- 14. doi: 10.19678/j.issn.1000-3428.0065790
	YU M, ZHONG Y X, WANG Y. A survey of facial micro-expression analysis methods. Computer Engineering, 2023, 49(2): 1- 14. doi: 10.19678/j.issn.1000-3428.0065790
2	COOTES T F, EDWARDS G J, TAYLOR C J. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6): 681- 685. doi: 10.1109/34.927467
3	CRISTINACCE D, COOTES T F. Feature detection and tracking with constrained local models[C]//Proceedings of the British Machine Vision Conference. Edinburgh, UK: British Machine Vision Association, 2006: 1-10.
4	NEWELL A, YANG K Y, DENG J. Stacked Hourglass networks for human pose estimation[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 483-499.
5	DOLLAR P, WELINDER P, PERONA P. Cascaded pose regression[C]//Proceedings of Computer Society Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2010: 1-10.
6	SUN Y, WANG X G, TANG X O. Deep convolutional network cascade for facial point detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2013: 1-10.
7	GUO X J, LI S Y, YU J K, et al. PFLD: a practical facial landmark detector[EB/OL]. [2023-01-10]. https://arxiv.org/abs/1902.10859.
8	FENG Z H, KITTLER J, AWAIS M, et al. Wing loss for robust facial landmark localisation with convolutional neural networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2018: 2235-2245.
9	ZHANG Z P, LUO P, LOY C C, et al. Facial landmark detection by deep multi-task learning[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 94-108.
10	YANG J, LIU Q S, ZHANG K H. Stacked Hourglass network for robust facial landmark localisation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition Workshops. Washington D. C. , USA: IEEE Press, 2017: 79-87.
11	DENG J K, TRIGEORGIS G, ZHOU Y X, et al. Joint multi-view face alignment in the wild. IEEE Transactions on Image Processing, 2019, 28(7): 3636- 3648. doi: 10.1109/TIP.2019.2899267
12	WU W Y, QIAN C, YANG S, et al. Look at boundary: a boundary-aware face alignment algorithm[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2018: 2129-2138.
13	WANG X Y, BO L F, LI F X. Adaptive Wing loss for robust face alignment via heatmap regression[C]//Proceedings of International Conference on Computer Vision. Washington D. C. , USA: IEEE Press, 2019: 6971-6981.
14	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2017: 2117-2125.
15	HARIHARAN B, ARBELáEZ P, GIRSHICK R, et al. Hypercolumns for object segmentation and fine-grained localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2015: 447-456.
16	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2015: 3431-3440.
17	XIE S N, TU Z W. Holistically-nested edge detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C. , USA: IEEE Press, 2015: 1395-1403.
18	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.
19	ZHANG K P, ZHANG Z P, LI Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 2016, 23(10): 1499- 1503. doi: 10.1109/LSP.2016.2603342
20	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2016: 770-778.
21	HE K M, ZHANG X Y, REN S Q, et al. Identity mappings in deep residual networks[C]//Proceedings of the 14th European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 630-645.
22	SAGONAS C, ANTONAKOS E, TZIMIROPOULOS G, et al. 300 faces in-the-wild challenge: database and results. Image and Vision Computing, 2016, 47, 3- 18. doi: 10.1016/j.imavis.2016.01.002
23	XU Z X, LI B H, YUAN Y, et al. AnchorFace: an anchor-based facial landmark detector across large poses[C]//Proceedings of the AAAI Conference on Artificial Intelligence. [S. l.]: AAAI Press, 2021: 3092-3100.
24	ZHAO Y, LIU Y F, SHEN C H, et al. MobileFAN: Transferring deep hidden representation for face alignment. Pattern Recognition, 2020, 100, 107114. doi: 10.1016/j.patcog.2019.107114
25	SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2019: 5693-5703.
26	YANG H L, YU J Y, CHENG P J, et al. LDDMM-Face: large deformation diffeomorphic metric learning for flexible and consistent face alignment[EB/OL]. [2023-01-10]. https://arxiv.org/abs/2108.00690.
27	ZHU C C, LI X Q, LI J D, et al. Improving robustness of facial landmark detection by defending against adversarial attacks[C]//Proceedings of International Conference on Computer Vision. Washington D. C. , USA: IEEE Press, 2021: 11751-11760.

[1]	谭巨全, 王然. 特征融合下田径录像3D人体动作DTW捕捉算法[J]. 计算机工程, 2024, 50(7): 71-78.
[2]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.
[3]	张洋, 刘畅, 李少青. 基于可控制性度量的图神经网络门级硬件木马检测方法[J]. 计算机工程, 2024, 50(7): 164-173.
[4]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[5]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[6]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[7]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[8]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.
[9]	王晋涛, 秦昂, 张元, 陈一飞, 王廷凤, 谢承霖, 邹刚. 基于注意力增强与特征融合的中文医学实体识别[J]. 计算机工程, 2024, 50(7): 324-332.
[10]	徐明亮, 李芳媛, 马浩然, 何飞. 大规模神经记录的峰电位聚类算法(特邀)[J]. 计算机工程, 2024, 50(6): 1-34.
[11]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[12]	李亚康, 陈刚. 小角中子散射物理模型自动化筛选[J]. 计算机工程, 2024, 50(6): 56-64.
[13]	李子杰, 周菊香, 韩晓瑜, 甘健侯, 鹿泽光, 王俊. 序列特征与学习过程融合的知识追踪模型[J]. 计算机工程, 2024, 50(6): 77-85.
[14]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.
[15]	于洋, 孙芳芳, 吕华, 李扬, 王晓民. 基于多尺度时空注意力网络的微表情检测方法[J]. 计算机工程, 2024, 50(6): 228-235.

选择文件类型/文献管理软件名称

选择包含的内容