结合特征融合和任务分组的人脸属性识别

doi:10.19678/j.issn.1000-3428.0066051

摘要/Abstract

摘要：

针对现有人脸属性识别模型存在的特征提取不足、划分属性组时未充分考虑属性相关性强弱等问题，为了实现对多个人脸属性的同时识别，建立一种结合特征融合和任务分组的人脸属性识别模型。在参数共享部分，通过多尺度特征融合模块将不同尺度的特征进行融合增强特征相关性，同时设计基于中心核对齐和谱聚类的属性分组策略进行属性识别，通过中心核对齐方法度量属性的相关程度，并以此为基础使用谱聚类算法得到属性的合理分组，使同一组内的属性相关性尽可能大，提高属性识别准确率。在分支部分，使用注意力机制加强对目标区域的关注，并通过不确定性加权方法表示任务间的相对难度，自动调整每组任务损失之间的相对权重，进一步优化模型性能。在CelebA公开数据集上的实验结果表明，所提模型的分类准确率相较于MOON、GNAS和DMM-CNN模型提升了0.78、0.09和0.02个百分点，参数量仅为上述对比模型的1.10%、17.08%和0.37%。

关键词: 人脸属性识别, 特征融合, 中心核对齐, 属性分组, 注意力机制

Abstract:

To address the issues of insufficient feature extraction and inadequate consideration of attribute correlation in existing face attribute recognition models, a face attribute recognition model combining feature fusion and task grouping is proposed to achieve the simultaneous recognition of multiple face attributes. In the parameter sharing part, features of different scales are fused using multiscale feature fusion modules to enhance feature relevance. Simultaneously, an attribute grouping strategy based on Centered Kernel Alignment-Spectral Clustering(CKA-SC) is designed for attribute identification. The extent of correlation of the attributes is measured using CKA. Based on this, the SC algorithm is utilized to obtain reasonable attribute groups, maximizing the relevance of attributes within the same group to improve attribute recognition accuracy. In the branch part, attention mechanisms are employed to enhance the focus on target regions, and the Uncertainty Weighting(UW) method is used to represent the relative difficulty between tasks. This automatically adjusts the relative weights between the losses of each task group to further optimize the model performance. Experimental results on the publicly available CelebA dataset demonstrate that the proposed model achieves classification accuracy improvements of 0.78, 0.09, and 0.02 percentage points compared with the MOON, GNAS, and DMM-CNN models, respectively, with parameter counts accounting for only 1.10%, 17.08%, and 0.37% of the mentioned comparison models.

Key words: face attribute recognition, feature fusion, Centered Kernel Alignment(CKA), attribute grouping, attention mechanism

刘英芳, 王松, 马亚彤. 结合特征融合和任务分组的人脸属性识别[J]. 计算机工程, 2023, 49(11): 211-219.

Yingfang LIU, Song WANG, Yatong MA. Face Attribute Recognition Combining Feature Fusion and Task Grouping[J]. Computer Engineering, 2023, 49(11): 211-219.

http://www.ecice06.com/CN/Y2023/V49/I11/211

图/表 12

图1 Slim-CNN网络结构

Fig.1 Slim-CNN network structure

图2 Slim Module和SSE Block结构

Fig.2 Slim Module and SSE Block structure

图3 结合特征融合和任务分组的人脸属性识别模型整体框架

Fig.3 Overall framework of face attribute recognition model combining feature fusion and task grouping

图4 特征金字塔融合结构

Fig.4 Feature pyramid fusion structure

图5 ECA结构

Fig.5 ECA structure

图6 CelebA数据集上的消融实验结果对比

Fig.6 Comparison of the results of ablation experiments on the CelebA dataset

参考文献 28

1	PARIKH D. Visual attributes for enhanced human-machine communication[C]//Proceedings of the 51st Annual Allerton Conference on Communication, Control, and Computing. Washington D. C., USA: IEEE Press, 2014: 1126-1127.
2	VAQUERO D A, FERIS R S, TRAN D, et al. Attribute-based people search in surveillance environments[C]//Proceedings of Workshop on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2010: 1-8.
3	KIM J, PAVLOVIC V. Attribute rating for classification of visual objects[C]//Proceedings of the 21st International Conference on Pattern Recognition. Washington D. C., USA: IEEE Press, 2012: 1611-1614.
4	SIDDIQUIE B, FERIS R S, DAVIS L S. Image ranking and retrieval based on multi-attribute queries[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2011: 801-808.
5	WU Z, KE Q F, SUN J, et al. Scalable face image retrieval with identity-based quantization and multireference reranking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (10): 1991- 2001. doi: 10.1109/TPAMI.2011.111
6	LECUN Y, BENGIO Y, HINTON G. Deep learning. Nature, 2015, 521 (7553): 436- 444. doi: 10.1038/nature14539
7	CHEN X Y, WANG W W, ZHENG S W. Research on face attribute recognition based on multi-task CNN network[C]//Proceedings of the 4th Information Technology, Networking, Electronic and Automation Control Conference. Washington D. C., USA: IEEE Press, 2020: 1221-1224.
8	赖心瑜, 陈思, 严严, 等. 基于深度学习的人脸属性识别方法综述. 计算机研究与发展, 2021, 58 (12): 2760- 2782. URL
	LAI X Y, CHEN S, YAN Y, et al. Survey on deep learning based facial attribute recognition methods. Journal of Computer Research and Development, 2021, 58 (12): 2760- 2782. URL
9	RUDD E M, GÜNTHER M, BOULT T E. MOON: a mixed objective optimization network for the recognition of facial attributes[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 19-35.
10	MAO L B, YAN Y, XUE J H, et al. Deep multi-task multi-label CNN for effective facial attribute classification. IEEE Transactions on Affective Computing, 2022, 13 (2): 818- 828. doi: 10.1109/TAFFC.2020.2969189
11	ZHENG X, HUANG H B, GUO Y Q, et al. BLAN: bi-directional ladder attentive network for facial attribute prediction. Pattern Recognition, 2020, 100, 107155. doi: 10.1016/j.patcog.2019.107155
12	姚树婧, 张立言, 李星燃. 结合特征解耦和静动态联合图卷积网络的人脸属性识别. 计算机辅助设计与图形学学报, 2022, 34 (7): 1020- 1027. URL
	YAO S J, ZHANG L Y, LI X R. Facial attributes recognition combined with feature decoupling and static-dynamic joint graph convolutional network. Journal of Computer-Aided Design & Computer Graphics, 2022, 34 (7): 1020- 1027. URL
13	HAND E, CHELLAPPA R. Attributes for improved attributes: a multi-task network utilizing implicit and explicit relationships for facial attribute classification[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2017: 4068-4074.
14	HUANG S Y, LI X, CHENG Z Q, et al. GNAS: a greedy neural architecture search method for multi-attribute learning[C]//Proceedings of the 26th ACM International Conference on Multimedia. New York, USA: ACM Press, 2018: 2049-2057.
15	ZHUANG N, YAN Y, CHEN S, et al. Multi-task learning of cascaded CNN for facial attribute classification[C]//Proceedings of the 24th International Conference on Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 2069-2074.
16	SAVCHENKO A V. Facial expression and attributes recognition based on multi-task learning of lightweight neural networks[C]//Proceedings of the 19th International Symposium on Intelligent Systems and Informatics. Washington D. C., USA: IEEE Press, 2021: 119-124.
17	LIU H W, CAI H L, LIN Q C, et al. Adaptive multilayer perceptual attention network for facial expression recognition. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32 (9): 6253- 6266. doi: 10.1109/TCSVT.2022.3165321
18	SHARMA A K, FOROOSH H. Slim-CNN: a light-weight CNN for face attribute prediction[C]//Proceedings of the 15th IEEE International Conference on Automatic Face and Gesture Recognition. Washington D. C., USA: IEEE Press, 2021: 329-335.
19	ZHONG Y, SULLIVAN J, LI H B. Leveraging mid-level deep representations for predicting face attributes in the wild[C]//Proceedings of IEEE International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2016: 3239-3243.
20	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 936-944.
21	SHI C J, ZHANG W M, DUAN C Y, et al. A pooling-based feature pyramid network for salient object detection. Image and Vision Computing, 2021, 107, 104099. doi: 10.1016/j.imavis.2021.104099
22	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11531-11539.
23	CORTES C, MOHRI M, ROSTAMIZADEH A. Algorithms for learning kernels based on centered alignment. Journal of Machine Learning Research, 2012, 13, 795- 828. URL
24	CRISTIANINI N, KANDOLA J, ELISSEEFF A, et al. On kernel-target alignment[C]//Proceedings of 2001 Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2001: 367-373.
25	KORNBLITH S, NOROUZI M, LEE H, et al. Similarity of neural network representations revisited[C]//Proceedings of 2019 International Conference on Machine Learning. Berlin, Germany: Springer, 2019: 3519-3529.
26	VON LUXBURG U. A tutorial on spectral clustering. Statistics and Computing, 2007, 17 (4): 395- 416. doi: 10.1007/s11222-007-9033-z
27	CIPOLLA R, GAL Y, KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7482-7491.
28	LIU Z W, LUO P, WANG X G, et al. Deep learning face attributes in the wild[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 3730-3738.

[1]	范文卓, 吴涛, 许俊平, 李庆庆, 张建林, 李美惠, 魏宇星. 基于多分辨率特征融合的任意尺度图像超分辨率重建[J]. 计算机工程, 2023, 49(9): 217-225.
[2]	杨静, 陆铭华, 马洁琼, 吴金平, 刘星璇. 基于交替循环神经网络的水下防御态势预测方法[J]. 计算机工程, 2023, 49(9): 69-78.
[3]	孙龙, 张荣芬, 刘宇红, 饶庭漓. 监控视角下密集人群口罩佩戴检测算法[J]. 计算机工程, 2023, 49(9): 313-320.
[4]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[5]	苏晓东, 李世洲, 赵佳圆, 亮洪宇, 张玉荣, 徐红岩. 基于多级叠加和注意力机制的图像语义分割[J]. 计算机工程, 2023, 49(9): 265-271, 278.
[6]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[7]	龙玉江, 卫薇, 舒彧, 张正刚, 王道累, 李峰. 基于自适应关键点的破损旋转绝缘子检测方法[J]. 计算机工程, 2023, 49(9): 272-278.
[8]	丰芳宇, 罗晓曙, 蒙志明, 王广宇. 基于抗混叠残差注意力网络的人脸表情识别[J]. 计算机工程, 2023, 49(8): 190-198.
[9]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[10]	张欣怡, 张飞, 郝斌, 高鹭, 任晓颖. 基于改进YOLOv5的口罩佩戴检测算法[J]. 计算机工程, 2023, 49(8): 265-274.
[11]	王书朋, 何引弟. 融合特征注意力机制的非均匀光照图像增强算法[J]. 计算机工程, 2023, 49(8): 232-239.
[12]	刘昊鑫, 董超, 勾智楠, 高凯. 融合混合表征的小样本关系抽取方法[J]. 计算机工程, 2023, 49(8): 63-68.
[13]	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(8): 85-95.
[14]	杨祖赫, 黎智辉, 唐云祁, 晏于文, 宋华青. 结合语义与图像信息的行人属性识别算法[J]. 计算机工程, 2023, 49(8): 215-222, 231.
[15]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.

选择文件类型/文献管理软件名称

选择包含的内容