EmoRepLKNet: 一种基于UniRepLKNet的面部情绪识别神经网络

doi:10.19678/j.issn.1000-3428.0069761

摘要/Abstract

摘要：

针对面部情绪识别过程中存在的难以捕获有效特征信息、无法使关键面部信息占据更主要地位的问题, 提出一种基于UniRepLKNet的面部情绪识别网络。为了更精确地提取面部情绪特征, 设计一个掩码极化自注意力模块, 其结合了U-Net和极化自注意力机制。这一模块能够深入挖掘通道和空间之间的依赖关系, 并通过多尺度特征融合策略, 强化人脸局部关键信息在情绪识别过程中的影响力。同时, 对大核卷积神经网络(CNN)UniRepLKNet进行优化, 提出EmoRepLKNet神经网络结构。在EmoRepLKNet中, 利用掩码极化自注意力模块使网络专注于提取面部情绪识别的关键信息, 并结合大核CNN感受野广的特点, 实现对面部情绪的有效识别。实验结果表明, 在面部情绪识别数据集FER2013上, 该方法达到了76.20%的准确率, 不仅超越了现有的对比模型, 而且相较于UniRepLKNet也显著提高了面部情绪识别的准确率。同时, 在RAF-DB数据集的单标签部分进行实验, 所提方法取得了89.67%的准确率。

关键词: 情绪识别, 深度学习, 大核卷积神经网络, 注意力机制, FER2013数据集, RAF-DB数据集

Abstract:

This study presents a facial emotion recognition network based on UniRepLKNet to address the difficulty in effectively capturing feature information and preventing key facial information from occupying a more prominent position in the facial emotion recognition process. Moreover, to extract facial emotional features more accurately, the study designs a masked polarized self-attention module that combines U-Net and a polarized self-attention mechanism. This module can deeply mine the dependency between channels and spaces. It can also strengthen the influence of local key information of the face on emotion recognition through a multi-scale feature fusion strategy. The study optimizes UniRepLKNet, a universal large kernel Convolutional Neural Network (CNN), and proposes the EmoRepLKNet neural network structure. In EmoRepLKNet, the mask-polarized self-attention module enables the network to extract key information for facial emotion recognition. Combined with the wide receptive field of large kernel CNN, facial emotions can be recognized effectively. Experimental results show that on the facial emotion recognition dataset FER2013, EmoRepLKNet achieves an accuracy of 76.20%, outperforming existing comparison models and significantly improving facial emotion recognition accuracy compared to that of UniRepLKNet. Additionally, on the single-label portion of the RAF-DB dataset, the proposed method achieves an accuracy of 89.67%.

Key words: emotion recognition, deep learning, large kernel Convolutional Neural Network (CNN), attention mechanism, FER2013 dataset, RAF-DB dataset

肖志鹏, 何书峰, 田春岐. EmoRepLKNet: 一种基于UniRepLKNet的面部情绪识别神经网络[J]. 计算机工程, 2025, 51(11): 54-62.

XIAO Zhipeng, HE Shufeng, TIAN Chunqi. EmoRepLKNet: Facial Emotion Recognition Neural Network Based on UniRepLKNet[J]. Computer Engineering, 2025, 51(11): 54-62.

https://www.ecice06.com/CN/Y2025/V51/I11/54

图/表 15

图1 UniRepLKNet结构

Fig.1 UniRepLKNet structure

图2 RPS Block结构

Fig.2 RPS Block structure

图3 RPS Block流程

Fig.3 Procedure of the RPS Block

图4 热力图分析

Fig.4 Thermographic analysis

图5 EmoRepLKNet整体结构

Fig.5 Overall structure of EmoRepLKNet

图6 EmoRepLKNet在FER2013上的混淆矩阵

Fig.6 The confusion matrix of EmoRepLKNet on FER2013

图7 EmoRepLKNet在RAF-DB上的混淆矩阵

Fig.7 The confusion matrix of EmoRepLKNet on RAF-DB

参考文献 36

1	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Washington D.C., USA: IEEE Press, 2005: 886-893.
2	LU C Y, YAN P F, ZHANG C S, et al. Face recognition using support vector machine[C]//Proceedings of 1998 International Conference on Neural Networks and Brain(ICNNB'98). Washington D.C., USA: IEEE Press, 1998: 699-702.
3	COVER T , HART P . Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 1967, 13 (1): 21- 27.
4	PAVLOV Y L. Random forests[EB/OL]. [2023-10-05]. https://link.springer.com/article/10.1023/A:1010933404324.
5	袁德荣, 张勇, 唐颖军, 等. 多尺度残差注意力网络及其表情识别算法. 小型微型计算机系统, 2024, 45 (1): 30- 36.
	YUAN D R , ZHANG Y , TANG Y J , et al. Multiscale residual attention network and its facial expression recognition algorithm. Journal of Chinese Computer Systems, 2024, 45 (1): 30- 36.
6	MEHRABIAN A. Communication without words[EB/OL]. [2023-10-05]. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315080918-15/communication-without-words-albert-mehrabian.
7	EKMAN P , FRIESEN W V . Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 1971, 17 (2): 124- 129.
8	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1706.03762.
9	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2010.11929.
10	LIU Z, MAO H Z, WU C Y, et al. A ConvNet for the 2020s[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2022: 11966-11976.
11	LIU Z M, WANG Y X, VAIDYA S, et al. KAN: Kolmogorov-Arnold Networks[EB/OL]. [2023-10-05]. https://doi.org/10.48550/arXiv.2404.19756.
12	LI R R , LIU W J , YANG L , et al. DeepUNet: a deep fully convolutional network for pixel-level sea-land segmentation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11 (11): 3954- 3962.
13	LIU H J , LIU F Q , FAN X Y , et al. Polarized self-attention: towards high-quality pixel-wise mapping. Neurocomputing, 2022, 506, 158- 167.
14	PECORARO R , BASILE V , BONO V . Local multi-head channel self-attention for facial expression recognition. Information, 2022, 13 (9): 419.
15	FARD A P , MAHOOR M H . Ad-Corre: adaptive correlation-based loss for facial expression recognition in the wild. IEEE Access, 2022, 10, 26756- 26768.
16	PHAM L, VU T H, TRAN T A. Facial expression recognition using residual masking network[C]//Proceedings of the 25th International Conference on Pattern Recognition (ICPR). Washington D.C., USA: IEEE Press, 2021: 4513-4519.
17	VIGNESH S , SAVITHADEVI M , SRIDEVI M , et al. A novel facial emotion recognition model using Segmentation VGG-19 architecture. International Journal of Information Technology, 2023, 15 (4): 1777- 1787.
18	EL BOUDOURI Y, BOHI A. EmoNeXt: an adapted ConvNeXt for facial emotion recognition[C]//Proceedings of the 25th IEEE International Workshop on Multimedia Signal Processing (MMSP). Washington D.C., USA: IEEE Press, 2023: 1-6.
19	DING X, ZHANG Y, GE Y, et al. UniRepLKNet: a universal perception large-kernel ConvNet for audio, video, point cloud, time-series and image recognition[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2311.15599.
20	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[EB/OL]. [2023-10-05]. https://arxiv.org/pdf/1505.04597.
21	LIU H, LIU F, FAN X, et al. Polarized self-attention: towards high-quality pixel-wise regression[EB/OL]. [2023-10-05]. https://arxiv.org/pdf/2107.00782.
22	GIANNOPOULOS P, PERIKOS I, HATZILYGEROUDIS I. Deep learning approaches for facial emotion recognition: a case study on FER2013[EB/OL]. [2023-10-05]. https://www.semanticscholar.org/paper/Deep-Learning-Approaches-for-Facial-Emotion-A-Case-Giannopoulos-Perikos/79a9ebfcccf9e7608da779b3616c28a9b6c98b36.
23	MINAEE S, MINAEI M, ABDOLRASHIDI A. Deep-emotion: facial expression recognition using attentional convolutional network[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1902.01019.
24	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1409.4842.
25	KHANZADA A, BAI C, CELEPCIKAY F T. Facial expression recognition with deep learning[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2004.11823.
26	KHAIREDDIN Y, CHEN Z. Facial emotion recognition: state of the art performance on FER2013[EB/OL]. [2023-10-05]. https://arxiv.org/pdf/2105.03588.
27	蓝峥杰, 王烈, 聂雄. 一种基于词频-逆文档频率和混合损失的表情识别算法. 计算机工程, 2023, 49 (1): 295-302, 310. doi: 10.19678/j.issn.1000-3428.0063455
	LAN Z J , WANG L , NIE X . An expression recognition algorithm based on term frequency-inverse document frequency and hybrid loss. Computer Engineering, 2023, 49 (1): 295-302, 310. doi: 10.19678/j.issn.1000-3428.0063455
28	GEORGESCU M I , IONESCU R T , POPESCU M . Local learning with deep and handcrafted features for facial expression recognition. IEEE Access, 2019, 7, 64827- 64836.
29	CHEN S K, WANG J F, CHEN Y D, et al. Label distribution learning on auxiliary label space graphs for facial expression recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 13981-13990.
30	王军, 赵凯, 程勇. 基于遮挡感知卷积神经网络的面部表情识别模型. 计算机工程, 2021, 47 (10): 242- 251. doi: 10.19678/j.issn.1000-3428.0059166
	WANG J , ZHAO K , CHENG Y . Facial expression recognition model based on convolutional neural network with occlusion perception. Computer Engineering, 2021, 47 (10): 242- 251. doi: 10.19678/j.issn.1000-3428.0059166
31	WANG K , PENG X J , YANG J F , et al. Region attention networks for pose and occlusion robust facial expression recognition. IEEE Transactions on Image Processing, 2020, 29, 4057- 4069.
32	WANG K, PENG X J, YANG J F, et al. Suppressing uncertainties for large-scale facial expression recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 6896-6905.
33	MA F Y , SUN B , LI S T . Facial expression recognition with visual transformers and attentional selective fusion. IEEE Transactions on Affective Computing, 2023, 14 (2): 1236- 1248.
34	丰芳宇, 罗晓曙, 蒙志明, 等. 基于抗混叠残差注意力网络的人脸表情识别. 计算机工程, 2023, 49 (8): 190- 198. doi: 10.19678/j.issn.1000-3428.0065224
	FENG F Y , LUO X S , MENG Z M , et al. Facial expression recognition based on anti-aliasing residual attention network. Computer Engineering, 2023, 49 (8): 190- 198. doi: 10.19678/j.issn.1000-3428.0065224
35	ZHAO Z, LIU Q, ZHOU F. Robust lightweight facial expression recognition network with label distribution training[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 3510-3519.
36	李晶, 李健, 陈海丰, 等. 基于关键区域遮挡与重建的人脸表情识别. 计算机工程, 2024, 50 (5): 241- 249. doi: 10.19678/j.issn.1000-3428.0067538
	LI J , LI J , CHEN H F , et al. Facial expression recognition based on key region masking and reconstruction. Computer Engineering, 2024, 50 (5): 241- 249. doi: 10.19678/j.issn.1000-3428.0067538

[1]	翟志鹏, 曹阳, 沈琴琴, 施佺. 基于多时空图融合与动态注意力的交通流预测[J]. 计算机工程, 2025, 51(9): 139-148.
[2]	黄金贵, 刘朋, 唐文胜. MMD-YOLOv7:黑暗条件下车辆检测方法[J]. 计算机工程, 2025, 51(9): 340-349.
[3]	符家成, 田瑾, 张玉金, 方志军. 结合前置三元组集的知识图谱推荐[J]. 计算机工程, 2025, 51(9): 101-109.
[4]	周晨阳, 刘雪宇, 梁少华, 吴永飞. 基于Swin Transformer的肾动脉血管检测分割与定量分析[J]. 计算机工程, 2025, 51(9): 252-267.
[5]	徐瀅, 傅紫薇, 张伟, 陈云芳. 基于抽象语法树嵌入的智能合约漏洞检测技术[J]. 计算机工程, 2025, 51(9): 149-157.
[6]	马淦, 谷雨, 彭冬亮. 结合改进YOLOv5s和动态数据增强的海面舰船检测[J]. 计算机工程, 2025, 51(9): 294-305.
[7]	陈彦如, 刘珂良, 冉茂亮. 基于深度强化学习的外卖即时配送实时优化[J]. 计算机工程, 2025, 51(9): 328-339.
[8]	倪源松, 韩军, 邹小燕, 胡广怡, 王文帅. 两阶段自适应分块输电线路螺栓缺陷检测方法[J]. 计算机工程, 2025, 51(8): 281-291.
[9]	郝宏达, 罗健旭. 基于多尺度区域特征融合的多器官语义分割模型[J]. 计算机工程, 2025, 51(8): 270-280.
[10]	林帆, 李建华. 基于多阶门控聚合网络的光学化学结构识别[J]. 计算机工程, 2025, 51(8): 364-372.
[11]	武东辉, 王金凤, 仇森, 刘国志. 基于EWBiLSTM-ATT的数据手套手语识别[J]. 计算机工程, 2025, 51(8): 107-119.
[12]	武东辉, 王金凤, 仇森, 刘国志. 基于EWBiLSTM-ATT的数据手套手语识别[J]. 计算机工程, 2025, 51(8): 107-119.
[13]	张昭理, 李家豪, 刘海, 石佛波, 何嘉文. 基于个性化遗忘建模的知识追踪方法[J]. 计算机工程, 2025, 51(8): 120-130.
[14]	闫建红, 刘芝妍, 王震. 融合时空注意力机制的多尺度卷积车辆轨迹预测[J]. 计算机工程, 2025, 51(8): 406-414.
[15]	余鹏, 杨佳琦, 陈欣然, 贺超波. 基于二部图对比学习的特征增强推荐算法[J]. 计算机工程, 2025, 51(7): 100-110.

选择文件类型/文献管理软件名称

选择包含的内容