基于改进轻量级秩扩展网络的人脸表情识别方法

doi:10.19678/j.issn.1000-3428.0062811

摘要/Abstract

摘要： 人脸表情识别作为人机交互的一种重要方法，广泛应用于智能医疗、公安测谎系统、车载安全系统等领域。现有人脸表情识别方法多数存在参数量冗余、计算成本高、特征表达瓶颈等问题。提出一种基于改进轻量级秩扩展网络ReXNet的人脸表情识别方法。通过构建改进的ReXNet以提取人脸表情特征，在参数量较少的条件下解决特征表达瓶颈的问题，增强对表情局部特征的关注，获得高层次的表情特征，同时融合坐标注意力模块，将位置信息嵌入到通道注意力中，精准地定位和识别感兴趣的特征，建立位置信息与局部特征之间的长依赖关系，减少计算开销。在此基础上，将细化模块引入到改进的网络架构中，利用类别上下文信息细化分类结果，增强类间的分化效果，从而提高人脸表情识别的准确率。实验结果表明，该方法在RAF-DB和FERPlus数据集上的人脸表情识别准确率分别达到88.43%和88.8%，相比VGG16-PLD、SHCNN、ResNet+VGG等方法，具有较高的准确率和较优的鲁棒性。

关键词: 人脸表情识别, 秩扩展网络, 表达瓶颈, 坐标注意力机制, 细化模块

Abstract: Facial Expression Recognition(FER), an important method in the field of human-computer interaction, is widely used for intelligent medical treatment and in public security lie detection systems, vehicle safety systems, and other applications.However, existing FER methods are problematic in that they are plagued by parameter redundancy, high computational cost, feature expression bottlenecks, and so on.This study proposes an FER method based on an improved lightweight Rank Expansion Network(ReXNet).By constructing improved ReXNet to exact facial expression features, solves the feature expression bottleneck by decreasing the number of parameters and enhancing attention to local features of expression to produce high-level features of expression.At the same time, a Coordinate Attention(CA) module is integrated, and the location information is embedded into the channel attention.These improvements enable the model to accurately locate and identify the features of interest, and to establish a long-term relationship between location information and local features, thereby reducing the computational overhead.On this basis, a refinement module is introduced to the improved model.The module refines the classification results by using the category context information to enhance the differentiation between classes to ultimately improve the accuracy of FER.The experimental results show that the accuracy of FER on the RAF-DB and FERPlus datasets reaches 88.43% and 88.8%, respectively.Compared with VGG16-PLD, SHCNN, ResNet+VGG, and other methods, the proposed method has higher accuracy and superior robustness.

Key words: Facial Expression Recognition(FER), Rank Expansion Network(ReXNet), expression bottleneck, Coordinate Attention(CA) mechanism, refinement module

中图分类号:

TP391

郑伟鹏, 罗晓曙, 蒙志明. 基于改进轻量级秩扩展网络的人脸表情识别方法[J]. 计算机工程, 2022, 48(9): 189-196.

ZHENG Weipeng, LUO Xiaoshu, MENG Zhiming. Facial Expression Recognition Method Based on Improved Lightweight Rank Expansion Network[J]. Computer Engineering, 2022, 48(9): 189-196.

https://www.ecice06.com/CN/Y2022/V48/I9/189

图/表 12

20221008183206

20221008183210

20221008183214

20221008183218

20221008183222

20221008183226

20221008183229

20221008183233

20221008183240

20221008183243

20221008183247

20221008183250

参考文献

[1] TAHA B, HATZINAKOS D.Emotion recognition from 2D facial expressions[C]//Proceedings of IEEE Canadian Conference of Electrical and Computer Engineering.Washington D.C., USA:IEEE Press, 2019:1-4.
[2] DING H, SRICHARAN K, CHELLAPPA R.ExprGAN:facial expression editing with controllable expression intensity[EB/OL].[2021-08-23].https://arxiv.org/pdf/1709.03842.pdf.
[3] PANG L, LI N Q, ZHAO L, et al.Facial expression recognition based on Gabor feature and neural network[C]//Proceedings of International Conference on Security, Pattern Analysis, and Cybernetics.Washington D.C., USA:IEEE Press, 2018:489-493.
[4] KUSHWAH K, SHARMA V, SINGH U.Neural network method through facial expression recognition[C]//Proceedings of International Conference on Electronics, Communication and Aerospace Technology.Washington D.C., USA:IEEE Press, 2017:532-537.
[5] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2021-08-23].https://arxiv.org/pdf/1409.1556.pdf.
[6] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J].Communications of the ACM, 2017, 60(6):84-90.
[7] HE K M, ZHANG X Y, REN S Q, et al.Identity mappings in deep residual networks[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:630-645.
[8] CHENG S, ZHOU G H.Facial expression recognition method based on improved VGG convolutional neural network[J].International Journal of Pattern Recognition and Artificial Intelligence, 2020, 34(7):1-10.
[9] ZHONG Y X, QIU S H, LUO X S, et al.Facial expression recognition based on optimized ResNet[C]//Proceedings of the 2nd World Symposium on Artificial Intelligence. Washington D.C., USA:IEEE Press, 2020:84-91.
[10] 方明, 陈文强.结合残差网络及目标掩膜的人脸微表情识别[J].吉林大学学报(工学版), 2021, 51(1):303-313. FANG M, CHEN W Q.Face micro-expression recognition based on ResNet with object mask[J].Journal of Jilin University (Engineering and Technology Edition), 2021, 51(1):303-313.(in Chinese)
[11] 亢洁, 李佳伟, 杨思力.基于域适应卷积神经网络的人脸表情识别[J].计算机工程, 2019, 45(12):201-206. KANG J, LI J W, YANG S L.Facial expression recognition based on convolutional neural network with domain adaption[J].Computer Engineering, 2019, 45(12):201-206.(in Chinese)
[12] 钱勇生, 邵洁, 季欣欣, 等.基于改进卷积神经网络的多视角人脸表情识别[J].计算机工程与应用, 2018, 54(24):12-19. QIAN Y S, SHAO J, JI X X, et al.Multi-view facial expression recognition based on improved convolutional neural network[J].Computer Engineering and Applications, 2018, 54(24):12-19.(in Chinese)
[13] 冯杨, 刘蓉, 鲁甜.基于小尺度核卷积的人脸表情识别[J]. 计算机工程, 2021, 47(4):262-267. FENG Y, LIU R, LU T.Facial expression recognition based on small-scale kernel convolution[J].Computer Engineering, 2021, 47(4):262-267.(in Chinese)
[14] WANG K, PENG X J, YANG J F, et al.Suppressing uncertainties for large-scale facial expression recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:6896-6905.
[15] CHENG Y, JIANG B, JIA K B.A deep structure for facial expression recognition under partial occlusion[C]//Proceedings of the 10th International Conference on Intelligent Information Hiding and Multimedia Signal Processing.Washington D.C., USA:IEEE Press, 2014:211-214.
[16] CHEN Y J, LIU S G.Deep partial occlusion facial expression recognition via improved CNN[C]//Proceedings of International Symposium on Visual Computing.Berlin, Germany:Springer, 2020:451-462.
[17] SHI H C, LI H L, MENG F M, et al.Key-word-aware network for referring expression image segmentation[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:38-54.
[18] HAN D, YUN S, HEO B, et al.Rethinking channel dimensions for efficient model design[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2021:732-741.
[19] HOU Q B, ZHOU D Q, FENG J S.Coordinate attention for efficient mobile network design[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2021:13708-13717.
[20] LI S, DENG W H, DU J P.Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D. C., USA:IEEE Press, 2017:1-10.
[21] BARSOUM E, ZHANG C, FERRER C C, et al.Training deep networks for facial expression recognition with crowd-sourced label distribution[C]//Proceedings of the 18th ACM International Conference on Multimodal Interaction. New York, USA:ACM Press, 2016:279-283.
[22] LI S, DENG W H, DU J P.Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:2584-2593.
[23] LI Y, ZENG J B, SHAN S G, et al.Occlusion aware facial expression recognition using CNN with attention mechanism[J].IEEE Transactions on Image Processing, 2019, 28(5):2439-2450.
[24] WANG K, PENG X J, YANG J F, et al.Region attention networks for pose and occlusion robust facial expression recognition[J].IEEE Transactions on Image Processing, 2020, 29:4057-4069.
[25] WANG K, PENG X J, YANG J F, et al.Suppressing uncertainties for large-scale facial expression recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:6896-6905.
[26] BARSOUM E, ZHANG C, FERRER C C, et al.Training deep networks for facial expression recognition with crowd-sourced label distribution[C]//Proceedings of the 18th ACM International Conference on Multimodal Interaction.New York, USA:ACM Press, 2016:279-283.
[27] MIAO S, XU H Y, HAN Z Q, et al.Recognizing facial expressions using a shallow convolutional neural network[J].IEEE Access, 2019, 7:78000-78011.
[28] HUANG C.Combining convolutional neural networks for emotion recognition[C]//Proceedings of IEEE MIT Undergraduate Research Technology Conference.Washington D.C., USA:IEEE Press, 2017:1-4.
[29] SELVARAJU R R, COGSWELL M, DAS A, et al.Grad-CAM:visual explanations from deep networks via gradient-based localization[J].International Journal of Computer Vision, 2020, 128(2):336-359.
[30] MAATEN V D, HINTON G.Visualizing data using t-SNE[J].Journal of Machine Learning Research, 2008, 9:2579-2605.

选择文件类型/文献管理软件名称

选择包含的内容