基于人脸关键特征提取的表情识别

doi:10.19678/j.issn.1000-3428.0063715

摘要/Abstract

摘要： 自然场景下人脸表情由于受遮挡、光照等因素影响，以及表情局部变化细微，导致现有人脸表情识别方法准确率较低。提出一种人脸表情识别的新方法，以ResNet18为主干网络，利用残差连接模块加深网络结构，以提取更多深层次的表情特征。通过引入裁剪掩码模块，在训练集图像上的某个区域进行掩码，向训练模型中增加遮挡等非线性因素，提升模型在遮挡情形下的鲁棒性。分别从特征图的通道和空间两个维度提取表情的关键特征，并分配更多的权重给表情变化明显的特征图，同时抑制非表情特征。在特征图输出前加入Dropout正则化策略，通过在训练中随机失活部分神经元，达到集成多个网络模型的训练效果，提升模型泛化能力。实验结果表明，与L2-SVMs、IcRL、DLP-CNN等方法相比，该方法有效提高了表情识别准确率，在2个公开表情数据集Fer2013和RAF-DB上的识别准确率分别为74.366%和86.115%。

关键词: 注意力机制, 残差网络, 人脸表情识别, 裁剪掩码, Dropout正则化

Abstract: The accuracy of existing facial expression recognition methods is typically low owing to the influence of occlusion, illumination, and other factors and to subtle local variations in facial expressions in natural scenes.This study presents a new method for facial expression recognition.A ResNet18 model is adopted as the backbone network, and a residual connection module is employed for a deeper network structure to extract deeper expression features.First, a cutout module is introduced.By masking a certain area of each image in the training set, the model learns to consider several nonlinear factors, such as occlusion, thus improving its robustness to these conditions.The key features of expressions are extracted from the channel and space of the graph, and more weights are assigned to feature maps with obvious expression changes to suppress non-expression features.Finally, prior to the output of the feature map, a Dropout regularization strategy is implemented to randomly deactivate some neurons and integrate multiple network models to improve the generalization ability of the model.The experimental results show that the proposed method exhibits improved accuracy in expression recognition tasks compared with L2-SVMs, IcRL, DLP-CNN, and other methods. Recognition accuracy values of 74.366% and 86.115% are achieved on two public expression datasets, Fer2013 and RAF-DB, respectively.

Key words: attention mechanism, residual network, facial expression recognition, cropping mask, Dropout regularization

中图分类号:

TP391

冉瑞生, 翁稳稳, 王宁, 彭顺顺. 基于人脸关键特征提取的表情识别[J]. 计算机工程, 2023, 49(2): 254-262.

RAN Ruisheng, WENG Wenwen, WANG Ning, PENG Shunshun. Expression Recognition Based on the Extraction of Key Facial Features[J]. Computer Engineering, 2023, 49(2): 254-262.

http://www.ecice06.com/CN/Y2023/V49/I2/254

图/表 16

20230216182048

20230216182051

20230216182054

20230216182057

20230216182101

20230216182104

20230216182108

20230216182111

20230216182114

20230216182117

20230216182121

20230216182125

20230216182128

20230216182132

20230216182138

20230216182142

参考文献

[1] MEHRABIAN A, RUSSELL J A.An approach to environmental psychology[M].Cambridge, USA:MIT Press, 1974.
[2] EKMAN P.Strong evidence for universals in facial expressions:a reply to Russell's mistaken critique[J].Frontiers in Cell and Developmental Biology, 1994, 115(2):268-287.
[3] OJALA T, PIETIKAINEN M, MAENPAA T.Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.
[4] ZHANG Z Y, MU X M, GAO L.Recognizing facial expressions based on Gabor filter selection[C]//Proceedings of the 4th International Congress on Image and Signal Processing.Washington D.C., USA:IEEE Press, 2011:1544-1548.
[5] LOWE D G.Object recognition from local scale-invariant features[C]//Proceedings of the 7th IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 1999:1150-1157.
[6] CORTES C, VAPNIK V.Support-vector networks[J].Machine Learning, 1995, 20(3):273-297.
[7] LUCEY P, COHN J F, KANADE T, et al.The extended Cohn-Kanade dataset (CK+):a complete dataset for action unit and emotion-specified expression[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops.Washington D.C., USA:IEEE Press, 2010:94-101.
[8] LYONS M, AKAMATSU S, KAMACHI M, et al.Coding facial expressions with Gabor wavelets[C]//Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition.Washington D.C., USA:IEEE Press, 1998:200-205.
[9] LECUN Y, BOTTOU L, BENGIO Y, et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE, 1998, 86(11):2278-2324.
[10] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2021-12-01].http://arXiv:1409.1556.
[11] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[12] TANG Y C.Deep learning using linear support vector machines[EB/OL].[2021-12-01].http://arXiv:1306.0239.
[13] CHEN Y Z, HU H F.Facial expression recognition by inter-class relational learning[J].IEEE Access, 2019, 7:94106-94117.
[14] 兰凌强, 李欣, 刘淇缘, 等.基于联合正则化策略的人脸表情识别方法[J].北京航空航天大学学报, 2020, 46(9):1797-1806. LAN L Q, LI X, LIU Q Y, et al.Facial expression recognition method based on a joint normalization strategy[J].Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(9):1797-1806.(in Chinese)
[15] LI S, DENG W H, DU J P.Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:2584-2593.
[16] LI Y, ZENG J B, SHAN S G, et al.Occlusion aware facial expression recognition using CNN with attention mechanism[J].IEEE Transactions on Image Processing, 2019, 28(5):2439-2450.
[17] LUO Z M, HU J N, DENG W H.Local subclass constraint for facial expression recognition in the wild[C]//Proceedings of the 24th International Conference on Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:3132-3137.
[18] LU Y, WANG S G, ZHAO W T, et al.WGAN-based robust occluded facial expression recognition[J].IEEE Access, 2019, 7:93594-93610.
[19] CHEN Y J, LIU S G.Deep partial occlusion facial expression recognition via improved CNN[M]//Advances in Visual Computing.Berlin, Germany:Springer, 2020:451-462.
[20] DEVRIES T, TAYLOR G W.Improved regularization of convolutional neural networks with cutout[EB/OL].[2021-12-01].http://arXiv:1708.04552.
[21] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[22] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al.Improving neural networks by preventing co-adaptation of feature detectors[EB/OL].[2021-12-01].https://arxiv.org/abs/1207.0580.
[23] DHALL A, GOECKE R, LUCEY S, et al.Static facial expression analysis in tough conditions:data, evaluation protocol and benchmark[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Washington D.C., USA:IEEE Press, 2011:2106-2112.
[24] HU J, SHEN L, SUN G.Squeeze-and-excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7132-7141.
[25] 张爱梅, 徐杨.注意力分层双线性池化残差网络的表情识别[J].计算机工程与应用, 2020, 56(23):161-166. ZHANG A M, XU Y.Attention hierarchical bilinear pooling residual network for expression recognition[J].Computer Engineering and Applications, 2020, 56(23):161-166.(in Chinese)
[26] 王军, 赵凯, 程勇.基于遮挡感知卷积神经网络的面部表情识别模型[J].计算机工程, 2021, 47(10):242-251. WANG J, ZHAO K, CHENG Y.Facial expression recognition model based on convolutional neural network with occlusion perception[J].Computer Engineering, 2021, 47(10):242-251.(in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容