RRA-InceptionV3 Combined Robust Sparse Representation Method for Expression Recognition

doi:10.19678/j.issn.1000-3428.0064914

Abstract

Abstract:

Facial expression recognition technology is an important subject in the field of computer vision.However, in real application, partial occlusion can lead to a sharp drop in the accuracy of facial expression recognition.In view of the low accuracy of facial expression recognition caused by partial occlusion of faces in real scenes, an RRA-InceptionV3 combined robust sparse representation method for expression recognition is proposed.First, the facial image is obtained through a multi-branch convolution operation and atrous convolution module to obtain rich expression features.the Asm-CBAM convolution attention mechanism then divides the weight of facial expression features.Multi-feature fusion and then stacking of dense residual blocks are used to adaptively extract face feature information from multiple channels.In addition, the Asm-CBAM convolutional attention mechanism is used again to improve the network's attention to key facial features.Finally, a Robust Sparse Representation Classifier(RSRC) method is used to classify expressions.The experimental results on face data sets FER2013 and CK+ show that the average recognition accuracy of the proposed method reaches 79.86% and 98.74%, respectively, which is 7.50 and 3.14 percentage points higher than that of the OAD Net algorithm, and can efficiently extract facial expression features.Moreover, it has strong robustness in the case of face occlusion, effectively improving the accuracy of expression recognition.

Key words: expression recognition, partial occlusion, Robust Sparse Representation Classifier(RSRC) method, dense residuals, Asm-CBAM module, atrous convolution

摘要：

针对现实场景中人脸局部遮挡导致的表情识别准确度较低的问题，提出一种RRA-InceptionV3结合鲁棒稀疏表示的表情识别方法。将人脸图像通过多支路卷积运算和空洞卷积模块来获取丰富的表情特征，基于Asm-CBAM卷积注意力机制划分人脸表情特征的权重并进行多特征融合，随后堆叠密集残差模块，从多通道中自适应提取人脸特征信息，通过Asm-CBAM卷积注意力机制提高网络对人脸关键特征的注意力。在此基础上，利用鲁棒稀疏表示分类器方法对表情进行分类。在人脸数据集FER2013和CK+上的实验结果表明，该方法的人脸表情平均识别精度分别达到79.86%和98.74%，与OAD Net算法相比，分别高出7.50和3.14个百分点，能够高效提取人脸表情特征。此外，在人脸被遮挡的情况下具有较强的鲁棒性，有效提高了在人脸遮挡情况下表情识别的准确度。

关键词: 表情识别, 局部遮挡, 鲁棒稀疏表示分类器方法, 密集残差, Asm-CBAM模块, 空洞卷积

Hong XIE, Wengang JIANG. RRA-InceptionV3 Combined Robust Sparse Representation Method for Expression Recognition[J]. Computer Engineering, 2023, 49(7): 196-203.

谢虹, 姜文刚. RRA-InceptionV3结合鲁棒稀疏表示的表情识别方法[J]. 计算机工程, 2023, 49(7): 196-203.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0064914

http://www.ecice06.com/EN/Y2023/V49/I7/196

Figures/Tables 19

References 25

1	XU Q T, ZHAO N J. A facial expression recognition algorithm based on CNN and LBP feature[C]//Proceedings of the 4th IEEE Information Technology, Networking, Electronic and Automation Control Conference. Washington D. C., USA: IEEE Press, 2020: 2304-2308.
2	WANG Y, WU Q L. Research on face recognition technology based on PCA and SVM[C]//Proceedings of the 7th International Conference on Big Data Analytics. Washington D. C., USA: IEEE Press, 2022: 248-252.
3	LI Y, MA H, FANG C X, et al. Fusion of PHOG and Gabor for facial expression recognition[C]//Proceedings of the 7th International Conference on Intelligent Computing and Signal Processing. Washington D. C., USA: IEEE Press, 2022: 968-971.
4	ZHOU Y, FENG Y Y, ZENG S Y, et al. Facial expression recognition based on convolutional neural network[C]//Proceedings of the 10th IEEE International Conference on Software Engineering and Service Science. Washington D. C., USA: IEEE Press, 2020: 410-413.
5	LESIANGI F S, MAUKO A Y, DJAHI B S. Feature extraction Hue, Saturation, Value(HSV) and Gray Level Cooccurrence Matrix(GLCM) for identification of woven fabric motifs in south central Timor regency. Journal of Physics Series, 2021, (1): 012010.
6	THAMBA M W, HEMAJOTHI S, MARY A E A. Real-time facial expression recognition for affect identification using multi-dimensional SVM. Journal of Ambient Intelligence and Humanized Computing, 2021, 12 (6): 6355- 6365. doi: 10.1007/s12652-020-02221-6
7	FAN Z, XIE J K, WANG Z Y, et al. Image classification method based on improved KNN algorithm. Journal of Physics Series, 2021, 30 (3): 012009.
8	WANG S Q, GUAN S E, LIN H, et al. Micro-expression recognition based on optical flow and PCANet. Sensors, 2022, 22 (11): 4296. doi: 10.3390/s22114296
9	ZHANG L, XU C, LI S. Facial expression recognition of infants based on multi-stream CNN fusion network[C]// Proceedings of the 5th IEEE International Conference on Signal and Image Processing. Washington D. C., USA: IEEE Press, 2020: 37-41.
10	ZHANG Q. Facial expression recognition in VGG network based on LBP feature extraction[C]//Proceedings of the 5th International Conference on Mechanical, Control and Computer Engineering. Washington D. C., USA: IEEE Press, 2021: 2089-2092.
11	BOUALLEGUE G, DJEMAL R. EEG person identification using facenet, LSTM-RNN and SVM[C]//Proceedings of the 17th International Multi-Conference on Systems, Signals & Devices. Washington D. C., USA: IEEE Press, 2021: 22-28.
12	WANG X P, LI W X, HUANG D. Expression-latent-space-guided GAN for facial expression animation based on discrete labels[C]//Proceedings of the 16th IEEE International Conference on Automatic Face and Gesture Recognition. Washington D. C., USA: IEEE Press, 2022: 1-8.
13	CHEN Y, LIU S. Deep partial occlusion facial expression recognition via improved CNN[C]//Proceedings of International Conference on Visual Computing. Berlin, Germany: Springer, 2020: 451-462.
14	XU X K, TAO R, FENG X Y, et al. A lightweight facial expression recognition network based on dense connections[C]//Proceedings of International Conference on Knowledge Management in Organizations. Berlin, Germany: Springer, 2022: 347-359.
15	XU X F, ZHANG L, DUAN C D, et al. Research on inception module incorporated Siamese convolutional neural networks to realize face recognition. IEEE Access, 2019, 8, 12168- 12178.
16	NAINAN N A, JALAN R. Real time face mask detection using MobileNetV2 and InceptionV3 models[C]//Proceedings of IEEE Mysore Sub Section International Conference. Washington D. C., USA: IEEE Press, 2021: 341-345.
17	ALI W, TIAN W H, DIN S U, et al. Classical and modern face recognition approaches: a complete review. Multimedia Tools and Applications, 2021, 80 (3): 4825- 4880. doi: 10.1007/s11042-020-09850-1
18	WRIGHT J, YANG A Y, GANESH A, et al. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31 (2): 210- 227. doi: 10.1109/TPAMI.2008.79
19	WANG W, JIN J X. Super-resolution reconstruction of human face image based on network of residual dense connection channel attentional[C]//Proceedings of the 5th International Conference on Mechanical, Control and Computer Engineering. Washington D. C., USA: IEEE Press, 2021: 1972-1977.
20	ZHANG X, CHEN Z, WEI Q J. Research and application of facial expression recognition based on attention mechanism[C]//Proceedings of IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers. [S. 1. ]: IEEE Press, 2021: 282-285.
21	LI J, JIN K, ZHOU D L, et al. Attention mechanism-based CNN for facial expression recognition. Neurocomputing, 2020, 411, 340- 350. doi: 10.1016/j.neucom.2020.06.014
22	GERA D, BALASUBRAMANIAN S, JAMI A. CERN: compact facial expression recognition net. Pattern Recognition Letters, 2022, 155, 9- 18. doi: 10.1016/j.patrec.2022.01.013
23	DING H, ZHOU P, CHELLAPPA R. Occlusion-adaptive deep network for robust facial expression recognition[C]//Proceedings of IEEE International Joint Conference on Biometrics. Washington D. C., USA: IEEE Press, 2021: 1-9.
24	XIE X Y, MA Y A, LIU B, et al. A deep-learning-based real-time detector for grape leaf diseases using improved convolutional neural networks. Frontiers in Plant Science, 2020, 11, 751. doi: 10.3389/fpls.2020.00751
25	ZHANG J W, YAN X D, CHENG Z L, et al. A face recognition algorithm based on feature fusion. Concurrency and Computation, 2022, 34 (14): e5748.

注意力模块	$ m_{\mathrm{mAP}} $
ScSE	90.22
CBAM	93.34
ECA	96.18
SE	91.53
Asm-CBAM	98.74

注意力模块	$ m_{\mathrm{mAP}} $
ScSE	90.22
CBAM	93.34
ECA	96.18
SE	91.53
Asm-CBAM	98.74

残差模块类型	$ m_{\mathrm{mAP}} $
常规残差模块	98.60
倒置残差模块	97.40
瓶颈残差模块	98.54
密集残差模块	98.74

残差模块类型	$ m_{\mathrm{mAP}} $
常规残差模块	98.60
倒置残差模块	97.40
瓶颈残差模块	98.54
密集残差模块	98.74

模块结构	$ m_{\mathrm{mAP}} $
结构A	98.32
结构B	98.74

Please choose a citation manager

Content to export