融合多尺度特征的脑电情感识别研究

doi:10.19678/j.issn.1000-3428.0064241

摘要/Abstract

摘要： 基于单尺度二维、三维卷积的脑电情感识别算法存在原始信号映射到高维特征矩阵过程中信息易丢失、模型参数量大、提取特征相对单一等问题。提出多尺度金字塔交互注意力残差网络（MPIAResnet）。利用多尺度一维卷积核直接提取原始脑电信号的多尺度空间特征，将标准卷积替换为分组卷积，相比二维、三维卷积具有更少的参数量，同时利用通道交互注意力机制优化特征提取过程。在此基础上，与双向GRU(BiGRU)融合组成MPIAResnet-BiGRU网络，进一步提取脑电信号的上下文语义信息，实现脑电信号的时空特征融合。基于公开数据集DEAP的实验结果表明:在受试者依赖实验中，该模型Valence和Arousal维度识别准确率达到97.60%和98.15%，相比单尺度模型提升8.56和8.36个百分点；在小批量训练集实验中，当训练集占比为30%时，测试集准确率依然可以保持在90%以上；在分频带实验中，2个高频带信号识别准确率优于低频带信号，证明了模型的有效性；而在受试者全部参与实验中，该模型的识别准确率也均优于对比方法。

关键词: 脑电信号, 多尺度卷积, 注意力机制, 双向GRU, 时空特征

Abstract: Electroencephalogram（EEG） emotion recognition algorithms based on two- and three-dimensional convolutions have several shortcomings，such as loss of information in the process of mapping the original signal to a high-dimensional feature matrix，large number of model parameters，and relatively poor diversity of extracted features.To address the above problems，this paper proposes a Multiscale Pyramid Interactive Attention Residual Convolution Network（MPIAResnet） using a multi-scale one-dimensional convolution kernel to directly extract multi-scale spatial features from the original EEG signals.The standard convolution is replaced by a grouped convolution，which has fewer parameters than two- and three-dimensional convolutions，and feature extraction is optimized using the channel attention mechanism. Finally，it is combined with Bidirectional GRU（BiGRU） to formulate the MPIAResnet-BiGRU network，which further extracts the contextual semantic information of EEG signals to fuse their spatiotemporal features. Experiments are conducted using the open dataset DEAP.In the subject dependent experiment，the recognition accuracy of the model in the Valence and Arousal dimensions reaches 97.60% and 98.15%，respectively，which are 8.56 and 8.36 percentage points higher than those of the single scale model. In small batch training set experiments，when the training set proportion is 30%，the test set accuracy can still be maintained at over 90%.In the frequency division band experiment，the recognition accuracy of two high-frequency band signals was optimized compared to low-frequency band signals，which proves the effectiveness of the model.In the experiments involving all subjects，the proposed model achieves superior recognition accuracy to the comparison method.

Key words: Electroencephalogram（EEG） signal, multi-scale convolution, attention mechanism, Bidirectional GRU（BiGRU）, spatio-temporal features

中图分类号:

TP18

焦义, 徐华兴, 毛晓波, 李楠, 姚国梁, 倪金红, 徐向阳. 融合多尺度特征的脑电情感识别研究[J]. 计算机工程, 2023, 49(5): 81-89,96.

JIAO Yi, XU Huaxing, MAO Xiaobo, LI Nan, YAO Guoliang, NI Jinhong, XU Xiangyang. Research on EEG Emotion Recognition with Fusion of Multi-Scale Features[J]. Computer Engineering, 2023, 49(5): 81-89,96.

http://www.ecice06.com/CN/Y2023/V49/I5/81

图/表 16

20230515185132

20230515185137

20230515185141

20230515185144

20230515185148

20230515185153

20230515185157

20230515185200

20230515185204

20230515185208

20230515185213

20230515185216

20230515185220

20230515185224

20230515185228

20230515185233

参考文献

[1] GROSS J J.Emotion regulation:conceptual and empirical foundations[M].New York,USA:Guilford Press,2014:3-20.
[2] LIU Y,DING Y F,LI C,et al.Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network[J].Computers in Biology and Medicine,2020,123(5596):1-11.
[3] ALI M,AL MACHOT F,MOSA A H,et al.A novel EEG-based emotion recognition approach for E-healthcare applications[C]//Proceedings of the 31st Annual ACM Symposium on Applied Computing.New York,USA:ACM Press,2016:162-164.
[4] ALARCÃO S M,FONSECA M J.Emotions recognition using EEG signals:a survey[J].IEEE Transactions on Affective Computing,2019,10(3):374-393.
[5] CHENG J,CHEN M Y,LI C,et al.Emotion recognition from multi-channel EEG via deep forest[J].IEEE Journal of Biomedical and Health Informatics,2021,25(2):453-464.
[6] SHUKLA J,BARREDA-ÁNGELES M,OLIVER J,et al.Feature extraction and selection for emotion recognition from electrodermal activity[J].IEEE Transactions on Affective Computing,2021,12(4):857-869.
[7] ACKERMANN P,KOHLSCHEIN C,BITSCH J Á,et al.EEG-based automatic emotion recognition:feature extraction,selection and classification methods[C]//Proceedings of the 18th IEEE International Conference on e-Health Networking,Applications and Services.Washington D.C.,USA:IEEE Press,2016:1-6.
[8] SULTHAN N,MOHAN N,AHMAD K K,et al.Emotion recognition using brain signals[C]//Proceedings of International Conference on Intelligent Circuits and Systems.Washington D.C.,USA:IEEE Press,2018:315-319.
[9] LI C,TAO W,CHENG J,et al.Robust multichannel EEG compressed sensing in the presence of mixed noise[J].IEEE Sensors Journal,2019,19(22):10574-10583.
[10] DUAN R N,ZHU J Y,LU B L.Differential entropy feature for EEG-based emotion classification[C]//Proceedings of the 6th International IEEE/EMBS Conference on Neural Engineering.Washington D.C.,USA:IEEE Press,2014:81-84.
[11] LI M,LU B L.Emotion classification based on Gamma-band EEG[C]//Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Washington D.C.,USA:IEEE Press,2009:1223-1226.
[12] CRAIK A,HE Y T.Deep learning for Electroencephalogram(EEG) classification tasks:a review[J].Journal of Neural Engineering,2019,16(3):28-56.
[13] GOH S K,ABBASS H A,TAN K C,et al.Spatio-spectral representation learning for electroencephalographic gait-pattern classification[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering:a Publication of the IEEE Engineering in Medicine and Biology Society,2018,26(9):1858-1867.
[14] ALHAGRY S,FAHMY A A,EL-KHORIBI R A.Emotion recognition based on EEG using LSTM recurrent neural network[J].Emotion,2017,8(10):355-358.
[15] SHAWKY E,EL-KHORIBI R,SHOMAN M A I,et al.EEG-based emotion recognition using 3d convolutional neural networks[J].International Journal of Advanced Computer Science and Applications,2018,9(8):329-337.
[16] SONG T F,ZHENG W M,SONG P,et al.EEG emotion recognition using dynamical graph convolutional neural networks[J].IEEE Transactions on Affective Computing,2020,11(3):532-541.
[17] CIMTAY Y,EKMEKCIOGLU E.Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition[J].Sensors(Basel,Switzerland),2020,20(7):2034.
[18] FDEZ J,GUTTENBERG N,WITKOWSKI O,et al.Cross-subject EEG-based emotion recognition through neural networks with stratified normalization[J].Frontiers in Neuroscience,2021,15:1-14.
[19] LI D D,CHAI B,WANG Z,et al.EEG emotion recognition based on 3-D feature representation and dilated fully convolutional networks[J].IEEE Transactions on Cognitive and Developmental Systems,2021,13(4):885-897.
[20] MRI C,MMI B,MMR D,et al.EEG channel correlation based model for emotion recognition[J].Computers in Biology and Medicine,2021,136(14):1-11.
[21] KOELSTRA S,MUHL C,SOLEYMANI M,et al.DEAP:a database for emotion analysis; using physiological signals[J].IEEE Transactions on Affective Computing,2012,3(1):18-31.
[22] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[23] 李晓,卢先领.基于双重注意力机制和GRU网络的短期负荷预测模型[J].计算机工程,2022,48(2):291-296,305. LI X,LU X L.Method for forecasting short-term power load based on dual-stage attention mechanism and gated recurrent unit network[J].Computer Engineering,2022,48(2):291-296,305.(in Chinese)
[24] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[25] CHUNG J,GULCEHRE C,CHO K,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].[2022-01-20].https://arxiv.org/abs/1412.3555.
[26] WANG H,LIU Z L,PENG D D,et al.Understanding and learning discriminant features based on multiattention 1DCNN for wheelset bearing fault diagnosis[J].IEEE Transactions on Industrial Informatics,2020,16(9):5735-5745.
[27] YANG Y L,WU Q F,QIU M,et al.Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network[C]//Proceedings of International Joint Conference on Neural Networks.Washington D.C.,USA:IEEE Press,2018:1-7.
[28] LU Y F,ZHENG W L,LI B B,et al.Combining eye movements and EEG to enhance emotion recognition[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2015:1170-1176.
[29] 聂聃,王晓韡,段若男,等.基于脑电的情绪识别研究综述[J].中国生物医学工程学报,2012,31(4):595-606. NIE D,WANG X W,DUAN R N,et al.A survey on EEG based emotion recognition[J].Chinese Journal of Biomedical Engineering,2012,31(4):595-606.(in Chinese)
[30] YIN Y Q,ZHENG X W,HU B,et al.EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM[J].Applied Soft Computing,2021,100:1-44.
[31] TAO W,LI C,SONG R C,et al.EEG-based emotion recognition via channel-wise attention and self attention[J].IEEE Transactions on Affective Computing,2023,14(1):382-393.
[32] 陈景霞,郝为,张鹏伟,等.基于混合神经网络的脑电时空特征情感分类[J].软件学报,2021,32(12):3869-3883. CHEN J X,HAO W,ZHANG P W,et al.Emotion classification of spatiotemporal EEG features using hybrid neural networks[J].Journal of Software,2021,32(12):3869-3883.(in Chinese)
[33] 孟明,胡家豪,高云园,等.结合互信息通道选择与混合深度神经网络的脑电情感识别方法[J].传感技术学报,2021,34(8):1089-1095. MENG M,HU J H,GAO Y Y,et al.EEG emotion recognition based on normalized mutual information channel selection and hybrid deep neural network[J].Chinese Journal of Sensors and Actuators,2021,34(8):1089-1095.(in Chinese)
[34] XING X F,LI Z Q,XU T Y,et al.SAE+LSTM:a new framework for emotion recognition from multi-channel EEG[J].Frontiers in Neurorobotics,2019,13:37.
[35] YANG H,HAN J,MIN K.A multi-column CNN model for emotion recognition from EEG signals[J].Sensors(Basel,Switzerland),2019,19(21):4736.
[36] CHAO H,DONG L,LIU Y L,et al.Emotion recognition from multiband EEG signals using CapsNet[J].Sensors(Basel,Switzerland),2019,19(9):2212.
[37] 周奕隽,李冬冬,王喆,等.融合倒谱特征的脑电(EEG)情感分类[J].计算机工程与应用,2020,56(21):164-169. ZHOU Y J,LI D D,WANG Z,et al.Cepstrum feature fusion for EEG emotion classification[J].Computer Engineering and Applications,2020,56(21):164-169.(in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容