基于动态脑网络特征的情绪识别方法

doi:10.19678/j.issn.1000-3428.0069633

摘要/Abstract

摘要： 情绪识别是人机交互(HCI)与情感智能领域的重要前沿课题之一。然而,目前基于脑电(EGG)信号的情绪识别方法主要提取静态特征,无法挖掘情绪的动态变化特性,难以提升情绪识别能力。在基于EGG构建动态脑功能网络的研究中,常采用滑动窗口方法,通过依次构建不同窗口内的功能连接网络以形成动态网络。但该方法存在主观设定窗长的问题,无法提取每个时间点情绪状态的连接模式,导致时间信息丢失和脑连接信息不完整。针对上述问题,提出动态线性相位测量(dyPLM)方法,该方法无需使用滑窗,即可自适应地在每个时间点构建情绪相关脑网络,更精准地刻画情绪的动态变化特性。此外,还提出一种卷积门控神经网络(CNGRU)情绪识别模型,该模型可进一步提取动态脑网络深层次特征,有效提高情绪识别准确性。在公开情绪识别脑电数据集DEAP(Database for Emotion Analysis using Physiological signals)上进行验证,所提方法四分类准确率高达99.71%,较MFBPST-3D-DRLF提高3.51百分点。在SEED(SJTU Emotion EEG Dataset)数据集上进行验证,所提方法三分类准确率达到99.99%,较MFBPST-3D-DRLF提高3.32百分点。实验结果证明了所提出的动态脑网络构建方法dyPLM和情绪识别模型CNGRU的有效性和实用性。

关键词: 脑电信号, 情绪识别, 动态脑网络, 卷积神经网络, 门控循环单元

Abstract: Emotion recognition is one of the most important frontier research topics in the field of Human-Computer Interaction (HCI) emotional intelligence. However, at present, Electroencephalogram (EEG)-based emotion recognition extracts static features and cannot mine the dynamic characteristics of emotions, and it is difficult to improve the emotion recognition ability. In current research on the construction of dynamic Brain Functional Networks (dBFNs) using EEG signals, a sliding window is usually used to form a dBFN by sequentially constructing a functional connectivity network in different windows. However, this method is limited by subjectively setting the window length and cannot extract the connection mode of the emotional state at each time point. Therefore, the loss of temporal information leads to the loss of brain connection information. To solve these problems, this study proposes a dynamic Phase Linearity Measurement (dyPLM) method that can adaptively construct an emotion-related brain network at each time point without sliding windows to characterize the dynamic characteristics of emotions. In addition, a Convolutional Neural Gate Recurrent Unit (CNGRU) emotion recognition model is proposed that could further extract the deep-seated features of the dynamic brain network and effectively improve the accuracy of emotion recognition. In experiments on the public emotion recognition EEG dataset, Database for Emotion Analysis using Physiological signals (DEAP), the four-class classification accuracy is as high as 99.71%, and the recognition accuracy is improved by 3.51 percentage points compared to MFBPST-3D-DRLF. On the SJTU Emotion EEG Dataset (SEED), the three-class classification accuracy is 99.99%, and the recognition accuracy is improved by 3.32 percentage points compared to MFBPST-3D-DRLF. This study demonstrates the effectiveness and practicality of the proposed methods.

Key words: Electroencephalogram (EEG) signal, emotion recognition, dynamic brain network, convolutional neural network, Gate Recurrent Unit (GRU)

中图分类号:

TH79

王海玲, 姜廷威, 方志军, 高宇飞. 基于动态脑网络特征的情绪识别方法[J]. 计算机工程, 2026, 52(2): 125-135.

WANG Hailing, JIANG Tingwei, FANG Zhijun, GAO Yufei. Emotion Recognition Method Based on Dynamic Brain Network Feature[J]. Computer Engineering, 2026, 52(2): 125-135.

https://www.ecice06.com/CN/Y2026/V52/I2/125

参考文献

[1] ALARCAO S M, FONSECA M J. Emotions recognition using EEG signals: a survey[J]. IEEE Transactions on Affective Computing, 2019, 10(3): 374-393.
[2] LI J Y, HUA H Q, XU Z H, et al. Cross-subject EEG emotion recognition combined with connectivity features and meta-transfer learning[J]. Computers in Biology and Medicine, 2022, 145: 105519.
[3] LI C, HOU Y M, SONG R C, et al. Multi-channel EEG-based emotion recognition in the presence of noisy labels[J]. Science China Information Sciences, 2022, 65(4): 140405.
[4] 王夙喆, 张雪英, 陈晓玉, 等. 基于有效注意力和GAN结合的脑卒中EEG增强算法[J]. 计算机工程, 2024, 50(8): 336-344. WANG S Z, ZHANG X Y, CHEN X Y, et al. EEG enhancement algorithm based on combination of effective attention and GAN[J]. Computer Engineering, 2024, 50(8): 336-344. (in Chinese)
[5] ALHAGRY S, FAHMY A A, EL-KHORIBI R A. Emotion recognition based on EEG using LSTM recurrent neural network[J]. International Journal of Advanced Computer Science and Applications, 2017, 8(10): 355-358.
[6] NAJI M, FIROOZABADI M, AZADFALLAH P. Classification of music-induced emotions based on information fusion of forehead biosignals and electrocardiogram[J]. Cognitive Computation, 2014, 6(2): 241-252.
[7] PETRANTONAKIS P C, HADJILEONTIADIS L J. Emotion recognition from EEG using higher order crossings[J]. IEEE Transactions on Information Technology in Biomedicine, 2010, 14(2): 186-197.
[8] LI P, LIU H, SI Y, et al. EEG based emotion recognition by combining functional connectivity network and local activations[J]. IEEE Transactions on Biomedical Engineering, 2019, 66(10): 2869-2881.
[9] LI X, ZHANG Y, TIWARI P, et al. EEG based emotion recognition: a tutorial and review[J]. ACM Computing Surveys, 2023, 55(4): 1-57.
[10] ZHENG W L, ZHU J Y, LU B L. Identifying stable patterns over time for emotion recognition from EEG[J]. IEEE Transactions on Affective Computing, 2019, 10(3): 417-429.
[11] ZHANG Y C, YAN G H, CHANG W W, et al. EEG-based multi-frequency band functional connectivity analysis and the application of spatio-temporal features in emotion recognition[J]. Biomedical Signal Processing and Control, 2023, 79: 104157.
[12] WU X, ZHENG W L, LI Z Y, et al. Investigating EEG-based functional connectivity patterns for multimodal emotion recognition[J]. Journal of Neural Engineering, 2022, 19(1): 016012.
[13] STAM C J, NOLTE G, DAFFERTSHOFER A. Phase lag index: assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources[J]. Human Brain Mapping, 2007, 28(11): 1178-1193.
[14] BASELICE F, SORRISO A, RUCCO R, et al. Phase linearity measurement: a novel index for brain functional connectivity[J]. IEEE Transactions on Medical Imaging, 2019, 38(4): 873-882.
[15] WU X, ZHENG W L, LU B L. Identifying functional brain connectivity patterns for EEG-based emotion recognition[C]//Proceedings of the 9th International IEEE/EMBS Conference on Neural Engineering (NER). San Francisco, USA: IEEE Press, 2019: 235-238.
[16] GUPTA R, FALK T H. Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization[J]. Neurocomputing, 2016, 174(1): 875-884.
[17] CHEN C, LI Z, WAN F, et al. Fusing frequency-domain features and brain connectivity features for cross-subject emotion recognition[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-15.
[18] GUO W H, LI Y X, LIU M X, et al. Functional connectivity-enhanced feature-grouped attention network for cross-subject EEG emotion recognition[J]. Knowledge-Based Systems, 2024, 283: 111199.
[19] LIU X C, LI T, TANG C, et al. Emotion recognition and dynamic functional connectivity analysis based on EEG[J]. IEEE Access, 2019, 7: 143293-143302.
[20] O’NEILL G C, TEWARIE P K, COLCLOUGH G L, et al. Measurement of dynamic task related functional networks using MEG[J]. NeuroImage, 2017, 146: 667-678.
[21] RIZKALLAH J, ANNEN J, MODOLO J, et al. Decreased integration of EEG source-space networks in disorders of consciousness[J]. NeuroImage: Clinical, 2019, 23: 101841.
[22] NOBUKAWA S, KIKUCHI M, TAKAHASHI T. Changes in functional connectivity dynamics with aging: a dynamical phase synchronization approach[J]. Neuroimage, 2019, 188: 357-368.
[23] YANG J, CHEN S S, HUANGFU H R, et al. Dynamic functional connectivity of electroencephalogram in the resting state[J]. Acta Physica Sinica, 2015, 64(5): 058701.
[24] XEFTERIS V R, TSANOUSA A, GEORGAKOPOULOU N, et al. Graph theoretical analysis of EEG functional connectivity patterns and fusion with physiological signals for emotion recognition[J]. Sensors, 2022, 22(21): 8198.
[25] 李存波, 杨蕾, 陈昭瑾, 等. 基于脑电网络图特征的情绪识别研究[J]. 数据采集与处理, 2023, 38(4): 815-823. LI C B, YANG L, CHEN Z J, et al. Emotion recognition based on graph features extracted from EEG networks[J]. Journal of Data Acquisition and Processing, 2023, 38(4): 815-823. (in Chinese)
[26] STOCKWELL R G, MANSINHA L, LOWE R P. Localization of the complex spectrum: the S transform[J]. IEEE Transactions on Signal Processing, 1996, 44(4): 998-1001.
[27] KIRANYAZ S, INCE T, HAMILA R, et al. Convolutional neural networks for patient-specific ECG classification[C]//Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Washington D. C., USA: IEEE Press, 2015: 2608-2611.
[28] KIRANYAZ S, AVCI O, ABDELJABER O, et al. 1D convolutional neural networks and applications: a survey[J]. Mechanical Systems and Signal Processing, 2021, 151: 107398.
[29] LU L M, ZHANG C L, CAO K, et al. A multichannel CNN-GRU model for human activity recognition[J]. IEEE Access, 2022, 10: 66797-66810.
[30] ZHONG M Y, YANG Q Y, LIU Y, et al. EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network[J]. Biomedical Signal Processing and Control, 2023, 79: 104211.
[31] BAGHERZADEH S, SHAHABI M S, SHALBAF A. Detection of schizophrenia using hybrid of deep learning and brain effective connectivity image from electroencephalogram signal[J]. Computers in Biology and Medicine, 2022, 146: 105570.
[32] DEY R, SALEM F M. Gate-variants of Gated Recurrent Unit (GRU) neural networks[C]//Proceedings of the IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS). Boston, USA: IEEE Press, 2017: 1597-1600.
[33] CHUNG J, GULCEHRE C, CHO K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].[2024-01-02]. https://arxiv.org/abs/1412.3555.
[34] KOELSTRA S, MUHL C, SOLEYMANI M, et al. DEAP: a database for emotion analysis; using physiological signals[J]. IEEE Transactions on Affective Computing, 2012, 3(1): 18-31.
[35] MERT A, AKAN A. Emotion recognition from EEG signals by using multivariate empirical mode decomposition[J]. Pattern Analysis and Applications, 2018, 21(1): 81-89.
[36] SUTSKEVER I, MARTENS J, DAHL G, et al. On the importance of initialization and momentum in deep learning[C]//Proceedings of the 30th International Conference on Machine Learning. Atlanta , USA:[s. n.], 2013: 1-10.
[37] YIN Y Q, ZHENG X W, HU B, et al. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM[J]. Applied Soft Computing, 2021, 100: 106954.
[38] 廉小亲, 罗志宏, 蔡沫豪, 等. 基于卷积神经网络的脑电情绪识别方法[J]. 计算机仿真, 2022, 39(8): 268-274. LIAN X Q, LUO Z H, CAI M H, et al. EEG emotion recognition method based on convolutional neural network[J]. Computer Simulation, 2022, 39(8): 268-274. (in Chinese)
[39] QIN C, LANLAN C, RUNQIANG J. Emotion recognition of EEG based on ensemble CapsNet[J]. Journal of Computer Engineering & Applications, 2022, 58(8): 175-184.
[40] ZHENG W, PAN B. A spatiotemporal symmetrical transformer structure for EEG emotion recognition[J]. Biomedical Signal Processing and Control, 2024, 87: 105487.
[41] MIAO M M, ZHENG L X, XU B G, et al. A multiple frequency bands parallel spatial—temporal 3D deep residual learning framework for EEG-based emotion recognition[J]. Biomedical Signal Processing and Control, 2023, 79: 104141.
[42] XIAO G W, SHI M, YE M W, et al. 4D attention-based neural network for EEG emotion recognition[J]. Cognitive Neurodynamics, 2022, 16(4): 805-818.

选择文件类型/文献管理软件名称

选择包含的内容