基于脑电信号瞬时能量的情感识别方法

doi:10.19678/j.issn.1000-3428.0050486

计算机工程 ›› 2019, Vol. 45 ›› Issue (4): 196-204. doi: 10.19678/j.issn.1000-3428.0050486

基于脑电信号瞬时能量的情感识别方法

陈田^1a,1b,陈占刚^1a,1b,袁晓辉²,鞠思航^1a,1b,任福继^1a,1b,3

1.合肥工业大学 a.计算机与信息学院; b.情感计算与先进智能机器安徽省重点实验室,合肥 230009; 2.北德克萨斯州大学计算机与工程学院,美国丹顿市 76203; 3.德岛大学工学部,日本德岛 770-8506

收稿日期:2018-02-11 出版日期:2019-04-15 发布日期:2019-04-15
作者简介:陈田(1974—),女,副教授,主研方向为情感计算;陈占刚,硕士研究生;袁晓辉,副教授;鞠思航,硕士研究生;任福继,教授。
基金资助:
国家自然科学基金重点项目(61432004);国家自然科学基金(61204046,61474035);国家留学基金(201706695016)。

Emotion Recognition Method Based on Instantaneous Energy of Electroencephalography

CHEN Tian^1a,1b,CHEN Zhan’gang^1a,1b ,YUAN Xiaohui²,JU Sihang^1a,1b,REN Fuji^1a,1b,3

1a.School of Computer and Information;1b.Anhui Province Key Laboratory of Affective Computing and Advanced Intelligent Machine,Hefei University of Technology,Hefei 230009,China; 2.School of Computer and Engineering,University of North Texas,Denton 76203,USA; 3.Faculty of Engineering,The University of Tokushima,Tokushima 770-8506,Japan

Received:2018-02-11 Online:2019-04-15 Published:2019-04-15

摘要/Abstract

摘要：

希尔伯特-黄变换(HHT)是一种处理脑电信号(EEG)的有效方法,包括经验模态分解(EMD)和Hilbert变换2个部分。但EMD无法分解包含低能量的信号,且在低频区域会产生不良的本征模态函数。为消除EMD的弊端,提出一种小波包变换(WPT)和HHT相结合的EEG处理方法。采用WPT将EEG分解成一组窄带信号,通过HHT得到Hilbert能量谱,求出平均瞬时能量作为EEG特征并封装成特征矩阵。将特征矩阵通过卷积神经网络(CNN)、递归神经网络(RNN)、支持向量机(SVM)组成的混合情感识别模型进行训练与分类。实验结果表明,该方法对高兴、悲伤、平静、恐惧4种情感的平均识别率为86.22%,最优识别率为93.45%。

关键词: 脑电信号, 情感识别, 希尔伯特-黄变换, 卷积神经网络, 递归神经网络

Abstract:

Hilbert-Huang Transform(HHT) is an effective method to deal with Electroencephalography(EEG) that includes two parts:Empirical Mode Decomposition(EMD) and Hilbert transform.However,the EMD cannot decompose a signal of low energy and will produce bad Intrinsic Mode Functions(IMF) in the low frequency region.To eliminate the effects of EMD,this paper proposes an EEG processing method which combines Wavelet Packet Transform(WPT) and HHT.Firstly,the EEG is decomposed into a set of narrow-band signals by WPT,the Hilbert energy spectrum of EEG is obtained by HHT,and the average value of the instantaneous energies is calculated as the EEG feature and packaged into a feature matrix.The feature matrix is trained and classified by mixed emotion recognition model which consists of Convolutional Neural Network(CNN),Recurrent Neural Network(RNN),and Support Vector Machine(SVM).Experimental results show that the average recognition rate and the best recognition rate of the four emotions which are happiness,sadness,calmness,and fear are 86.22% and 93.45%.

Key words: Electroencephalography(EEG), emotion recognition, Hilbert-Huang Transform(HHT), Convolutional Neural Network(CNN), Recurrent Neural Network(RNN)

中图分类号:

TP391

陈田,陈占刚,袁晓辉,鞠思航,任福继. 基于脑电信号瞬时能量的情感识别方法[J]. 计算机工程, 2019, 45(4): 196-204.

CHEN Tian,CHEN Zhan’gang,YUAN Xiaohui,JU Sihang,REN Fuji. Emotion Recognition Method Based on Instantaneous Energy of Electroencephalography[J]. Computer Engineering, 2019, 45(4): 196-204.

https://www.ecice06.com/CN/Y2019/V45/I4/196

参考文献

［1］CHU S,NARAYANAN S,KUO C C J.Environmental sound recognition using MP-based features［C］//Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Washington D.C.,USA:IEEE Press,2008:1-4.
［2］CANNON W B.The James-Lange theory of emotions:a critical examination and an alternative theory［J］.The American Journal of Psychology,1987,100(3/4):567-586.
［3］陆文娟.基于脑电信号的情感识别研究［D］.南京:南京邮电大学,2017.
［4］BAJAJ V,PACHORI R B.Detection of human emotions using features based on the multiwavelet transform of EEG signals［M］.Berlin,Germany:Springer,2015:215-240.
［5］ARNAU-GONZLEZ P,AREVALILLO-HERREZ M,RAMZAN N.Fusing highly dimensional energy and connectivity features to identify affective states from EEG signals［J］.Neurocomputing,2017,244:81-89.
［6］LIN Y P,WANG C H,JUNG T P.EEG-based emotion recognition in music listening［J］.IEEE Transactions on Biomedical Engineering,2010,57(7):1798-1806.
［7］KARLSSON L,LOUTFI A.Sleep stage classification using unsupervised feature learning［J］.Advances in Artificial Neural Systems,2012.
［8］LI K,LI X,ZHANG Y.Affective state recognition from EEG with deep belief networks［C］//Proceedings of IEEE International Conference on Bioinformatics and Biomedicine.Washington D.C.,USA:IEEE Press,2013:305-310.
［9］吴琛,周瑞忠.Hilbert-Huang变换在提取地震信号动力特性中的应用［J］.地震工程与工程振动,2006,26(5):41-46.
［10］PETRANTONAKIS P C,HADJILEONTIADIS L J.A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition［J］.IEEE Transactions on Information Technology in Biomedicine,2011,15(5):737-46.
［11］XU Y L,CHEN J.Characterizing nonstationary wind speed using empirical mode decomposition［J］.Journal of Structural Engineering,2004,130(6):912-920.
［12］PENG Z K,TSE P W,CHU F L.An improved Hilbert-Huang transform and its application in vibration signal analysis［J］.Journal of Sound and Vibration,2005,286(1/2):187-205.
［13］赵双乐.浅析小波变换理论及其应用［J］.科技风,2014(15):106-106.
［14］石瑞敏,杨兆建.基于改进EMD的多绳摩擦提升机载荷信息特征提取［J］.煤炭学报,2014,39(4):782-788.
［15］BADSHAH A M,AHMAD J,RAHIM N,et al.Speech emotion recognition from spectrograms with deep convolutional neural network［C］//Proceedings of International Conference on Platform Technology and Service.Washington D.C.,USA:IEEE Press,2017:1-5.
［16］BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult［J］.IEEE Transactions on Neural Networks,1994,5(2):157-166.
［17］李幼军,黄佳进,王海渊.基于SAE和LSTM RNN的多模态生理信号融合和情感识别研究［J］.通信学报,2017,38(12):109-120.
［18］CORTES C,VAPNIK V.Support-vector networks［J］.Machine Learning,1995,20(3):273-297.
［19］NIE D,WANG X W,SHI L C,et al.EEG-based emotion recognition during watching movies［C］//Proceedings of International IEEE/EMBS Conference on Neural Engineering.Washington D.C.,USA:IEEE Press,2011:667-670.
［20］PAUL S,MAZUMDER A,GHOSH P,et al.EEG based emotion recognition system using MFDFA as feature extractor［C］//Proceedings of International Conference on Robotics,Automation,Control and Embedded Systems.Washington D.C.,USA:IEEE Press,2015:1-5.
［21］ZHENG W L,LU B L.Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks［J］.IEEE Transactions on Autonomous Mental Development,2015,7(3):162-175.
［22］LIU S,MENG J,ZHANG D,et al.Emotion recognition based on EEG changes in movie viewing［C］//Proceedings of International IEEE/EMBS Conference on Neural Engineering.Washington D.C.,USA:IEEE Press,2015:1036-1039.
［23］AFTANAS L I,REVA N V,VARLAMOV A A,et al.Analysis of evoked EEG synchronization and desynchronization in conditions of emotional activation in humans:temporal and topographic characteristics［J］.Neuroscience and Behavioral Physiology,2004,34(8):859-67.
［24］JENKE R,PEER A,BUSS M.Feature extraction and selection for emotion recognition from EEG［J］.IEEE Transactions on Affective Computing,2017,5(3):327-339.

[1]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.
[2]	张洋, 刘畅, 李少青. 基于可控制性度量的图神经网络门级硬件木马检测方法[J]. 计算机工程, 2024, 50(7): 164-173.
[3]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[4]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[5]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[6]	于洋, 孙芳芳, 吕华, 李扬, 王晓民. 基于多尺度时空注意力网络的微表情检测方法[J]. 计算机工程, 2024, 50(6): 228-235.
[7]	张雷, 沈国琛, 欧冬秀. 用于热成像数据的卷积神经网络特征图筛选方法[J]. 计算机工程, 2024, 50(4): 31-40.
[8]	张雷, 沈国琛, 欧冬秀. 用于热成像数据的卷积神经网络特征图筛选方法[J]. 计算机工程, 2024, 50(4): 31-40.
[9]	李政学, 李枝名, 彭德中, 陈杰. 基于特征对比学习和图卷积的社交网络用户分类[J]. 计算机工程, 2024, 50(4): 258-266.
[10]	姜百浩, 刘静, 仇大伟, 姜良. 深度学习在脊柱图像分割中的应用综述[J]. 计算机工程, 2024, 50(3): 1-15.
[11]	王柏涵, 姜晓燕, 范柳伊. 基于深度监督隐空间构建的语义分割改进方法[J]. 计算机工程, 2024, 50(3): 191-199.
[12]	谢新林, 尹东旭, 张涛源, 谢刚. 基于注意力机制的多尺度融合人群计数算法[J]. 计算机工程, 2024, 50(3): 290-297.
[13]	赵佳圆, 张玉茹, 苏晓东, 徐红岩, 李世洲, 张玉荣. 基于注意力机制的人体关键点隐式建模网络[J]. 计算机工程, 2024, 50(3): 317-325.
[14]	徐浩宸, 刘满华. 基于多层次自注意力网络的人脸特征点检测[J]. 计算机工程, 2024, 50(2): 239-246.
[15]	李倩, 魏伟波, 杨光宇, 宋金涛, 孙璐, 潘振宽. 基于先验驱动深度神经网络的泊松去噪变分模型[J]. 计算机工程, 2024, 50(2): 273-280.

选择文件类型/文献管理软件名称

选择包含的内容

基于脑电信号瞬时能量的情感识别方法

Emotion Recognition Method Based on Instantaneous Energy of Electroencephalography

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于脑电信号瞬时能量的情感识别方法

Emotion Recognition Method Based on Instantaneous Energy of Electroencephalography

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价