基于多核学习特征融合的语音情感识别方法

doi:10.19678/j.issn.1000-3428.0053232

计算机工程 ›› 2019, Vol. 45 ›› Issue (8): 248-254. doi: 10.19678/j.issn.1000-3428.0053232

基于多核学习特征融合的语音情感识别方法

王忠民^a,b, 刘戈^a, 宋辉^a,b

西安邮电大学 a. 计算机学院;b. 陕西省网络数据分析与智能处理重点实验室, 西安 710121

收稿日期:2018-11-23 修回日期:2019-01-11 出版日期:2019-08-15 发布日期:2019-08-08
作者简介:王忠民(1967-),男,教授、博士,主研方向为语音情感识别、嵌入式系统、智能信息处理;刘戈,硕士研究生;宋辉,讲师、硕士。
基金资助:
国家自然科学基金（61373116）；陕西省科技统筹创新工程计划项目（2016KTZDGY04-01）；陕西省教育厅专项科研计划项目（16JK1706）；西安市科技局科技计划项目（2017084CG/RC047（XAYD001））；西安邮电大学研究生创新创业基金（CXJJ2017061）。

Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion

WANG Zhongmin^a,b, LIU Ge^a, SONG Hui^a,b

a. School of Computer Science and Technology;b. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an University of Posts and Telecommunications, Xi'an 710121, China

Received:2018-11-23 Revised:2019-01-11 Online:2019-08-15 Published:2019-08-08

摘要/Abstract

摘要： 在语音情感识别中提取梅尔频率倒谱系数（MFCC）会丢失谱特征信息，导致情感识别准确率较低。为此，提出一种结合MFCC和语谱图特征的语音情感识别方法。从音频信号中提取MFCC特征，将信号转换为语谱图，利用卷积神经网络提取图像特征。在此基础上，使用多核学习算法融合音频特征，并将生成的核函数应用于支持向量机进行情感分类。在2种语音情感数据集上的实验结果表明，与单一特征的分类器相比，该方法的语音情感识别准确率高达96%。

关键词: 语音情感识别, 多核学习, 卷积神经网络, 梅尔频率倒谱系数, 语谱图

Abstract: Extracting the Mel-Frequency Cepstral Coefficients(MFCC) in speech emotion recognition will lose the spectral feature information,resulting in a low accuracy of emotion recognition.Therefore,a speech emotion recognition method combining MFCC and spectrogram features is proposed.The MFCC features are extracted from the audio signal,the signal is converted into a spectral map,and the image features are extracted using a Convolutional Neural Network(CNN).On this basis,the Multiple Kernel Learning(MKL) algorithm is used to fuse the audio features,and the generated kernel functions are used to support the vector machine for emotion classification.Experimental results in two kinds of speech emotion data sets show that the speech emotion recognition accuracy of this method is as high as 96% compared with the classifier based on single feature.

Key words: speech emotion recognition, Multiple Kernel Learning(MKL), Convolution Neural Network(CNN), Mel-Frequency Cepstral Coefficients(MFCC), spectrogram

中图分类号:

TP391.4

王忠民, 刘戈, 宋辉. 基于多核学习特征融合的语音情感识别方法[J]. 计算机工程, 2019, 45(8): 248-254.

WANG Zhongmin, LIU Ge, SONG Hui. Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion[J]. Computer Engineering, 2019, 45(8): 248-254.

https://www.ecice06.com/CN/Y2019/V45/I8/248

参考文献 27

[1]	DAHAKE P P,SHAW K,MALATHI P.Speaker dependent speech emotion recognition using MFCC and support vector machine[C]//Proceedings of International Conference on Automatic Control and Dynamic Optimization Techniques.Washington D.C.,USA:IEEE Press,2017:1080-1084.
[2]	韩文静.语音情感识别关键技术研究[D].哈尔滨:哈尔滨工业大学,2013.
[3]	LI Longfei,ZHAO Yong,JIANG Dongmei,et al.Hybrid deep neural network-hidden markov model based speech emotion recognition[C]//Proceedings of Humaine Association Conference on Affective Computing and Intelligent Interaction.Washington D.C.,USA:IEEE Press,2013:312-317.
[4]	SCHULLER B,MÜLLER R,LANG M,et al.Speaker independent emotion recognition by early fusion of acoustic and linguistic features within ensembles[C]//Proceedings of the 9th European Conference on Speech Communication and Technology.[S.l.]:Curran Associates Inc.,2005:805-808.
[5]	ÖZSEVEN T.Investigation of the effect of spectrogram images and different texture analysis methods on speech emotion recognition[J].Applied Acoustics,2018,142:70-77.
[6]	MIRSAMADI S,BARSOUM E,ZHANG Cha.Auto-matic speech emotion recognition using recurrent neural networks with local attention[C]//Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Washington D.C.,USA:IEEE Press,2017:2227-2231.
[7]	HUANG C W,NARAYANAN S S.Deep convolutional recurrent neural network with attention mechanism for robust speech emotion recognition[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C.,USA:IEEE Press,2017:583-588.
[8]	XIA Rui,LIU Yang.A multi-task learning framework for emotion recognition using 2D continuous space[J].IEEE Transactions on Affective Computing,2017,8(1):3-14.
[9]	卢官明,袁亮,杨文娟,等.基于长短期记忆和卷积神经网络的语音情感识别[J].南京邮电大学学报(自然科学版),2018,38(5):63-69.
[10]	陈晓敏.基于时序深度学习模型的语音情感识别方法研究[D].哈尔滨:哈尔滨工业大学,2018.
[11]	曾润华,张树群.改进卷积神经网络的语音情感识别方法[J].应用科学学报,2018,36(5):837-844
[12]	HAN Kun,YU Dong,TASHEV I.Speech emotion recognition using deep neural network and extreme learning machine[EB/OL].[2018-10-20].http://193.6.4.39/~czap/letoltes/IS14/IS2014/PDF/AUTHOR/IS140441.PDF. URL
[13]	LIM W,JANG D,LEE T.Speech emotion recognition using convolutional and recurrent neural networks[C]//Proceedings of Signal and Information Processing Asso-ciation Annual Summit and Conference.Washington D.C.,USA:IEEE Press,2016:1-4.
[14]	FAYEK H M,LECH M,CAVEDON L.Evaluating deep learning architectures for speech emotion recognition[J].Neural Networks,2017,92(8):60-68.
[15]	汪洪桥,孙富春,蔡艳宁,等.多核学习方法[J].自动化学报,2010,36(8):1037-1050.
[16]	RAKOTOMAMONJY A,BACH F R,CANU S,et al.SimpleMKL[J].Journal of Machine Learning Research,2008,9(11):2491-2521.
[17]	ONG C S,SMOLA A J,WILLIAMSON R C.Learning the kernel with hyperkernels[J].Journal of Machine Learning Research,2005,6(7):1043-1071.
[18]	AIOLLI F,DONINI M.EasyMKL:a scalable multiple kernel learning algorithm[J].Neurocomputing,2015,169:215-224.
[19]	PORIA S,CHATURVEDI I,CAMBRIA E,et al.Convolutional MKL based multimodal emotion recognition and sentiment analysis[C]//Proceedings of the 16th International Conference on Data Mining.Washington D.C.,USA:IEEE Press,2016:439-448.
[20]	JIN Yun,SONG Peng,ZHENG Wenming,et al.Novel feature fusion method for speech emotion recognition based on multiple kernel learning[J].Journal of Southeast University,2013,29(2):129-133.
[21]	JIN Yun,SONG Peng,ZHENG Wenming,et al.Speaker-independent speech emotion recognition based on two-layer multiple kernel learning[J].IEICE Transactions on Information and Systems,2013,96(10):2286-2289.
[22]	XU Xinzhou,DENG Jun,CUMMINS N,et al.A two-dimensional framework of multiple kernel subspace learning for recognizing emotion in speech[J].IEEE/ACM Transactions on Audio,Speech and Language Processing,2017,25(7):1436-1449.
[23]	SUN Yaxin,WEN Guihua,WANG Jiabing.Weighted spectral features based on local hu moments for speech emotion recognition[J].Biomedical Signal Processing and Control,2015,18(4):80-90.
[24]	MAO Qirong,DONG Ming,HUANG Zhengwei,et al.Learning salient features for speech emotion recognition using convolutional neural networks[J].IEEE Tran-sactions on Multimedia,2014,16(8):2203-2213.
[25]	BURKHARDT F,PAESCHKE A,ROLFES M,et al.A database of German emotional speech[C]//Proceedings of the 9th European Conference on Speech Communication and Technology.[S.l.]:Curran Associates Inc.,2005:1-4.
[26]	KUCHIBHOTLA S,VANKAYALAPATI H D,ANNE K R.An optimal two stage feature selection for speech emotion recognition using acoustic features[J].Inter-national Journal of Speech Technology,2016,19(4):657-667.
[27]	ZHU Bing,ZHOU Wenkai,WANG Yutian,et al.End-to-end speech emotion recognition based on neural network[C]//Proceedings of the 17th International Con-ference on Communication Technology.Washington D.C.,USA:IEEE Press,2017:1634-1638.

选择文件类型/文献管理软件名称

选择包含的内容

基于多核学习特征融合的语音情感识别方法

Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.
[2]	张洋, 刘畅, 李少青. 基于可控制性度量的图神经网络门级硬件木马检测方法[J]. 计算机工程, 2024, 50(7): 164-173.
[3]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[4]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[5]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[6]	于洋, 孙芳芳, 吕华, 李扬, 王晓民. 基于多尺度时空注意力网络的微表情检测方法[J]. 计算机工程, 2024, 50(6): 228-235.
[7]	张雷, 沈国琛, 欧冬秀. 用于热成像数据的卷积神经网络特征图筛选方法[J]. 计算机工程, 2024, 50(4): 31-40.
[8]	张雷, 沈国琛, 欧冬秀. 用于热成像数据的卷积神经网络特征图筛选方法[J]. 计算机工程, 2024, 50(4): 31-40.
[9]	李政学, 李枝名, 彭德中, 陈杰. 基于特征对比学习和图卷积的社交网络用户分类[J]. 计算机工程, 2024, 50(4): 258-266.
[10]	姜百浩, 刘静, 仇大伟, 姜良. 深度学习在脊柱图像分割中的应用综述[J]. 计算机工程, 2024, 50(3): 1-15.
[11]	王柏涵, 姜晓燕, 范柳伊. 基于深度监督隐空间构建的语义分割改进方法[J]. 计算机工程, 2024, 50(3): 191-199.
[12]	谢新林, 尹东旭, 张涛源, 谢刚. 基于注意力机制的多尺度融合人群计数算法[J]. 计算机工程, 2024, 50(3): 290-297.
[13]	赵佳圆, 张玉茹, 苏晓东, 徐红岩, 李世洲, 张玉荣. 基于注意力机制的人体关键点隐式建模网络[J]. 计算机工程, 2024, 50(3): 317-325.
[14]	徐浩宸, 刘满华. 基于多层次自注意力网络的人脸特征点检测[J]. 计算机工程, 2024, 50(2): 239-246.
[15]	李倩, 魏伟波, 杨光宇, 宋金涛, 孙璐, 潘振宽. 基于先验驱动深度神经网络的泊松去噪变分模型[J]. 计算机工程, 2024, 50(2): 273-280.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于多核学习特征融合的语音情感识别方法

Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价