基于多核学习特征融合的语音情感识别方法

doi:10.19678/j.issn.1000-3428.0053232

计算机工程 ›› 2019, Vol. 45 ›› Issue (8): 248-254. doi: 10.19678/j.issn.1000-3428.0053232

基于多核学习特征融合的语音情感识别方法

王忠民^a,b, 刘戈^a, 宋辉^a,b

西安邮电大学 a. 计算机学院;b. 陕西省网络数据分析与智能处理重点实验室, 西安 710121

收稿日期:2018-11-23 修回日期:2019-01-11 出版日期:2019-08-15 发布日期:2019-08-08
作者简介:王忠民(1967-),男,教授、博士,主研方向为语音情感识别、嵌入式系统、智能信息处理;刘戈,硕士研究生;宋辉,讲师、硕士。
基金资助:
国家自然科学基金（61373116）；陕西省科技统筹创新工程计划项目（2016KTZDGY04-01）；陕西省教育厅专项科研计划项目（16JK1706）；西安市科技局科技计划项目（2017084CG/RC047（XAYD001））；西安邮电大学研究生创新创业基金（CXJJ2017061）。

Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion

WANG Zhongmin^a,b, LIU Ge^a, SONG Hui^a,b

a. School of Computer Science and Technology;b. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an University of Posts and Telecommunications, Xi'an 710121, China

Received:2018-11-23 Revised:2019-01-11 Online:2019-08-15 Published:2019-08-08

摘要/Abstract

摘要： 在语音情感识别中提取梅尔频率倒谱系数（MFCC）会丢失谱特征信息，导致情感识别准确率较低。为此，提出一种结合MFCC和语谱图特征的语音情感识别方法。从音频信号中提取MFCC特征，将信号转换为语谱图，利用卷积神经网络提取图像特征。在此基础上，使用多核学习算法融合音频特征，并将生成的核函数应用于支持向量机进行情感分类。在2种语音情感数据集上的实验结果表明，与单一特征的分类器相比，该方法的语音情感识别准确率高达96%。

关键词: 语音情感识别, 多核学习, 卷积神经网络, 梅尔频率倒谱系数, 语谱图

Abstract: Extracting the Mel-Frequency Cepstral Coefficients(MFCC) in speech emotion recognition will lose the spectral feature information,resulting in a low accuracy of emotion recognition.Therefore,a speech emotion recognition method combining MFCC and spectrogram features is proposed.The MFCC features are extracted from the audio signal,the signal is converted into a spectral map,and the image features are extracted using a Convolutional Neural Network(CNN).On this basis,the Multiple Kernel Learning(MKL) algorithm is used to fuse the audio features,and the generated kernel functions are used to support the vector machine for emotion classification.Experimental results in two kinds of speech emotion data sets show that the speech emotion recognition accuracy of this method is as high as 96% compared with the classifier based on single feature.

Key words: speech emotion recognition, Multiple Kernel Learning(MKL), Convolution Neural Network(CNN), Mel-Frequency Cepstral Coefficients(MFCC), spectrogram

中图分类号:

TP391.4

王忠民, 刘戈, 宋辉. 基于多核学习特征融合的语音情感识别方法[J]. 计算机工程, 2019, 45(8): 248-254.

WANG Zhongmin, LIU Ge, SONG Hui. Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion[J]. Computer Engineering, 2019, 45(8): 248-254.

http://www.ecice06.com/CN/Y2019/V45/I8/248

参考文献 27

[1]	DAHAKE P P,SHAW K,MALATHI P.Speaker dependent speech emotion recognition using MFCC and support vector machine[C]//Proceedings of International Conference on Automatic Control and Dynamic Optimization Techniques.Washington D.C.,USA:IEEE Press,2017:1080-1084.
[2]	韩文静.语音情感识别关键技术研究[D].哈尔滨:哈尔滨工业大学,2013.
[3]	LI Longfei,ZHAO Yong,JIANG Dongmei,et al.Hybrid deep neural network-hidden markov model based speech emotion recognition[C]//Proceedings of Humaine Association Conference on Affective Computing and Intelligent Interaction.Washington D.C.,USA:IEEE Press,2013:312-317.
[4]	SCHULLER B,MÜLLER R,LANG M,et al.Speaker independent emotion recognition by early fusion of acoustic and linguistic features within ensembles[C]//Proceedings of the 9th European Conference on Speech Communication and Technology.[S.l.]:Curran Associates Inc.,2005:805-808.
[5]	ÖZSEVEN T.Investigation of the effect of spectrogram images and different texture analysis methods on speech emotion recognition[J].Applied Acoustics,2018,142:70-77.
[6]	MIRSAMADI S,BARSOUM E,ZHANG Cha.Auto-matic speech emotion recognition using recurrent neural networks with local attention[C]//Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Washington D.C.,USA:IEEE Press,2017:2227-2231.
[7]	HUANG C W,NARAYANAN S S.Deep convolutional recurrent neural network with attention mechanism for robust speech emotion recognition[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C.,USA:IEEE Press,2017:583-588.
[8]	XIA Rui,LIU Yang.A multi-task learning framework for emotion recognition using 2D continuous space[J].IEEE Transactions on Affective Computing,2017,8(1):3-14.
[9]	卢官明,袁亮,杨文娟,等.基于长短期记忆和卷积神经网络的语音情感识别[J].南京邮电大学学报(自然科学版),2018,38(5):63-69.
[10]	陈晓敏.基于时序深度学习模型的语音情感识别方法研究[D].哈尔滨:哈尔滨工业大学,2018.
[11]	曾润华,张树群.改进卷积神经网络的语音情感识别方法[J].应用科学学报,2018,36(5):837-844
[12]	HAN Kun,YU Dong,TASHEV I.Speech emotion recognition using deep neural network and extreme learning machine[EB/OL].[2018-10-20].http://193.6.4.39/~czap/letoltes/IS14/IS2014/PDF/AUTHOR/IS140441.PDF. URL
[13]	LIM W,JANG D,LEE T.Speech emotion recognition using convolutional and recurrent neural networks[C]//Proceedings of Signal and Information Processing Asso-ciation Annual Summit and Conference.Washington D.C.,USA:IEEE Press,2016:1-4.
[14]	FAYEK H M,LECH M,CAVEDON L.Evaluating deep learning architectures for speech emotion recognition[J].Neural Networks,2017,92(8):60-68.
[15]	汪洪桥,孙富春,蔡艳宁,等.多核学习方法[J].自动化学报,2010,36(8):1037-1050.
[16]	RAKOTOMAMONJY A,BACH F R,CANU S,et al.SimpleMKL[J].Journal of Machine Learning Research,2008,9(11):2491-2521.
[17]	ONG C S,SMOLA A J,WILLIAMSON R C.Learning the kernel with hyperkernels[J].Journal of Machine Learning Research,2005,6(7):1043-1071.
[18]	AIOLLI F,DONINI M.EasyMKL:a scalable multiple kernel learning algorithm[J].Neurocomputing,2015,169:215-224.
[19]	PORIA S,CHATURVEDI I,CAMBRIA E,et al.Convolutional MKL based multimodal emotion recognition and sentiment analysis[C]//Proceedings of the 16th International Conference on Data Mining.Washington D.C.,USA:IEEE Press,2016:439-448.
[20]	JIN Yun,SONG Peng,ZHENG Wenming,et al.Novel feature fusion method for speech emotion recognition based on multiple kernel learning[J].Journal of Southeast University,2013,29(2):129-133.
[21]	JIN Yun,SONG Peng,ZHENG Wenming,et al.Speaker-independent speech emotion recognition based on two-layer multiple kernel learning[J].IEICE Transactions on Information and Systems,2013,96(10):2286-2289.
[22]	XU Xinzhou,DENG Jun,CUMMINS N,et al.A two-dimensional framework of multiple kernel subspace learning for recognizing emotion in speech[J].IEEE/ACM Transactions on Audio,Speech and Language Processing,2017,25(7):1436-1449.
[23]	SUN Yaxin,WEN Guihua,WANG Jiabing.Weighted spectral features based on local hu moments for speech emotion recognition[J].Biomedical Signal Processing and Control,2015,18(4):80-90.
[24]	MAO Qirong,DONG Ming,HUANG Zhengwei,et al.Learning salient features for speech emotion recognition using convolutional neural networks[J].IEEE Tran-sactions on Multimedia,2014,16(8):2203-2213.
[25]	BURKHARDT F,PAESCHKE A,ROLFES M,et al.A database of German emotional speech[C]//Proceedings of the 9th European Conference on Speech Communication and Technology.[S.l.]:Curran Associates Inc.,2005:1-4.
[26]	KUCHIBHOTLA S,VANKAYALAPATI H D,ANNE K R.An optimal two stage feature selection for speech emotion recognition using acoustic features[J].Inter-national Journal of Speech Technology,2016,19(4):657-667.
[27]	ZHU Bing,ZHOU Wenkai,WANG Yutian,et al.End-to-end speech emotion recognition based on neural network[C]//Proceedings of the 17th International Con-ference on Communication Technology.Washington D.C.,USA:IEEE Press,2017:1634-1638.

选择文件类型/文献管理软件名称

选择包含的内容

基于多核学习特征融合的语音情感识别方法

Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	曹坪, 杨怀志, 薄一军, 尤嘉, 张淳杰, 李丹勇. 面向低质量裂缝图像的多知识蒸馏分类[J]. 计算机工程, 2023, 49(7): 204-213.
[2]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[3]	代祖华, 刘园园, 狄世龙. 语义增强的图神经网络方面级文本情感分析[J]. 计算机工程, 2023, 49(6): 71-80.
[4]	沈学利, 田桂源, 姜彦吉, 马琳琳. 基于双阶段Conv-Transformer的时频域语音增强算法[J]. 计算机工程, 2023, 49(6): 123-130.
[5]	宋羽凯, 谢江. 基于多任务学习的轻量级语音情感识别模型[J]. 计算机工程, 2023, 49(5): 122-128.
[6]	丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建[J]. 计算机工程, 2023, 49(5): 231-238.
[7]	陈治旭, 靳雁霞, 芦烨, 杨晶, 刘亚变, 史志儒. 基于子图卷积神经网络的多精度服装建模方法[J]. 计算机工程, 2023, 49(4): 174-181.
[8]	徐康, 李霏, 姬东鸿. 结合依存图卷积与文本片段搜索的方面情感三元组抽取[J]. 计算机工程, 2023, 49(4): 61-67.
[9]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[10]	钟宝荣, 吴夏灵. 基于高分辨率网络的轻量型人体姿态估计研究[J]. 计算机工程, 2023, 49(4): 226-232,239.
[11]	杨晶晶, 谢海燕, 薛妮妮, 张傲明. 基于双通道残差网络的水下图像去噪研究[J]. 计算机工程, 2023, 49(4): 188-198.
[12]	耿磊, 傅洪亮, 陶华伟, 卢远, 郭歆莹, 赵力. 基于动态卷积递归神经网络的语音情感识别[J]. 计算机工程, 2023, 49(4): 125-130,137.
[13]	刘晶晶, 黄浩. 引入非局部模块卷积神经网络的基频提取模型[J]. 计算机工程, 2023, 49(3): 128-133,160.
[14]	邹长龙, 安敬民, 李冠宇. 基于邻域聚合与CNN的知识图谱实体类型补全[J]. 计算机工程, 2023, 49(3): 134-141.
[15]	翟社平, 张宇航, 柏晓夏. 融合实体邻域信息的知识图谱嵌入负采样方法[J]. 计算机工程, 2023, 49(3): 95-104.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于多核学习特征融合的语音情感识别方法

Speech Emotion Recognition Method Based on Multiple Kernel Learning Feature Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价