基于标签分解的口语理解模型

doi:10.19678/j.issn.1000-3428.0051291

计算机工程 ›› 2019, Vol. 45 ›› Issue (7): 237-241. doi: 10.19678/j.issn.1000-3428.0051291

基于标签分解的口语理解模型

许莹莹, 黄浩

新疆大学信息科学与工程学院, 乌鲁木齐 830046

收稿日期:2018-04-23 修回日期:2018-05-24 出版日期:2019-07-15 发布日期:2019-07-23
作者简介:许莹莹(1990-),女,硕士研究生,主研方向为自然语言处理、语音通信;黄浩,教授、博士。
基金资助:
国家自然科学基金（61663044，61365005）。

Spoken Language Understanding Model Based on Label Decomposition

XU Yingying, HUANG Hao

College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China

Received:2018-04-23 Revised:2018-05-24 Online:2019-07-15 Published:2019-07-23

摘要/Abstract

摘要： 在双向长短时记忆网络的基础上，提出一种用于口语理解的标签拆分策略，并构建一个联合模型。通过将1次127种标签分类转换成3次独立的分类，平衡ATIS数据集的标签。针对ATIS数据集资源较少的问题，引入外部词向量以提升模型的分类性能。实验结果表明，与循环神经网络及其变体结构相比，该模型的F1值有显著提升，最高可达95.63%。

关键词: 口语理解, 槽填充, 双向长短时记忆网络, 词向量, 联合模型

Abstract: Based on the Bi-Long Short Term Memory (BiLSTM),this paper proposes a label splitting strategy for Spoken Language Understanding(SLU)and constructs a joint model.The model convert a classification of 127 labels into 3 independent classifications to balance the labels in the ATIS database.Due to the scarcity of ATIS data,this paper introduces external word embedding to improve the classification performance of the model.Experimental results show that compared with the traditional recurrent neural network and its variants,the proposed joint model obtains significantly improvement in F1 value,which can be up to 95.63%.

Key words: Spoken Language Understanding(SLU), slot filling, Bi-Long Short Term Memory(BiLSTM), word embedding, hibrid model

中图分类号:

TP391

许莹莹, 黄浩. 基于标签分解的口语理解模型[J]. 计算机工程, 2019, 45(7): 237-241.

XU Yingying, HUANG Hao. Spoken Language Understanding Model Based on Label Decomposition[J]. Computer Engineering, 2019, 45(7): 237-241.

http://www.ecice06.com/CN/Y2019/V45/I7/237

参考文献

[1] MORI R D,FREDERIC B,HAKKANI-TUR D,et al.Spoken language understanding[J].IEEE Signal Processing Magazine,2008,25(3):50-58.
[2] HAFFNER P,TUR G,WRIGHT J H.Optimizing SVMs for complex call classification[C]//Proceedings of IEEE International Conference on Acoustics,Speech,and Signal Processing.Washington D.C.,USA:IEEE Press,2003:632-635.
[3] SARIKAYA R,HINTON G E,RAMABHADRAN B.Deep belief nets for natural language call-routing[C]//Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing.Washington D.C.,USA:IEEE Press,2011:5680-5683.
[4] MCCALLUM A,FREITAG D,PEREIRA F.Maximum entropy Markov models for information extraction and segmentation[C]//Proceedings of the 17th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers Inc.,2000:591-598.
[5] RAYMOND C,RICCARDI G.Generative and discriminative algorithms for spoken language understanding[C]//Proceedings of International Speech Communication Association.Antwerp,Belgium:[s.n.],2007:1605-1608.
[6] YAO Kaisheng,PENG Baolin,ZHANG Yu,et al.Spoken language understanding using long short-term memory neural networks[C]//Proceedings of Spoken Language Technology Workshop.Washington D.C.,USA:IEEE Press,2015:189-194.
[7] MESNIL G,DAUPHIN Y,YAO Kaisheng,et al.Using recurrent neural networks for slot filling in spoken language understanding[J].IEEE/ACM Transactions on Audio Speech and Language Processing,2015,23(3):530-539.
[8] LIU Bing,LANE I.Recurrent neural network structured output prediction for spoken language understanding[C]//Proceedings of NIPS Workshop on Machine Learning for Spoken Language Understanding and Interactions.Montreal,Canada:[s.n.],2015:1-9.
[9] GUO D,TUR G,YIH W T,et al.Joint semantic utterance classification and slot filling with recursive neural networks[C]//Proceedings of Spoken Language Technology Workshop.Washington D.C.,USA:IEEE Press,2015:554-559.
[10] XU Puyang,SARIKAYA R.Convolutional neural network based triangular CRF for joint intent detection and slot filling[C]//Proceedings of Automatic Speech Recognition and Understanding.Washington D.C.,USA:IEEE Press,2014:78-83.
[11] MINKER W,BENNACEF S,GAUVAIN J.A stochastic case frame approach for natural language understanding[C]//Proceedings of International Conference on Spoken Language.Washington D.C.,USA:IEEE Press,1996:1013-1016.
[12] RAYMOND C,RICCARDI G.Generative and discriminative algorithms for spoken language understanding[C]//Proceedings of International Speech Communication Association.Antwerp,Belgium:[s.n.],2007:1605-1608.
[13] LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning.San Francisco,USA:Morgan Kaufmann Publishers Inc.,2001:282-289.
[14] MESNIL G,HE Xiaodong,DENG Li,et al.Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding[C]//Proceedings of International Speech Communication Association.Lyon,France:[s.n.],2013:3771-3775.
[15] KOLBOEK M,TAN Zhenghua,JENSEN J.Speech enhancement using long short-term memory based recurrent neural networks for noise robust speaker verification[C]//Proceedings of IEEE Spoken Language Technology Workshop.Washington D.C.,USA:IEEE Press,2016:305-311.
[16] BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2006,3(6):1137-1155.
[17] 吴旭康,杨旭光,陈园园,等.主题联合词向量模型[J].计算机工程,2018,44(2):233-237.
[18] 余冲,李晶,孙旭东,等.基于词嵌入与概率主题模型的社会媒体话题识别[J].计算机工程,2017,43(12):184-191.
[19] HEMPHILL C T,GODFREY J J,DODDINGTON G R.The ATIS spoken language systems pilot corpus[C]//Proceedings of the Darpa Speech and Natural Language Workshop.Hidden Valley,USA:[s.n.],1990:96-101.
[20] TUR G,HAKKANI-TUR D,HECK L.What is left to be understood in ATIS?[C]//Proceedings of Spoken Language Technology Workshop.Washington D.C.,USA:IEEE Press,2011:19-24.
[21] RAMSHAW L A,MARCUS M P.Text chunking using transformation-based learning[J].Text Speech and Language Technology,1995,11:82-94.
[22] ELMAN J.Finding structure in time[J].Cognitive Science,1990,14(2):179-211.
[23] PENG Baolin,YAO Kaisheng,JING Li,et al.Recurrent neural networks with external memory for spoken language understanding[C]//Proceeding of Natural Language Processing and Chinese Computing.Berlin,Germany:Springer,2015:25-35.
[24] JORDAN M I.Serial order:A parallel distributed processing approach[J].Advances in Psychology,1997,121:471-495.

选择文件类型/文献管理软件名称

选择包含的内容

基于标签分解的口语理解模型

Spoken Language Understanding Model Based on Label Decomposition

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	石磊, 张吉涛, 高宇飞, 卫琳, 陶永才. 基于Transformer与BiLSTM的网络流量入侵检测[J]. 计算机工程, 2023, 49(3): 29-36,57.
[2]	刘高军, 王岳, 段建勇, 何丽, 王昊. 利用知识强化语言模型的口语理解方法[J]. 计算机工程, 2023, 49(3): 73-79.
[3]	李军怀, 陈苗苗, 王怀军, 崔颖安, 张爱华. 基于ALBERT-BGRU-CRF的中文命名实体识别方法[J]. 计算机工程, 2022, 48(6): 89-94,106.
[4]	李冉冉, 刘大明, 刘正, 常高祥. 融合笔画特征的胶囊网络文本分类[J]. 计算机工程, 2022, 48(3): 69-73,80.
[5]	廖涛, 黄荣梅, 张顺香, 段松松. 基于交互式特征融合的嵌套命名实体识别[J]. 计算机工程, 2022, 48(12): 119-126,133.
[6]	雷恒林, 古兰拜尔·吐尔洪, 买日旦·吾守尔, 曾琪. 基于Hellinger距离与词向量的终身机器学习主题模型[J]. 计算机工程, 2022, 48(11): 89-95.
[7]	江旭, 钱雪忠, 宋威. 结合残差BiLSTM与句袋注意力的远程监督关系抽取[J]. 计算机工程, 2022, 48(10): 110-115,122.
[8]	彭俊利, 谷雨, 张震, 耿小航. 融合单词贡献度与Word2Vec词向量的文档表示[J]. 计算机工程, 2021, 47(4): 62-67.
[9]	袁志祥, 任冬冬, 洪旭东, 孙国华. 结合数据库结构及内容的问句理解方法研究[J]. 计算机工程, 2021, 47(3): 71-76,82.
[10]	贾旭楠, 魏庭新, 曲维光, 顾彦慧, 周俊生. 基于神经网络的复句判定及其关系识别研究[J]. 计算机工程, 2021, 47(11): 54-61.
[11]	张冬梅, 陈永乐, 杨玉丽. 基于分层特征的代码克隆检测方法[J]. 计算机工程, 2021, 47(10): 125-131.
[12]	李俊, 吕学强. 融合BERT语义加权与网络图的关键词抽取方法[J]. 计算机工程, 2020, 46(9): 89-94.
[13]	陈俊月, 郝文宁, 张紫萱, 唐新德, 康睿智, 莫斐. 基于改进句子相似度算法的释义识别研究[J]. 计算机工程, 2020, 46(9): 76-82.
[14]	王义, 沈洋, 戴月明. 基于细粒度多通道卷积神经网络的文本情感分析[J]. 计算机工程, 2020, 46(5): 102-108.
[15]	王任华, 沈剑宇, 蒋敏. 基于自适应多模型联合的目标跟踪算法[J]. 计算机工程, 2019, 45(8): 266-274.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于标签分解的口语理解模型

Spoken Language Understanding Model Based on Label Decomposition

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价