一种集成深度学习模型的旅游问句文本分类算法

doi:10.19678/j.issn.1000-3428.0055990

计算机工程 ›› 2020, Vol. 46 ›› Issue (11): 70-76. doi: 10.19678/j.issn.1000-3428.0055990

一种集成深度学习模型的旅游问句文本分类算法

马喆康^a,c, 迪力亚尔·帕尔哈提^a,c, 早克热·卡德尔^b,c, 吐尔根·依布拉音^b,c, 西尔艾力·色提^b,c, 艾山·吾买尔^b,c

新疆大学 a. 软件学院;b. 信息科学与工程学院;c. 新疆多语种信息技术重点实验室, 乌鲁木齐 830046

收稿日期:2019-09-11 修回日期:2019-11-06 发布日期:2019-11-12
作者简介:马喆康(1995-),男,硕士研究生,主研方向为自然语言处理、问句文本分类;迪力亚尔·帕尔哈提,硕士研究生;早克热·卡德尔(通信作者),实验师、硕士;吐尔根·依布拉音,教授、博士;西尔艾力·色提,硕士研究生;艾山·吾买尔,副教授、博士。
基金资助:
国家自然科学基金（61762084）；国家重点研发计划（2017YFB1002103）；新疆维吾尔自治区重点实验室开放课题（2018D04019）。

A Classification Algorithm for Tourist Question Texts Integrated with Deep Learning Models

MA Zhekang^a,c, Diliyaer Paerhati^a,c, Zaokere Kadeer^b,c, Tuergen Yibulayin^b,c, Xerali Setti^b,c, Aishan Wumaier^b,c

a. College of Software;b. College of Information Science and Engineering;c. Xinjiang Key Laboratory of Multi-language Information Technology, Xinjiang University, Urumqi 830046, China

Received:2019-09-11 Revised:2019-11-06 Published:2019-11-12

摘要/Abstract

摘要： 为提高旅游问句文本中关键特征的利用率，提出一种集成词级卷积神经网络（WL-CNN）与句级双向长短期记忆（SL-Bi-LSTM）网络的旅游问句文本分类算法。利用WL-CNN和SL-Bi-LSTM分别学习词序列子空间向量和句序列深层语义信息，通过多头注意力机制将两种深度学习模型进行集成以实现旅游问句文本的语法和语义信息互补，并通过SoftMax分类器得到最终的旅游问句文本分类结果。实验结果表明，与基于传统深度学习模型的旅游问句文本分类算法相比，该算法在准确率和损失率上分别取得了0.986 6和0.127 7的最优结果，具有更好的分类效果。

关键词: 子空间结构信息, 深层语义信息, 多头注意力机制, 卷积神经网络, 双向长短期记忆网络

Abstract: To improve the utilization of key features of tourist question texts, this paper proposes a classification algorithm for tourist question texts integrated with the Word Level Convolutional Neural Network(WL-CNN) and the Sentence Level Bi-directional Long Short-Term Memory(SL-Bi-LSTM) network.The algorithm uses WL-CNN and SL-Bi-LSTM to learn the subspace vector of the word sequence and the deep semantic information of the sentence sequence.Then the two deep learning models are integrated by using the Multi-Head Attention Mechanism(MH-AM) to realize the syntactic and semantic information complementary of tourist question texts.Finally,the SoftMax classifier is used to obtain the classification results of tourist question texts.Experimental results show that the proposed algorithm has better classification performance than the tourist question text classification algorithms based on traditional deep learning models,increasing the accuracy to 0.986 6 and loss rate to 0.127 7.

Key words: subspace structure information, deep semantic information, Multi-Head Attention Mechanism(MH-AM), Convolutional Neural Network(CNN), Bi-directional Long Short-Term Memory(Bi-LSTM) network

中图分类号:

TP391

马喆康, 迪力亚尔·帕尔哈提, 早克热·卡德尔, 吐尔根·依布拉音, 西尔艾力·色提, 艾山·吾买尔. 一种集成深度学习模型的旅游问句文本分类算法[J]. 计算机工程, 2020, 46(11): 70-76.

MA Zhekang, Diliyaer Paerhati, Zaokere Kadeer, Tuergen Yibulayin, Xerali Setti, Aishan Wumaier. A Classification Algorithm for Tourist Question Texts Integrated with Deep Learning Models[J]. Computer Engineering, 2020, 46(11): 70-76.

http://www.ecice06.com/CN/Y2020/V46/I11/70

图/表 5

20201124084943

20201124084946

20201124084949

20201124084952

20201124084955

参考文献

[1] LIU Xinsheng,LI Kun.Design and implementation of tourism events text classification system based on BP-NN[J].Computer and Modernization,2011,7(7):192-194.
[2] LIU Jingzhou,CHANG Weicheng,WU Yuexin,et al.Deep learning for extreme multi-label text classification[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval.New York,USA:ACM Press,2017:115-124.
[3] XIE Yufei,LÜ Zhao.Question fine-grained classification based on semantic expansion and attention network[J].Computer Engineering,2019,45(1):171-177,183.(in Chinese)谢雨飞,吕钊.基于语义扩展与注意力网络的问题细粒度分类[J].计算机工程,2019,45(1):171-177,183.
[4] DALAL M K,ZAVERI M A.Automatic text classification of sports blog data[C]//Proceedings of 2012 Computing,Communications and Applications Conference.Washington D.C.,USA:IEEE Press,2012:61-80.
[5] SALLES T,ROCHA L,MOURÃO F,et al.A two-stage machine learning approach for temporally-robust text classification[J].Information Systems,2017,69:40-58.
[6] ELARNAOTY M,FARGHALY A.Machine learning implementations in Arabic text classification[M]//SHAALAN K,HASSANIEN A E,TOLBA F.Intelligent natural language processing:trends and applications.Berlin,Germany:Springer,2017:295-324.
[7] GOUDJIL M,KOUDIL M,BEDDA M,et al.A novel active learning method using SVM for text classification[J].International Journal of Automation and Computing,2018,15(3):290-298.
[8] HUANG Xianying,XIE Jin,LONG Shuyan.Question classification method combining word vector and BTM model[J].Computer Engineering and Design,2019,40(2):384-388.(in Chinese)黄贤英,谢晋,龙姝言.融合词向量及BTM模型的问题分类方法[J].计算机工程与设计,2019,40(2):384-388.
[9] YU Bengong,XU Qingtang,ZHANG Peihang.Question classification based on MAC-LSTM[J].Application Research of Computers,2020,37(1):40-43.(in Chinese)余本功,许庆堂,张培行.基于MAC-LSTM的问题分类研究[J].计算机应用研究,2020,37(1):40-43.
[10] LI Chenbin,ZHAN Guohua,LI Zhihua.News text classification based on improved Bi-LSTM-CNN[C]//Proceedings of the 9th International Conference on Information Technology in Medicine and Education.Washington D.C.,USA:IEEE Press,2018:890-893.
[11] ZHANG Dong,LI Shoushan,WANG Jingjing.Semi-supervised question classification with jointly learning question and answer representations[J].Journal of Chinese Information Processing,2017,31(1):1-7.(in Chinese)张栋,李寿山,王晶晶.基于问题与答案联合表示学习的半监督问题分类方法[J].中文信息学报,2017,31(1):1-7.
[12] LIU Guolong,XU Xiaofei,DENG Bailong,et al.A hybrid method for bilingual text sentiment classification based on deep learning[C]//Proceedings of the 17th IEEE/ACIS International Conference on Software Engineering,Artificial Intelligence,Networking and Parallel/Distributed Computing.Washington D.C.,USA:IEEE Press,2016:93-98.
[13] ZHARMAGAMBETOV A S,PAK A A.Sentiment analysis of a document using deep learning approach and decision trees[C]//Proceedings of the 12th International Conference on Electronics Computer and Computation.Washington D.C.,USA:IEEE Press,2016:93-98.
[14] ZHANG Qing,LÜ Zhao.Domain question classification method based on topic expansion[J].Computer Engineering,2016,42(9):202-207.(in Chinese)张青,吕钊.基于主题扩展的领域问题分类方法[J].计算机工程,2016,42(9):202-207.
[15] DUQUE A B,SANTOS L L J,MACÊDO D,et al.Squeezed very deep convolutional neural networks for text classification[C]//Proceedings of ICANN'19.Berlin,Germany:Springer,2019:193-207.
[16] WIGINGTON C,STEWART S,DAVIS B,et al.Data augmentation for recognition of handwritten words and lines using a CNN-LSTM network[C]//Proceedings of IAPR International Conference on Document Analysis & Recognition.Washington D.C.,USA:IEEE Press,2018:1-10.
[17] ADAK C,CHAUDHURI B B,BLUMENSTEIN M.Offline cursive Bengali word recognition using CNNs with a recurrent model[C]//Proceedings of the 15th International Conference on Frontiers in Handwriting Recognition.Washington D.C.,USA:IEEE Press,2016:429-434.
[18] NIU Jianwei,SUN Mingsheng,MO Shasha.Sentiment analysis of Chinese words using word embedding and sentiment morpheme matching[C]//Proceedings of International Conference on Collaborative Computing:Networking,Applications and Worksharing.Berlin,Germany:Springer,2018:3-12.
[19] ZHANG Yangsen,ZHENG Jia,JIANG Yuru,et al.A text sentiment classification modeling method based on coordinated CNN-LSTM-attention model[J].Chinese Journal of Electronics,2019,28(1):124-130.
[20] LI Lishuang,JIANG Yuxin.Biomedical named entity recognition based on the two channels and sentence-level reading control conditioned LSTM-CRF[C]//Proceedings of 2017 IEEE International Conference on Bioinformatics and Biomedicine.Washington D.C.,USA:IEEE Press,2017:380-385.
[21] ZHOU Botong,SUN Chengjie,LIN Lei,et al.LSTM based question answering for large scale knowledge base[J].Acta Scientiarum Naturalium Universitatis Pekinensis,2018,54(2):286-292.(in Chinese)周博通,孙承杰,林磊,等.基于LSTM的大规模知识库自动问答[J].北京大学学报(自然科学版),2018,54(2):286-292.
[22] WU Lin,WANG Yang,LI Xue,et al.Deep attention-based spatially recursive networks for fine-grained visual recognition[J].IEEE Transactions on Cybernetics,2019,49(5):1791-1802.
[23] ALKHOULI T,BRETSCHNER G,NEY H.On the alignment problem in multi-head attention-based neural machine translation[EB/OL].[2019-08-03].https://arxiv.org/abs/1809.03985?context=cs.
[24] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York,USA:ACM Press,2017:6000-6010.
[25] GU Chengwei,WU Ming,ZHANG Chuang.Chinese sentence classification based on convolutional neural network[J].IOP Conference Series:Materials Science and Engineering,2017,261:12-28.
[26] PRANCKEVICIUS T,MARCINKEVICIUS V.Application of logistic regression with part-of-the-speech tagging for multi-class text classification[C]//Proceedings of the 4th Workshop on Advances in Information,Electronic and Electrical Engineering.Washington D.C.,USA:IEEE Press,2016:9-18.
[27] SHIH C H,YAN B C,LIU S H,et al.Investigating Siamese LSTM networks for text categorization[C]//Proceedings of 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.Washington D.C.,USA:IEEE Press,2017:641-646.

选择文件类型/文献管理软件名称

选择包含的内容

一种集成深度学习模型的旅游问句文本分类算法

A Classification Algorithm for Tourist Question Texts Integrated with Deep Learning Models

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	曹坪, 杨怀志, 薄一军, 尤嘉, 张淳杰, 李丹勇. 面向低质量裂缝图像的多知识蒸馏分类[J]. 计算机工程, 2023, 49(7): 204-213.
[2]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[3]	代祖华, 刘园园, 狄世龙. 语义增强的图神经网络方面级文本情感分析[J]. 计算机工程, 2023, 49(6): 71-80.
[4]	沈学利, 田桂源, 姜彦吉, 马琳琳. 基于双阶段Conv-Transformer的时频域语音增强算法[J]. 计算机工程, 2023, 49(6): 123-130.
[5]	丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建[J]. 计算机工程, 2023, 49(5): 231-238.
[6]	陈治旭, 靳雁霞, 芦烨, 杨晶, 刘亚变, 史志儒. 基于子图卷积神经网络的多精度服装建模方法[J]. 计算机工程, 2023, 49(4): 174-181.
[7]	徐康, 李霏, 姬东鸿. 结合依存图卷积与文本片段搜索的方面情感三元组抽取[J]. 计算机工程, 2023, 49(4): 61-67.
[8]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[9]	钟宝荣, 吴夏灵. 基于高分辨率网络的轻量型人体姿态估计研究[J]. 计算机工程, 2023, 49(4): 226-232,239.
[10]	杨晶晶, 谢海燕, 薛妮妮, 张傲明. 基于双通道残差网络的水下图像去噪研究[J]. 计算机工程, 2023, 49(4): 188-198.
[11]	刘晶晶, 黄浩. 引入非局部模块卷积神经网络的基频提取模型[J]. 计算机工程, 2023, 49(3): 128-133,160.
[12]	邹长龙, 安敬民, 李冠宇. 基于邻域聚合与CNN的知识图谱实体类型补全[J]. 计算机工程, 2023, 49(3): 134-141.
[13]	翟社平, 张宇航, 柏晓夏. 融合实体邻域信息的知识图谱嵌入负采样方法[J]. 计算机工程, 2023, 49(3): 95-104.
[14]	程小辉, 李钰, 康燕萍. 基于中间图特征提取的卷积网络双标准剪枝[J]. 计算机工程, 2023, 49(3): 105-112.
[15]	陈柏霖, 王天极, 任丽娜, 黄瑞章. 融合ELECTRA和文本局部信息的中文语法错误检测方法[J]. 计算机工程, 2023, 49(3): 304-311.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

一种集成深度学习模型的旅游问句文本分类算法

A Classification Algorithm for Tourist Question Texts Integrated with Deep Learning Models

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献

相关文章 15

编辑推荐

Metrics

本文评价