基于神经网络的文本分类方法研究

doi:10.19678/j.issn.1000-3428.0053748

摘要/Abstract

摘要： 海量文本分析是实现大数据理解和价值发现的重要手段，其中文本分类作为自然语言处理的经典问题受到研究者广泛关注，而人工神经网络在文本分析方面的优异表现使其成为目前的主要研究方向。在此背景下，介绍卷积神经网络、时间递归神经网络、结构递归神经网络和预训练模型等主流方法在文本分类中应用的发展历程，比较不同模型基于常用数据集的分类效果，表明利用人工神经网络结构自动获取文本特征，可避免繁杂的人工特征工程，使文本分类效果得到提升。在此基础上，对未来文本分类的研究方向进行展望。

关键词: 大数据, 自然语言处理, 文本分类, 神经网络, 文本分析

Abstract: Large-scale text analysis is an important means of understanding and finding value of big data.Hence text classification,as a classical natural language processing problem,has been widely concerned by researchers,and its main research direction is artificial neural network due to its excellent performance in text analysis.This paper introduces the history of Convolutional Neural Network(CNN),Recurrent Neural Network(RNN),recursive neural network structure and the pretraining model applied to text classification.Then this paper compares classification performance of different models based on the common dataset,demonstrating that artificial neural network structure can reduce manual characterization work by automatically obtaining text features,and thus improve text classification effects.On this basis,this paper prospects the future research directions of text classification.

Key words: big data, natural language processing, text classification, neural network, text analysis

中图分类号:

TP391

王芝辉, 王晓东. 基于神经网络的文本分类方法研究[J]. 计算机工程, 2020, 46(3): 11-17.

WANG Zhihui, WANG Xiaodong. Research on Text Classification Methods Based on Neural Network[J]. Computer Engineering, 2020, 46(3): 11-17.

https://www.ecice06.com/CN/Y2020/V46/I3/11

图/表 3

参考文献

[1] ZHANG Chuan.Research and implementation of personalized microblog recommendation algorithm based on content and user behavior[D].Beijing:Beijing University of Posts and Telecommunications,2018.(in Chinese)张川.基于内容和用户行为的个性化微博推荐算法研究与实现[D].北京:北京邮电大学,2018.
[2] SU Jinshu,ZHANG Bofeng,XU Xin.Advances in machine learning based text categorization[J].Journal of Software,2006,17(9):1848-1859.(in Chinese)苏金树,张博锋,徐昕.基于机器学习的文本分类技术研究进展[J].软件学报,2006,17(9):1848-1859.
[3] DOU Xiaofan.Review of KNN algorithm[J].Telecom World,2018(10):273-274.(in Chinese)窦小凡.KNN算法综述[J].通讯世界,2018(10):273-274.
[4] ZHANG Zhaohui,LU Yuchang,ZHANG Bo.Discovering classification rules by using the neural networks[J].Chinese Journal of Computers,1999,22(1):108-112.(in Chinese)张朝晖,陆玉昌,张钹.利用神经网络发现分类规则[J].计算机学报,1999,22(1):108-112.
[5] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Doha,Qatar:[s.n.],2014:1746-1751.
[6] MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of NIPS'13.Lake Tahoe,USA:[s.n.],2013:3111-3119.
[7] KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics.[S.l.]:ACL,2014:655-665.
[8] YIN W,SCHÜTZE H.Multichannel variable-size convolution for sentence classification[C]//Proceedings of the 19th Conference on Computational Natural Language Learning.Beijing,China:[s.n.],2015:204-214.
[9] PENNINGTON J,SOCHER R,MANNING C D.Glove:global vectors for word representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Doha,Qatar:[s.n.],2014:1532-1543.
[10] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost) from Scratch[J].Journal of Machine Learning Research,2011,12:2493-2537.
[11] HANG X,ZHAO J,LeCUN Y.Character-level convolutional networks for text classification[C]//Proceedings of International Conference on Neural Information Processing Systems.Montréal,Canada:MIT Press,2015:1-9.
[12] TAI K S,SOCHER R,MANNING C D.Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics.[S.l.]:ACL,2015:1556-1566.
[13] DING Zixiang,XIA Rui,YU Jianfei,et al.Densely connected bidirectional LSTM with applications to sentence classification[EB/OL].[2018-12-02].https://arxiv.org/pdf/1802.00889.pdf.
[14] LAI Siwei,XU Liheng,LIU Kang,et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence.Austin,USA:AAAI,2015:2267-2273.
[15] ZHOU Chunting,SUN Chonglin,LIU Zhiyuan,et al.A C-LSTM neural network for text classification[J].Computer Science,2015,1(4):39-44.
[16] ZHOU Peng,QI Zhenyu,ZHENG Suncong,et al.Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling[C]//Proceedings of COLING'16.Osaka,Japan:[s.n.],2016:3485-3495.
[17] JOHNSON R,ZHANG T.Supervised and semi-supervised text categorization using LSTM for region embeddings[C]//Proceedings of International Conference on Machine Learning.New York,USA:[s.n.],2016:526-534.
[18] ZHANG R,LEE H,RADEV D.Dependency sensitive convolutional neural networks for modeling sentences and documents[C]//Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.[S.l.]:ACL,2016:1512-1521.
[19] SOCHER R,PERELYGIN A,WU J Y,et al.Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of 2013 Conference on Empirical Methods in Natural Language Processing.Seattle,USA:[s.n.],2013:1631-1642.
[20] CARDIE C.Deep recursive neural networks for composi-tionality in language[C]//Proceedings of International Conference on Neural Information Processing Systems.Montréal,Canada:MIT Press,2014:2096-2104.
[21] PETERS M E,NEUMANN M,IYYER M,et al.Deep contextualized word representations[C]//Proceedings of North American Chapter of the Association for Computational Linguistics.[S.l.]:ACL,2018:1-9.
[22] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of Annual Conference on Neural Information Processing Systems.Long Beach,USA:[s.n.],2017:1-5.
[23] DEVLIN J,CHANG M W,LEE K.BERT:pre-training of deep bidirectional transformers for language under-standing[EB/OL].[2018-12-02].https://arxiv.org/pdf/1810.04805.pdf.
[24] JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of tricks for efficient text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics.[S.l.]:ACL,2017:1-5.
[25] YANG Z,YANG D,DYER C,et al.Hierarchical attention networks for document classification[C]//Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.[S.l.]:ACL,2016:1480-1489.
[26] IYYER M,MANJUNATHA V,BOYD-GRABER J,et al.Deep unordered composition rivals syntactic methods for text classification[C]//Proceedings of Meeting of the Association for Computational Linguistics.[S.l.]:ACL,2015:1681-1691.
[27] PANG B,LEE L.Seeing stars:exploiting class relation-ships for sentiment categorization with respect to rating scales[C]//Proceedings of Meeting of Association for Computational Linguistics.[S.l.]:ACL,2005:115-124.
[28] SOCHER R,PERELYGIN A,WU J,et al.Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of 2013 Conference on Empirical Methods in Natural Language Processing.Seattle,USA:[s.n.],2013:1631-1642.
[29] PANG B,LEE L.A sentimental education:sentiment analysis using subjectivity summarization based on minimum cuts[C]//Proceedings of the Meeting of Association for Computational Linguistics.[S.l.]:ACL,2004:271-278.
[30] HINGMIRE S,CHOUGULE S,PALSHIKAR G.K,et al.Document classification by topic labeling[C]//Proceedings of Special Interest Group on Information Retrieval.Dublin,Ireland:[s.n.],2013:877-880.
[31] WANG Canhui,ZHANG Min,MA Shaoping,et al.Auto-matic online news issue construction in Web environ-ment[C]//Proceedings of the 17th International Conference on World Wide Web.Beijing,China:[s.n.],2008:457-466.
[32] LI X,ROTH D.Learning question classifiers[C]//Proceedings of the 1st International Joint Conference on Natural Language Processing.Sanya,China:[s.n.],2004:556-562.
[33] MCAULEY J,LESKOVEC J.Hidden factors and hidden topics:understanding rating dimensions with review text[C]//Proceedings of the 7th ACM Conference on Recommender Systems.New York,USA:ACM Press,2013:165-172.
[34] CAI Huiping.Research on short text classification based on convolutional neural network[D].Chongqing:Southwest University,2016.(in Chinese)蔡慧苹.基于卷积神经网络的短文本分类方法研究[D].重庆:西南大学,2016.
[35] HUANG Lei,DU Changshun.Research on text classification based on recursive neural network[J].Journal of Beijing University of Chemical Technology(Natural Science),2017,44(1):98-104.(in Chinese)黄磊,杜昌顺.基于递归神经网络的文本分类研究[J].北京化工大学学报(自然科学版),2017,44(1):98-104.

选择文件类型/文献管理软件名称

选择包含的内容