融合笔画特征的胶囊网络文本分类

doi:10.19678/j.issn.1000-3428.0060560

摘要/Abstract

摘要： 目前多数文本分类方法无法有效反映句子中不同单词的重要程度，且在神经网络训练过程中获得的词向量忽略了汉字本身的结构信息。构建一种GRU-ATT-Capsule混合模型，并结合CW2Vec模型训练中文词向量。对文本数据进行预处理，使用传统的词向量方法训练的词向量作为模型的第1种输入，通过CW2Vec模型训练得到的包含汉字笔画特征的中文词向量作为第2种输入，完成文本表示。利用门控循环单元分别提取2种不同输入的上下文特征并结合注意力机制学习文本中单词的重要性，将2种不同输入提取出的上下文特征进行融合，通过胶囊网络学习文本局部与全局之间的关系特征实现文本分类。在搜狗新闻数据集上的实验结果表明，GRU-ATT-Capsule混合模型相比TextCNN、BiGRU-ATT模型在测试集分类准确率上分别提高2.35和4.70个百分点，融合笔画特征的双通道输入混合模型相比单通道输入混合模型在测试集分类准确率上提高0.45个百分点，证明了GRU-ATT-Capsule混合模型能有效提取包括汉字结构在内的更多文本特征，提升文本分类效果。

关键词: 词向量, 笔画特征, 门控循环单元, 注意力机制, 胶囊网络, 文本分类

Abstract: Most current text classification methods have the problem that they cannot effectively reflect the importance of different words in a sentence, and the word vectors obtained in the neural network training process ignore the structural information of the Chinese characters.A GRU-ATT-Capsule hybrid model is proposed and combined with the CW2Vec model to train Chinese word vectors.First, the text data are preprocessed, and the word vector trained by the traditional word vector method is used as the first input of the model.The Chinese word vectors containing the stroke features of Chinese characters obtained by the CW2Vec model training are used as the second input to represent the text. Second, the contextual features of two different inputs are extracted through the Gated Recurrent Unit(GRU), and the attention mechanism is used to learn the importance of words in the text.The contextual features extracted from the two different inputs are fused, allowing the capsule network to learn the characteristics of the relationship between the local and global features of the text for text classification.The experimental results on the Sogou news dataset show that the GRU-ATT-Capsule hybrid model improves the classification accuracy of the test set by 2.35 and 4.70 percentage points, respectively, compared with the TextCNN and BiGRU-ATT models, and the dual-channel input hybrid model fused with stroke features compared with the single-channel input mixture model, the classification accuracy of the test set is improved by 0.45 percentage points, which proves that the GRU-ATT-Capsule hybrid model can effectively extract more text features, including Chinese character structure, and improve the text classification effect.

Key words: word vector, stroke feature, Gated Recurrent Unit(GRU), attention mechanism, capsule network, text classification

中图分类号:

TP391

李冉冉, 刘大明, 刘正, 常高祥. 融合笔画特征的胶囊网络文本分类[J]. 计算机工程, 2022, 48(3): 69-73,80.

LI Ranran, LIU Daming, LIU Zheng, CHANG Gaoxiang. Text Classification Using Capsule Network Integrating Stroke Features[J]. Computer Engineering, 2022, 48(3): 69-73,80.

http://www.ecice06.com/CN/Y2022/V48/I3/69

图/表 5

20220330200638

20220330200641

20220330200644

20220330200648

20220330200651

参考文献

[1] 叶雪梅, 毛雪岷, 夏锦春, 等.文本分类TF-IDF算法的改进研究[J].计算机工程与应用, 2019, 55(2):104-109, 161. YE X M, MAO X M, XIA J C, et al.Improved approach to TF-IDF algorithm in text classification[J].Computer Engineering and Applications, 2019, 55(2):104-109, 161.(in Chinese)
[2] WANG Z Q, SUN X, ZHANG D X, et al.An optimal SVM-based text classification algorithm[C]//Proceedings of 2006 International Conference on Machine Learning and Cybernetics.Washington D.C., USA:IEEE Press, 2006:1378-1381.
[3] ZHANG L G, JIANG L X, LI C Q, et al.Two feature weighting approaches for naive Bayes text classifiers[J].Knowledge-Based Systems, 2016, 100:137-144.
[4] OTTER D W, MEDINA J R, KALITA J K.A survey of the usages of deep learning for natural language processing[J].IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(2):604-624.
[5] MIKOLOV T, CHEN K, CORRADO G, et al.Efficient estimation of word representations in vector space[EB/OL].[2020-12-15].https://arxiv.org/pdf/1301.3781.pdf.
[6] SUN Y M, LIN L, YANG N, et al.Radical-enhanced Chinese character embedding[C]//Proceedings of 2014 International Conference on Neural Information Processing.Berlin, Germany:Springer, 2014:279-286.
[7] 武娇, 洪彩凤, 顾永春, 等.基于类邻域字典的线性回归文本分类[J].计算机工程, 2021, 47(8):93-99, 108. WU J, HONG C F, GU Y C, et al.Linear regression text classification based on class-wise nearest neighbor dictionary[J].Computer Engineering, 2021, 47(8):93-99, 108.(in Chinese)
[8] CAO S, LU W, ZHOU J, et al.CW2Vec:learning Chinese word embeddings with stroke n-gram information[EB/OL].[2020-12-15].https://ojs.aaai.org/index.php/AAAI/article/view/12029.
[9] KIM Y.Convolutional neural networks for sentence classification[EB/OL].[2020-12-15].https://arxiv.org/abs/1408.5882.
[10] TIAN Z X, RONG W G, SHI L B, et al.Attention aware bidirectional gated recurrent unit based framework for sentiment analysis[C]//Proceedings of International Conference on Knowledge Science, Engineering and Management.Berlin, Germany:Springer, 2018:67-68.
[11] DU C S, HUANG L.Text classification research with attention-based recurrent neural networks[J].International Journal of Computers Communications & Control, 2018, 13(1):50-60.
[12] KIM J, JANG S, PARK E, et al.Text classification using capsules[J].Neurocomputing, 2020, 376:214-221.
[13] HARRIS Z S.Distributional structure[M].Berlin, Germany:Springer, 1970.
[14] PENNINGTON J, SOCHER R, MANNING C.Glove:global vectors for word representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2014:1532-1534.
[15] 赵浩新, 俞敬松, 林杰.基于笔画中文字向量模型设计与研究[J].中文信息学报, 2019, 33(5):17-23. ZHAO H X, YU J S, LIN J.Design and research on Chinese word embedding model based on strokes[J].Journal of Chinese Information Processing, 2019, 33(5):17-23.(in Chinese)
[16] CHEN X, XU L, LIU Z, et al.Joint learning of character and word embeddings[C]//Proceedings of International Joint Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2015:1236-1242.
[17] SU T R, LEE H Y.Learning Chinese word representations from glyphs of characters[EB/OL].[2020-12-15].https://arxiv.org/abs/1708.04755.
[18] HINTON G E, KRIZHEVSKY A, WANG S D.Transforming auto-encoders[C]//Proceedings of International Conference on Artificial Neural Networks.Berlin, Germany:Springer, 2011:44-51.
[19] SABOUR S, FROSST N, HINTON G E.Dynamic routing between capsules[EB/OL].[2020-12-15].https://arxiv.org/abs/1710.09829.
[20] 王丽亚, 刘昌辉, 蔡敦波, 等.CNN-BiGRU网络中引入注意力机制的中文文本情感分析[J].计算机应用, 2019, 39(10):2841-2846. WANG L Y, LIU C H, CAI D B, et al.Chinese text sentiment analysis based on CNN-BiGRU network with attention mechanism[J].Journal of Computer Applications, 2019, 39(10):2841-2846.(in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容