ELMo-CNN-BiGRU双通道文本情感分类模型

doi:10.19678/j.issn.1000-3428.0062047

摘要/Abstract

摘要： 文本情感分类通过对带有情感色彩的主观性文本进行分析和推理，帮助用户更好地做出判断与决策。针对传统情感分类模型难以根据上下文信息调整词向量的问题，提出一种双通道文本情感分类模型。利用ELMo和Glove预训练模型分别生成动态和静态词向量，通过堆叠嵌入2种词向量生成输入向量。采用自注意力机制处理输入向量，计算内部的词依赖关系。构建融合卷积神经网络（CNN）和双向门控递归单元（BiGRU）的双通道神经网络结构，同时获取文本局部特征和全局特征。最终将双通道处理结果进行拼接，经过全连接层处理后输入分类器获得文本情感分类结果。实验结果表明，与同类情感分类模型中性能较优的H-BiGRU模型相比，ELMo-CNN-BiGRU模型在IMDB、yelp和sentiment140数据集上的准确率和F1值分别提升了2.42、1.98、2.52和2.40、1.94、2.43个百分点，具有更好的短文本情感分类效果和稳定性。

关键词: 文本情感分类, 双通道, 预训练模型, 深度学习, 自注意力机制

Abstract: Text sentiment classification helps users make better decisions by analyzing and reasoning subjective texts with emotional colors.Addressing the difficulty in adjusting the word vector according to the context information in traditional sentiment classification models, a dual-channel text sentiment classification method is proposed.To begin, pretrained ELMo and Glove models are used to generate dynamic and static word vectors, respectively.The input vector is generated by stacking and embedding two-word vectors.Second, the self-attention mechanism is used to process the input vector and calculate the internal word dependencies.The dual-channel neural network structure is constructed by a Convolutional Neural Network (CNN) and Bi-directional Gated Recurrent Unit(BiGRU).The local and global features of the text can be obtained simultaneously.Finally, the dual-channel processing results are spliced, processed by the fully connected layer, and sent to the classifier.The classification results can be obtained.The results show that, compared with the H-BiGRU model with the best performance among contrastive sentiment classification models, the accuracy of the proposed ELMo-CNN-BiGRU model on the IMDB, yelp, and sentiment140 datasets improved by 2.42, 1.98, 2.52, respectively, and the F1 value improved by 2.40, 1.94, 2.43 percentage points, respectively.It achieves a better sentiment classification effect and stability for short texts.

Key words: text sentiment classification, dual channel, pretrained model, deep learning, self-attention mechanism

中图分类号:

TP391

吴迪, 王梓宇, 赵伟超. ELMo-CNN-BiGRU双通道文本情感分类模型[J]. 计算机工程, 2022, 48(8): 105-112.

WU Di, WANG Ziyu, ZHAO Weichao. ELMo-CNN-BiGRU Dual-Channel Text Sentiment Classification Model[J]. Computer Engineering, 2022, 48(8): 105-112.

http://www.ecice06.com/CN/Y2022/V48/I8/105

图/表 15

20220825091534

20220825091541

20220825091544

20220825091548

20220825091553

20220825091556

20220825091600

20220825091604

20220825091608

20220825091613

20220825091616

20220825091620

20220825091623

20220825091627

20220825091631

参考文献

[1] 何力, 郑灶贤, 项凤涛, 等.基于深度学习的文本分类技术研究进展[J].计算机工程, 2021, 47(2):1-11. HE L, ZHENG Z X, XIANG F T, et al.Research progress of text classification technology based on deep learning[J].Computer Engineering, 2021, 47(2):1-11.(in Chinese)
[2] XU D L, TIAN Z H, LAI R F, et al.Deep learning based emotion analysis of microblog texts[J].Information Fusion, 2020, 64:1-11.
[3] MA X H.A novel hybrid model by using convolutional neural network and long short-term memory for text sentiment analysis[J].IEEE Access, 2020, 8:527-533.
[4] 周锦峰, 叶施仁, 王晖.基于深度卷积神经网络模型的文本情感分类[J].计算机工程, 2019, 45(3):300-308. ZHOU J F, YE S R, WANG H.Text sentiment classification based on deep convolutional neural network model[J].Computer Engineering, 2019, 45(3):300-308.(in Chinese)
[5] JIN N, WU J X, MA X, et al.Multi-task learning model based on multi-scale CNN and LSTM for sentiment classification[J].IEEE Access, 2020, 8:77060-77072.
[6] 蔡林森, 彭超, 陈思远, 等.基于多样化特征卷积神经网络的情感分析[J].计算机工程, 2019, 45(4):169-174, 180. CAI L S, PENG C, CHEN S Y, et al.Sentiment analysis based on multiple features convolutional neural networks[J].Computer Engineering, 2019, 45(4):169-174, 180.(in Chinese)
[7] WU D, ZHANG J P, ZHAO Q C.A text emotion analysis method using the dual-channel convolution neural network in social networks[J].Mathematical Problems in Engineering, 2020(3):1-10.
[8] LI W, ZHU L Y, SHI Y, et al.User reviews:sentiment analysis using lexicon integrated two-channel CNN-LSTM family models[J].Applied Soft Computing, 2020, 94:106435.
[9] UMER M, ASHRAF I, MEHMOOD A, et al.Sentiment analysis of tweets using a unified convolutional neural network-long short-term memory network model[J].Computational Intelligence, 2021, 37(1):409-434.
[10] 李卫疆, 漆芳.基于多通道双向长短期记忆网络的情感分析[J].中文信息学报, 2019, 33(12):119-128. LI W J, QI F.Sentiment analysis based on multi-channel bidirectional long short term memory network[J].Journal of Chinese Information Processing, 2019, 33(12):119-128.(in Chinese)
[11] 王义, 沈洋, 戴月明.基于细粒度多通道卷积神经网络的文本情感分析[J].计算机工程, 2020, 46(5):102-108. WANG Y, SHEN Y, DAI Y M.Sentiment analysis of texts based on fine-grained multi-channel convolutional neural network[J].Computer Engineering, 2020, 46(5):102-108.(in Chinese)
[12] USAMA M, AHMAD B, SONG E M, et al.Attention-based sentiment analysis using convolutional and recurrent neural network[J].Future Generation Computer Systems, 2020, 113:571-578.
[13] 陈思远, 彭超, 蔡林森, 等.一种用于特定目标情感分析的深度网络模型[J].计算机工程, 2019, 45(3):286-292. CHEN S Y, PENG C, CAI L S, et al.A deep network model for specific target sentiment analysis[J].Computer Engineering, 2019, 45(3):286-292.(in Chinese)
[14] NISTOR S C, MOCA M, MOLDOVAN D, et al.Building a twitter sentiment analysis system with recurrent neural networks[J].Sensors, 2021, 21(7):2266.
[15] 彭祝亮, 刘博文, 范程岸, 等.基于BLSTM与方面注意力模块的情感分类方法[J].计算机工程, 2020, 46(3):60-65, 72. PENG Z L, LIU B W, FAN C G, et al.Sentiment classification method based on BLSTM and aspect attention module[J].Computer Engineering, 2020, 46(3):60-65, 72.(in Chinese)
[16] 高玮军, 杨杰, 张春霞, 等.基于AT-DPCNN模型的情感分析研究[J].计算机工程, 2020, 46(11):53-60. GAO W J, YANG J, ZHANG C X, et al.Sentiment analysis research based on AT-DPCNN model[J].Computer Engineering, 2020, 46(11):53-60.(in Chinese)
[17] BASIRI M E, NEMATI S, ABDAR M, et al.ABCDM:an attention-based bidirectional CNN-RNN deep model for sentiment analysis[J].Future Generation Computer Systems, 2021, 115:279-294.
[18] LIU J, LIU P Y, ZHU Z F, et al.Graph convolutional networks with bidirectional attention for aspect-based sentiment classification[J].Applied Sciences, 2021, 11(4):1528.
[19] PETERS M E, NEUMANN M, IYYER M, et al.Deep contextualized word representations[EB/OL].[2021-06-17].https://arxiv.org/abs/1802.05365.
[20] 关鹏飞, 李宝安, 吕学强, 等.注意力增强的双向LSTM情感分析[J].中文信息学报, 2019, 33(2):105-111. GUAN P F, LI B A, LÜX Q, et al.Attention enhanced Bi-directional LSTM for sentiment analysis[J].Journal of Chinese Information Processing, 2019, 33(2):105-111.(in Chinese)
[21] 胡玉琦, 李婧, 常艳鹏, 等.引入注意力机制的BiGRU-CNN情感分类模型[J].小型微型计算机系统, 2020, 41(8):1602-1607. HU Y Q, LI J, CHANG Y P, et al.BiGRU-CNN sentiment classification model with attention mechanism[J].Journal of Chinese Computer Systems, 2020, 41(8):1602-1607.(in Chinese)
[22] 胡均毅, 李金龙.基于情感评分的分层文本表示情感分类方法[J].计算机工程, 2020, 46(3):46-52, 59. HU J Y, LI J L.Sentiment evaluation based hierarchical text representation method for sentiment classification[J].Computer Engineering, 2020, 46(3):46-52, 59.(in Chinese)
[23] 周泳东, 章韵, 曹艳蓉, 等.基于特征融合分段卷积神经网络的情感分析[J].计算机工程与设计, 2019, 40(10):3009-3013, 3029. ZHOU Y D, ZHANG Y, CAO Y R, et al.Sentiment analysis based on piecewise convolutional neural network combined with features[J].Computer Engineering and Design, 2019, 40(10):3009-3013, 3029.(in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容