Text Sentiment Analysis Model Based on Parallel Hybrid Network and Attention Mechanism

doi:10.19678/j.issn.1000-3428.0061995

Abstract

Abstract: Existing text sentiment analysis models based on deep learning and neural networks typically have problems, such as incomplete text feature extraction.In addition, they do not consider the impact of key information on text sentiment tendencies.Based on the parallel hybrid network and dual-way attention mechanism, an improved text sentiment analysis model is proposed.GloVe and Word2vec are used to vectorize text based on the characteristics of different networks to obtain richer text information.The Bidirectional Gated Recurrent Unit(BiGRU) and Convolutional Neural Network(CNN) are used to construct a parallel hybrid network to improve the feature extraction capacity of the model, and the two-word vectors are input into the parallel hybrid network to extract global and local features.The dual-way attention mechanism is used to strengthen the key information in the global and local features and perform feature fusion to improve the capacity of the model to recognize key information.The fused entire-text features are input to the fully connected layer for final sentiment polarity classification.The experimental results for the IMDb and SST-2 public datasets show that the classification accuracy of the model reaches 91.73% and 91.16%, respectively.This study proves that the dual-way attention mechanism can comprehensively capture the key information in the text and improve the text sentiment classification effect.

Key words: nature language processing, text sentiment analysis, Bidirectional Gated Recurrent Unit(BiGRU), Convolutional Neural Network(CNN), dual-way attention mechanism, feature fusion

摘要： 现有基于深度学习和神经网络的文本情感分析模型通常存在文本特征提取不全面，且未考虑关键信息对文本情感倾向的影响等问题。基于并行混合网络与双路注意力机制，提出一种改进的文本情感分析模型。根据不同神经网络的特点分别采用GloVe和Word2vec两种词向量训练工具将文本向量化，得到更丰富的文本信息。将两种不同的词向量并行输入由双向门控循环单元与卷积神经网络构建的并行混合网络，同时提取上下文全局特征与局部特征，提高模型的特征提取能力。使用双路注意力机制分别对全局特征和局部特征中的关键信息进行加强处理及特征融合，增强模型识别关键信息的能力。将融合后的整个文本特征输入全连接层，实现最终的情感极性分类。在IMDb和SST-2公开数据集上的实验结果表明，该模型的分类准确率分别达到91.73%和91.16%，相比于同类文本情感分析模型有不同程度的提升，从而证明了双路注意力机制可以更全面地捕获文本中的关键信息，提高文本情感分类效果。

关键词: 自然语言处理, 文本情感分析, 双向门控循环单元, 卷积神经网络, 双路注意力机制, 特征融合

CLC Number:

TP391

TIAN Qiaoxin, KONG Weiwei, TENG Jinbao, WANG Zhaoqian. Text Sentiment Analysis Model Based on Parallel Hybrid Network and Attention Mechanism[J]. Computer Engineering, 2022, 48(8): 266-273.

田乔鑫, 孔韦韦, 滕金保, 王照乾. 基于并行混合网络与注意力机制的文本情感分析模型[J]. 计算机工程, 2022, 48(8): 266-273.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0061995

http://www.ecice06.com/EN/Y2022/V48/I8/266

Figures/Tables 12

References

[1] PANG B, LEE L.Opinion mining and sentiment analysis[J].Foundations and Trends in Information Retrieval, 2008, 2(1/2):1-35.
[2] ZHANG S X, WEI Z L, WANG Y, et al.Sentiment analysis of Chinese micro-blog text based on extended sentiment dictionary[J].Future Generation Computer Systems, 2018, 81:395-403.
[3] HEMMATIAN F, SOHRABI M K.A survey on classification techniques for opinion mining and sentiment analysis[J].Artificial Intelligence Review, 2019, 52(3):1495-1545.
[4] HINTON G.Learning distributed representations of concepts[C]//Proceedings of the 8th Annual Conference of the Cognitive Science Society.London, UK:Psychology Press, 1986:1-12.
[5] MIKOLOV T, SUTSKEVER I, CHEN K, et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2013:3111-3119.
[6] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2014:1746-1751.
[7] XIAO Z, LIANG P J.Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proceedings of 2016 International Conference on Cloud Computing and Security.Berlin, Germany:Springer, 2016:601-610.
[8] LAI S, XU L, LIU K, et al.Recurrent convolutional neural networks for text classification[C]//Proceedings of the 29th Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2015:2267-2273.
[9] ABID F, ALAM M, YASIR M, et al.Sentiment analysis through recurrent variants latterly on convolutional neural network of Twitter[J].Future Generation Computer Systems, 2019, 95:292-308.
[10] BAHDANAU D, CHO K, BENGIO Y.Neural machine translation by jointly learning to align and translate[EB/OL].[2021-06-28].https://arxiv.org/abs/1409.0473v7.
[11] YANG Z C, YANG D Y, DYER C, et al.Hierarchical attention networks for document classification[C]//Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2016:1480-1489.
[12] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need?[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Long Beach, USA:NIPS, 2017:6000-6010.
[13] 赵亚南, 刘渊, 宋设.融合多头自注意力机制的金融新闻极性分析[J].计算机工程, 2020, 46(8):85-92. ZHAO Y N, LIU Y, SONG S.Financial news polarity analysis fusing with multi-head self-attention mechanism[J].Computer Engineering, 2020, 46(8):85-92.(in Chinese)
[14] 袁和金, 张旭, 牛为华, 等.融合注意力机制的多通道卷积与双向GRU模型的文本情感分析研究[J].中文信息学报, 2019, 33(10):109-118. YUAN H J, ZHANG X, NIU W H, et al.Sentiment analysis based on multi-channel convolution and Bi-directional GRU with attention mechanism[J].Journal of Chinese Information Processing, 2019, 33(10):109-118.(in Chinese)
[15] 郭宝震, 左万利, 王英.采用词向量注意力机制的双路卷积神经网络句子分类模型[J].浙江大学学报(工学版), 2018, 52(9):1729-1737. GUO B Z, ZUO W L, WANG Y.Double CNN sentence classification model with attention mechanism of word embeddings[J].Journal of Zhejiang University(Engineering Science), 2018, 52(9):1729-1737.(in Chinese)
[16] WANG K, SHEN W Z, YANG Y Y, et al.Relational graph attention network for aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2020:3229-3238.
[17] PENNINGTON J, SOCHER R, MANNING C.GloVe:global vectors for word representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2014:1532-1543.
[18] HOCHREITER S, SCHMIDHUBER J.Long short-term memory[J].Neural Computation, 1997, 9(8):1735-1780.
[19] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2014:1724-1734.
[20] MAAS A L, DALY R E, PHAM P T, et al.Learning word vectors for sentiment analysis[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2011:142-150.
[21] 关鹏飞, 李宝安, 吕学强, 等.注意力增强的双向LSTM情感分析[J].中文信息学报, 2019, 33(2):105-111. GUAN P F, LI B A, LÜX Q, et al.Attention enhanced Bi-directional LSTM for sentiment analysis[J].Journal of Chinese Information Processing, 2019, 33(2):105-111.(in Chinese)
[22] 徐菲菲, 芦霄鹏.结合卷积神经网络和最小门控单元注意力的文本情感分析[J].计算机应用与软件, 2020, 37(9):75-80, 125. XU F F, LU X P.Text sentiment analysis based on attention C_MGU[J].Computer Applications and Software, 2020, 37(9):75-80, 125.(in Chinese)
[23] 陈虎, 高波涌, 陈莲娜, 等.结合注意力机制与双向切片GRU的情感分类模型[J].小型微型计算机系统, 2020, 41(9):1793-1799. CHEN H, GAO B Y, CHEN L N, et al.Sentiment classification model combining attention mechanism and bidirectional slice GRU[J].Journal of Chinese Computer Systems, 2020, 41(9):1793-1799.(in Chinese)

Please choose a citation manager

Content to export