Bidirectional Cascade Network for Aspect Sentiment Triplet Extraction

doi:10.19678/j.issn.1000-3428.0068534

Abstract

Abstract:

Aspect Sentiment Triplet Extraction(ASTE) is a subtask of aspect-based sentiment analysis. The goal is to extract all mentioned aspects, their corresponding opinions, and the sentiment polarity expressed, forming triplets from a given text sequence. To achieve this, a bidirectional cascade network is proposed, leveraging span-level interactions and end-to-end extraction. Conditional layer normalization is applied within the encoding blocks to facilitate deep-level interactions between the aspect and opinion spans, creating a cascading effect. Triplets are extracted in both ″aspect-to-opinion″ and ″opinion-to-aspect″ directions, with decoding strategies that merge results from both directions. To address class imbalance, a sparse factor is introduced into the multilabel cross-entropy loss, increasing the penalty for sparse positive classes during training. To mitigate exposure bias, span drifting and output sampling used to construct negative training samples. Experimental results on the ASTE-Data-V2-EMNLP2020 dataset show improvements in F1 scores over the SBN-ASTE model across four sub-datasets: 14LAP, 14RES, 15RES, and 16RES, with increases of 0.91, 0.17, 2.0, and 1.56 percentage points, respectively.

Key words: deep-level interaction, bidirectional cascade, bidirectional aggregation, exposure bias, class imbalance

摘要：

方面情感三元组抽取(ASTE)是基于方面的情感分析子任务之一, 其目标是从给定文本序列中抽取所有提及的方面及其对应的观点、观点表达的情感倾向, 构成三元组。基于跨度级交互和端到端两个前提, 提出双向级联网络。在编码块里应用条件层归一化让方面跨度和观点跨度的特征进行深层交互, 实现两者间的级联; 在“方面到观点”和“观点到方面”两个方向上进行三元组抽取, 并设计解码策略聚合两个方向上的结果; 为缓解类别不平衡问题, 在多标签交叉熵损失中加入稀疏因子来提高训练环节对稀疏正类的惩罚力度; 为缓解曝光偏差问题, 设计跨度漂移和输出采样两种策略来构造负样本加入训练。在ASTE-Data-V2-EMNLP2020数据集上进行实验, 结果表明, 所提模型在4个子数据集14LAP、14RES、15RES、16RES上较SBN-ASTE模型的F1值分别提高0.91、0.17、2.0、1.56个百分点。

关键词: 深层交互, 双向级联, 双向聚合, 曝光偏差, 类别不平衡

ZHENG Ayong, GU Xingsheng. Bidirectional Cascade Network for Aspect Sentiment Triplet Extraction[J]. Computer Engineering, 2024, 50(12): 90-98.

郑阿勇, 顾幸生. 用于方面情感三元组抽取的双向级联网络[J]. 计算机工程, 2024, 50(12): 90-98.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068534

https://www.ecice06.com/EN/Y2024/V50/I12/90

Figures/Tables 8

Fig.1 Model structure

Fig.2 Span score matrices

Fig.3 Negative samples

Fig.4 Bidirectional aggregation

Fig.5 Task prompt template

References 30

1	ZHANG W X , LI X , DENG Y , et al. A survey on aspect-based sentiment analysis: tasks, methods, and challenges. IEEE Transactions on Knowledge and Data Engineering, 2023, 35 (11): 11019- 11038. doi: 10.1109/TKDE.2022.3230975
2	谭翠萍. 文本细粒度情感分析研究综述. 大学图书馆学报, 2022, 40 (4): 85-99, 119.
	TAN C . Review of fine-grained sentiment analysis based on text. Journal of Academic Libraries, 2022, 40 (4): 85-99, 119.
3	PENG H Y , XU L , BING L D , et al. Knowing what, how and why: a near complete solution for aspect-based sentiment analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34 (5): 8600- 8607. doi: 10.1609/aaai.v34i05.6383
4	MAO Y, SHEN Y, YU C, et al. A joint training dual-MRC framework for aspect based sentiment analysis[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence. [S. l. ]: AAAI Press, 2021: 13543-13551.
5	XU L, LI H, LU W, et al. Position-aware tagging for aspect sentiment triplet extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP). Stroudsburg, USA: Association for Computational Linguistics, 2020: 2339-2349.
6	WU Z, YING C C, ZHAO F, et al. Grid tagging scheme for aspect-oriented fine-grained opinion extraction[C]//Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg, USA: Association for Computational Linguistics, 2020: 2576-2585.
7	徐康, 李霏, 姬东鸿. 结合依存图卷积与文本片段搜索的方面情感三元组抽取. 计算机工程, 2023, 49 (4): 61- 67. URL
	XU K , LI F , JI D H . Aspect sentiment triple extraction by combining dependency graph convolution and text span search. Computer Engineering, 2023, 49 (4): 61- 67. URL
8	XU L, CHIA Y K, BING L D. Learning span-level interactions for aspect sentiment triplet extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 1: Long Papers). Stroudsburg, USA: Association for Computational Linguistics, 2021: 4755-4766.
9	CHEN Y Q, CHEN K M, SUN X, et al. A span-level bidirectional network for aspect sentiment triplet extraction[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2022: 4300-4309.
10	WANG Y, YU B, ZHANG Y, et al. TPLinker: single-stage joint extraction of entities and relations through token pair linking[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2010.13415v1.
11	ZHANG R H, LIU Q, FAN A X, et al. Minimize exposure bias of seq2seq models in joint entity and relation extraction[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2009.07503.
12	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2019: 4171-4186.
13	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. [2023-08-10]. https://arxiv.org/pdf/1706.03762.
14	SU J, MURTADHA A, PAN S, et al. Global pointer: novel efficient span-based approach for named entity recognition[EB/OL]. [2023-08-10]. https://arxiv.org/pdf/2208.03054.
15	SU J, LU Y, PAN S, et al. RoFormer: enhanced transformer with rotary position embedding[EB/OL]. [2023-08-10]. https://arxiv.org/pdf/2104.09864.
16	LEE K, HE L H, LEWIS M, et al. End-to-end neural conference resolution[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: association for Computational Linguistics, 2017: 1-10.
17	XU J, SUN X, ZHANG Z, et al. Understanding and improving layer normalization[EB/OL]. [2023-08-10]. https://arxiv.org/abs/1911.07013.
18	ZHANG Z H, LI X Q, LI Y X, et al. Neural noise embedding for end-to-end speech enhancement with conditional layer normalization[C]//Proceedings of 2021 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). Washington D. C., USA: IEEE Press, 2021: 7113-7117.
19	XIONG R, YANG Y, HE D, et al. On layer normalization in the transformer architecture[C]//Proceedings of International Conference on Machine Learning. [S.l.]: PMLR, 2020: 10524-10533.
20	LEE D, TIAN Z, XUE L, et al. Enhancing content preservation in text style transfer using reverse attention and conditional layer normalization[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2108.00449.
21	SCHMIDT F. Generalization in generation: a closer look at exposure bias[EB/OL]. [2023-08-10]. https://arxiv.org/abs/1910.00292.
22	DR A U R . Binary cross entropy with deep learning technique for Image classification. International Journal of Advanced Trends in Computer Science and Engineering, 2020, 9 (4): 5393- 5397. doi: 10.30534/ijatcse/2020/175942020
23	JOHNSON J M , KHOSHGOFTAAR T M . Survey on deep learning with class imbalance. Journal of Big Data, 2019, 6 (1): 27. doi: 10.1186/s40537-019-0192-5
24	WANG L , HAN M , LI X J , et al. Review of classification methods on unbalanced data sets. IEEE Access, 2021, 9, 64606- 64628. doi: 10.1109/ACCESS.2021.3074243
25	SU J, ZHU M, MURTADHA A, et al. ZLPR: a novel loss for multi-label classification[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2208.02955.
26	CHEN S W , WANG Y , LIU J , et al. Bidirectional machine reading comprehension for aspect sentiment triplet extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35 (14): 12666- 12674. doi: 10.1609/aaai.v35i14.17500
27	TOUVRON H, MARTIN L, STONE K, et al. Llama 2: open foundation and fine-tuned chat models[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2307.09288.
28	HU E J, SHEN Y, WALLIS P, et al. Lora: Low-rank adaptation of large language models[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2106.09685.
29	STANDLEY T, ZAMIR A, CHEN D, et al. Which tasks should be learned together in multi-task learning?[C]//Proceedings of International Conference on Machine Learning. [S.l]: PMLR, 2020: 9120-9132.
30	ZHANG Y , YANG Q . A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering, 2022, 34 (12): 5586- 5609. doi: 10.1109/TKDE.2021.3070203

[1]	CHEN Yanfei, LIU Sanmin. Online Learning Method for Class Imbalanced and Feature Evolvable Streams [J]. Computer Engineering, 2024, 50(9): 92-103.
[2]	Anqi CHEN, Rui CHEN, Zhufang KUANG, Huajun HUANG. Research on Imbalance Fraud Detection Based on Graph Neural Network [J]. Computer Engineering, 2023, 49(11): 150-159.
[3]	WANG Zheng, YOU Ming-yu, LIU Jia-ming, LI Guo-zheng. Cough Signal Detection Based on Partial Least Squares Method [J]. Computer Engineering, 2014, 40(6): 281-284,290.
[4]	NI Huang-Jing, WANG Wei. Comparison Research of Classifier Performance on Multi-class Imbalanced Data [J]. Computer Engineering, 2011, 37(10): 160-161.

Please choose a citation manager

Content to export