Few-Shot Relation Extraction Method Fusing with Hybrid Representation

doi:10.19678/j.issn.1000-3428.0065258

Abstract

Abstract:

Addressing the problem in which the traditional prototypical network cannot effectively solve the prototype deviation caused by noisy data, this study proposes a few-shot relation extraction method fusing with hybrid representation.This method introduces relation representation and entity mention as auxiliary information, and the prototype points of the relation categories are dynamically constructed to improve the processing ability of the model for noisy data. Therefore, a more accurate prototype discriminant representation is obtained. First, the BERT pre-trained language model extracts textual features.Simultaneously, entity mentions are used to extract entity relation representations, establishing local prototypes. A relation attention module is proposed to perform semantic computation and key feature selection on relation representations and support instances.Second, a dynamic prototypical network building module is proposed based on hybrid representation, where the local prototype and the relation attention module are used to dynamically construct entity representation prototype points and relation representation prototype points, respectively. Finally, the two prototype points are fused to form a more discriminative hybrid representation prototype point, which further enhances the relation representation ability of the prototype.The experimental results show that on the public dataset FewRel 1.0, the few-shot relation extraction method fusing with hybrid representation achieves up to 96.26% higher accuracy than the baseline model under different subtask settings.These results validate the efficacy of the proposed method in utilizing auxiliary information to alleviate the problem of prototype deviation and obtaining a better relationship extraction effect.

Key words: relation extraction, few-shot learning, prototypical network, attention mechanism, hybrid representation

摘要：

针对传统原型网络无法有效解决噪声数据带来的类原型偏差的问题，提出一种融合混合表征的小样本关系抽取方法，通过引入关系表述和实体提及作为辅助信息，并通过动态构建关系类别原型表示来提升模型对噪声数据的处理能力，进而获得更精确的原型判别性表示。首先，使用BERT预训练语言模型对文本进行特征提取，利用实体提及提取实体关系表示，构建局部原型，并设计关系注意力模块对关系表述和支持实例进行语义计算和关键特征选择；然后，提出基于混合表征的动态原型网络构建模块，分别利用局部原型和关系注意力模块动态构建实体表征类原型和关系表征类原型；最后，融合两者形成更具判别性的混合表征原型点，进一步增强原型的关系表示能力。实验结果表明，在公开数据集FewRel 1.0上，融合混合表征的小样本关系抽取方法相较于基线模型，在不同子任务设置下均取得了较高的准确率，最高可达96.26%，验证了所提方法能有效利用辅助信息缓解类原型偏差问题，获得较好的关系抽取效果。

关键词: 关系抽取, 小样本学习, 原型网络, 注意力机制, 混合表征

Haoxin LIU, Chao DONG, Zhinan GOU, Kai GAO. Few-Shot Relation Extraction Method Fusing with Hybrid Representation[J]. Computer Engineering, 2023, 49(8): 63-68.

刘昊鑫, 董超, 勾智楠, 高凯. 融合混合表征的小样本关系抽取方法[J]. 计算机工程, 2023, 49(8): 63-68.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0065258

http://www.ecice06.com/EN/Y2023/V49/I8/63

Figures/Tables 5

References 25

1	WANG H L, QIN K, ZAKARI R Y, et al. Deep neural network-based relation extraction: an overview. Neural Computing and Applications, 2022, 34 (6): 4781- 4801. doi: 10.1007/s00521-021-06667-3
2	LIU C Y, SUN W B, CHAO W H, et al. Convolution neural network for relation extraction[C]//Proceedings of International Conference on Advanced Data Mining and Applications. Berlin, Germany: Springer, 2013: 231-242.
3	ZHANG D X, WANG D. Relation classification via recurrent neural network[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1508.01006.
4	NAYAK T, NG H T. Effective modeling of encoder-decoder architecture for joint entity and relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34 (5): 8528- 8535. doi: 10.1609/aaai.v34i05.6374
5	XUE F Z, SUN A X, ZHANG H, et al. GDPNet: refining latent multi-view graph for relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35 (16): 14194- 14202. doi: 10.1609/aaai.v35i16.17670
6	YE D M, LIN Y K, LI P, et al. Packed levitated marker for entity and relation extraction[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg, USA: Association for Computational Linguistics, 2022: 4904-4917.
7	胡晗, 刘鹏远. 小样本关系分类研究综述. 中文信息学报, 2022, 36 (2): 1- 11. URL
	HU H, LIU P Y. Few-shot relation classification: a survey. Journal of Chinese Information Processing, 2022, 36 (2): 1- 11. URL
8	MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. New York, USA: ACM Press, 2009: 1003-1011.
9	BHARTIYA A, BADOLA K, MAUSAM. DiS-ReX: a multilingual dataset for distantly supervised relation extraction[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2104.08655.
10	VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1606.04080.
11	SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 4080-4090.
12	MUNKHDALAI T, YU H. Meta networks[C]//Proceedings of International Conference on Machine Learning. New York, USA: PMLR, 2017: 2554-2563.
13	HAN X, ZHU H, YU P F, et al. FewRel: a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1810.10147.
14	XIE Y X, XU H, LI J E, et al. Heterogeneous graph neural networks for noisy few-shot relation classification. Knowledge-Based Systems, 2020, 194, 105548. doi: 10.1016/j.knosys.2020.105548
15	QU M, GAO T, XHONNEUX L P, et al. Few-shot relation extraction via Bayesian meta-learning on relation graphs[C]//Proceedings of International Conference on Machine Learning. New York, USA: PMLR, 2020: 7867-7876.
16	GAO T Y, HAN X, ZHU H, et al. FewRel 2.0: towards more challenging few-shot relation classification[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1910.07124.
17	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional Transformers for language understanding[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1810.04805.
18	SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: distributional similarity for relation learning[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1906.03158.
19	PAHDE F, PUSCAS M, KLEIN T, et al. Multimodal prototypical networks for few-shot learning[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2021: 2643-2652.
20	PARNAMI A, LEE M. Few-shot keyword spotting with prototypical networks[C]//Proceedings of the 7th International Conference on Machine Learning Technologies. New York, USA: ACM Press, 2022: 277-283.
21	GAO T Y, HAN X, LIU Z Y, et al. Hybrid attention-based prototypical networks for noisy few-shot relation classification. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33 (1): 6407- 6414. doi: 10.1609/aaai.v33i01.33016407
22	YANG K J, ZHENG N T, DAI X Y, et al. Enhance prototypical network with text descriptions for few-shot relation classification[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York, USA: ACM Press, 2020: 2273-2276.
23	YANG S, ZHANG Y F, NIU G L, et al. Entity concept-enhanced few-shot relation extraction[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2106.02401.
24	WANG Y Y, BAO J W, LIU G Y, et al. Learning to decouple relations: few-shot relation classification with entity-guided attention and confusion-aware training[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2010.10894.
25	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.

key	value
关系ID	P102
关系名称	member of political party
关系表述	party of which this politician is or has been a member
实例	["In", "1992", ", ", "her", "Republican", "opponent", "was", "real", "estate", "broker", "Zach", "Wamp", "."]
头实体	["zach wamp", "Q139389", (10, 11)]
尾实体	["republican", "Q29468", (4)]

key	value
关系ID	P102
关系名称	member of political party
关系表述	party of which this politician is or has been a member
实例	["In", "1992", ", ", "her", "Republican", "opponent", "was", "real", "estate", "broker", "Zach", "Wamp", "."]
头实体	["zach wamp", "Q139389", (10, 11)]
尾实体	["republican", "Q29468", (4)]

参数名称	参数值
Max-Length	128
Hidden-Size	768
Weight-Decay	1×10^-2
Learning-Rate	2×10^-5

参数名称	参数值
Max-Length	128
Hidden-Size	768
Weight-Decay	1×10^-2
Learning-Rate	2×10^-5

模型	准确率
模型	5-way 1-shot	5-way 5-shot	10-way 1-shot	10-way 5-shot
Proto-CNN	69.20	84.79	56.44	75.55
HATT-Proto-CNN	—	90.12	—	83.05
PAIR-BERT	88.32	93.22	80.63	87.02
TD-Proto-BERT	84.76	92.38	74.32	85.92
CTEG-BERT	88.11	95.25	81.29	91.33
HR-DProto	90.16	96.26	85.24	94.18

Please choose a citation manager

Content to export