融合实体类别信息的实体关系联合抽取

doi:10.19678/j.issn.1000-3428.0060745

摘要/Abstract

摘要： 针对实体关系抽取任务中的三元组重叠问题，基于编码器-解码器结构的联合抽取方法能够通过序列生成的方式加以解决。但现有方法没有充分利用实体类别信息，而实体类别信息对于构建更丰富的语义特征并进一步优化关系模型的效果具有重要意义。在使用编码器-解码器结构的基础上，融合实体类别信息构建实体关系联合抽取模型FETI。编码器采用经典Bi-LSTM结构，解码器采用树状解码替代传统的一维线性解码。同时，在解码阶段增加头尾实体类别的预测，并通过辅助损失函数进行约束，使模型能够更有效地利用实体类别信息。在百度公开的中文数据集DuIE上进行实验，结果表明，FETI的F1值达到0.758，相对于CopyMTL、WDec、MHS、Seq2UMTree模型提升了2.02%~9.86%，验证了融合实体类别信息对于提升实体关系抽取模型性能的有效性。此外，基于不同解码顺序和不同权重损失函数的实验结果表明，解码顺序对模型性能影响较大，而对主要任务的损失函数赋予较高权重，能够保证辅助任务为主要任务提供有效的背景知识，同时限制噪声的影响。

关键词: 实体关系抽取, 联合抽取, 实体类别信息, 三元组重叠, 编码器, 解码器

Abstract: For the problem of triplet overlapping in entity and relation extraction, a joint extraction method based on an encoder-decoder structure can be used to solve it using sequence generation.However, the existing methods do not fully use the entity type information, which is of great significance for building richer semantic features and for further improving the effect of the relationship model.Based on an encoder-decoder structure, the joint entity and relation extraction model FETI is constructed by fusing entity type information.The encoder adopts the classical Bidirectional Long-Short Term Memory(Bi-LSTM) structure, and the decoder adopts tree decoding to replace the traditional one-dimensional linear decoding.At the same time, the prediction of the head and tail entity types is added in the decoding stage and is constrained by the auxiliary loss function, so that the model can more effectively use the entity type information.The experiment is conducted on the Chinese dataset DuIE published by Baidu.The experimental results show that the F1 value of FETI reached 0.758, which is 2.02%~9.86% higher than those of the CopyMTL, WDec, MHS and Seq2UMTree models.The results confirm the effectiveness of integrating the entity category information to improve the performance of the entity and relation extraction model.In addition, the experimental results based on different decoding sequences and different weight loss functions show that, the decoding sequence has a great impact on the performance of the model, and that giving higher weight to the loss function of the main task, which ensures that the auxiliary task can provide effective background knowledge for the main task and limit the impact of noise.

Key words: entity and relation extraction, joint extraction, entity type information, triplet overlapping, encoder, decoder

中图分类号:

TP391

陈仁杰, 郑小盈, 祝永新. 融合实体类别信息的实体关系联合抽取[J]. 计算机工程, 2022, 48(3): 46-53.

CHEN Renjie, ZHENG Xiaoying, ZHU Yongxin. Joint Entity and Relation Extraction Fusing Entity Type Information[J]. Computer Engineering, 2022, 48(3): 46-53.

http://www.ecice06.com/CN/Y2022/V48/I3/46

图/表 6

20220330200513

20220330200516

20220330200520

20220330200523

20220330200526

20220330200530

参考文献

[1] NADEAU D, SEKINE S.A survey of named entity recognition and classification[J].Lingvisticae Investigationes, 2007, 30(1):3-26.
[2] CHAN Y S, ROTH D.Exploiting syntactico-semantic structures for relation extraction[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg, USA:Association for Computational Linguistics, 2011:551-560.
[3] ROTH D, YIH W.Global inference for entity and relation identification via a linear programming formulation[EB/OL].[2021-01-02].http://scottyih.org/files/RothYih07.pdf.
[4] ADEL H, SCHÜTZE H.Global normalization of convolutional neural networks for joint entity and relation classification[EB/OL].(2017-06-24)[2021-01-02].https://arxiv.org/pdf/1707.07719.pdf.
[5] ZHENG S C, WANG F, BAO H Y, et al.Joint extraction of entities and relations based on a novel tagging scheme[EB/OL].(2017-06-07)[2021-01-02].https://arxiv.org/pdf/1706.05075v1.pdf.
[6] DAI D, XIAO X Y, LYU Y J, et al.Joint extraction of entities and overlapping relations using position-attentive sequence labeling[C]//Proceedings of AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2019:6300-6308.
[7] ZENG X R, ZENG D J, HE S Z, et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).Stroundsburg, USA:Association for Computational Linguistics, 2018:1-10.
[8] ZENG D J, ZHANG H R, LIU Q Y.CopyMTL:copy mechanism for joint extraction of entities and relations with multi-task learning[C]//Proceedings of AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2020:9507-9514.
[9] ZHANG H R, LIU Q Y, FAN X M, et al.Minimize exposure Bias of Seq2Seq models in joint entity and relation extraction[EB/OL].(2020-09-16)[2021-01-02].https://arxiv.org/pdf/2009.07503.pdf.
[10] PENG H, GAO T Y, HAN X, et al.Learning from context or names?An empirical study on neural relation extraction[EB/OL].[2021-01-02].https://www.aclweb.org/anthology/2020.emnlp-main.298.pdf.
[11] ZHONG Z X, CHEN D Q.A frustratingly easy approach for entity and relation extraction[EB/OL].(2020-10-24)[2021-01-02].https://arxiv.org/pdf/2010.12812v1.pdf.
[12] 孙新, 申长虹, 姜景虎, 等.结合实体描述信息的跨句包关系抽取方法[J].计算机工程, 2021, 47(6):68-75. SUN X, SHEN C H, JIANG J H, et al.Cross-sentence bagrelation extraction method combining entity description information[J].Computer Engineering, 2021, 47(6):68-75.(in Chinese)
[13] LI S J, HE W, SHI Y B, et al.DuIE:a large-scale Chinese dataset for information extraction[C]//Proceedings of CCF International Conference on Natural Language Processing and Chinese Computing.Beijing, China:CCF, 2019:791-800.
[14] LI Q, JI H.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).Stroundsburg, USA:Association for Computational Linguistics, 2014:402-412.
[15] MIWA M, SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing.Berlin, Germany:Springer, 2014:1-10.
[16] TAKANOBU R, ZHANG T Y, LIU J X, et al.A hierarchical framework for relation extraction with reinforcement learning[C]//Proceedings of AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2018:1-9.
[17] 孙紫阳, 顾君忠, 杨静.基于深度学习的中文实体关系抽取方法[J].计算机工程, 2018, 44(9):164-170. SUN Z Y, GU J Z, YANG J.Chinese entity relation extraction method based on deep learning[J].Computer Engineering, 2018, 44(9):164-170.(in Chinese)
[18] FU T J, LI P H, MA W Y.GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroundsburg, USA:Association for Computational Linguistics, 2019:1-10.
[19] LI X Y, YIN F, SUN Z J, et al.Entity-relation extraction as multi-turn question answering[EB/OL].(2019-05-14)[2021-01-02].https://arxiv.org/pdf/1905.05529v1.pdf.
[20] NAYAK T, NG H T.Effective modeling of encoder-decoder architecture for joint entity and relation extraction[C]//Proceedings of AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2020:8528-8535.
[21] RANZATO M, CHOPRA S, AULI M, et al.Sequence level training with recurrent neural networks[EB/OL].(2015-11-20)[2021-01-02].http://de.arxiv.org/pdf/1511.06732.
[22] CARUANA R.Multitask learning[J].Machine Learning, 1997, 28(1):41-75.
[23] VANDENHENDE S, GEORGOULIS S, GANSBEKE W, et al.Multi-task learning for dense prediction tasks:a survey[J/OL].IEEE Transactions on Pattern Analysis and Machine Intelligence:1-10[2021-01-26] https://ieeexplore.ieee.org/document/9336293.
[24] CIPOLLA R, GAL Y, KENDALL A.Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7482-7491.
[25] CHEN Z, BADRINARAYANAN V, LEE C Y, et al.GradNorm:gradient normalization for adaptive loss balancing in deep multitask networks[C]//Proceedings of International Conference on Machine Learning.Berlin, Germany:Springer, 2017:1-10.
[26] SUTSKEVER I, VINYALS O, LE Q V.Sequence to sequence learning with neural networks[EB/OL].[2021-01-02].https://proceedings.neurips.cc/paper/2014/file/a14ac55a4f27472c5d894ec1c3c743d2-Paper.pdf.
[27] GRAVES A, SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J].Neural Networks, 2005, 18(5/6):602-610.
[28] HOCHREITER S, SCHMIDHUBER J.Long short-term memory[J].Neural Computation, 1997, 9(8):1735-1780.
[29] LUONG T, PHAM H, MANNING C D.Effective approaches to attention-based neural machine translation[EB/OL].(2015-08-17)[2021-01-02].https://arxiv.org/pdf/1508.04025v1.pdf.
[30] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al.Dropout:a simple way to prevent neural networks from overfitting[J].The Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[31] KINGMA D P, BA J.Adam:a method for stochastic optimization[EB/OL].(2014-12-22)[2021-01-02].https://arxiv.org/pdf/1412.6980.pdf.
[32] BEKOULIS G, DELEU J, DEMEESTER T, et al.Joint entity recognition and relation extraction as a multi-head selection problem[J].Expert Systems with Applications, 2018, 114:34-45.

选择文件类型/文献管理软件名称

选择包含的内容