基于跨度和特征融合的实体关系联合抽取模型

doi:10.19678/j.issn.1000-3428.0065261

摘要/Abstract

摘要： 实体关系联合抽取模型在实体关系抽取中具有重要作用，针对现有的实体关系联合抽取模型无法有效识别重叠关系中的实体关系三元组问题，提出一种新型的基于跨度和特征融合的实体关系联合抽取模型SFFM。将文本输入BERT预训练模型转变为词向量，根据跨度进行词向量划分形成跨度序列，并基于卷积神经网络过滤跨度序列中不包含实体的跨度序列，使用双向长短时记忆提取剩余跨度序列融合文本信息后的特征并通过Softmax回归实现实体识别，将文本中的实体和关系映射到不同的跨度序列中，当重叠关系中的实体和距离较远的实体之间存在关系时，按照跨度进行划分使可能存在关系的实体对划分到同一个跨度序列中，以更好地利用文本中的重叠关系。在此基础上，通过注意力机制获取跨度序列中的依赖关系，运用Softmax回归对跨度序列中的关系进行分类。实验结果表明，与基线模型相比，该模型在CoNLL04数据集上的微平均和宏平均分别提升了1.87和1.73个百分点，在SciERC数据集上的微平均提升了5.95个百分点。

关键词: 联合抽取, 实体关系抽取, 神经网络, 跨度, 特征融合

Abstract: The entity-relationship joint extraction model plays an important role in entity-relationship extraction； however，the existing entity-relationship joint extraction model cannot effectively identify entity-relation triples in overlapping relationships.This paper proposes a novel entity-relationship extraction model SFFM based on span and feature fusion. The model first converts the text input to the BERT pre-training model into word vectors. Then，it divides the word vectors based on the span to form a span sequence，filters the span sequences that do not contain entities based on Convolutional Neural Network（CNN），and uses Bi-directional Long Short-Term Memory（Bi-LSTM） to extract the features of the remaining span sequences.It uses Softmax regression to perform entity recognition. The span sequence formed by the division can map the entities and relationships in the text to different span sequences.When there is a relationship between an entity in an overlapping relationship and an entity with a long distance，the division is performed according to the span so that the entities that may have a relationship are paired with each other. Dividing into the same span sequence can effectively utilize the overlapping relationship proposed in this paper.Finally，the attention mechanism is used to obtain the dependencies in the span sequence，and Softmax regression is used to classify the relationships in the span sequence.The experimental results show that compared with the baseline model，the micro-average and macro-average of the CoNLL04 dataset increase by 1.87 and 1.73 percentage points， respectively，and the micro-average increases by 5.95 percentage points in the SciERC dataset.

Key words: joint extraction, entity-relation extraction, neural network, span, feature fusion

中图分类号:

TP18

廖涛, 孙皓洁, 张顺香. 基于跨度和特征融合的实体关系联合抽取模型[J]. 计算机工程, 2023, 49(6): 107-114.

LIAO Tao, SUN Haojie, ZHANG Shunxiang. Entity-Relation Joint Extraction Model Based on Span and Feature Fusion[J]. Computer Engineering, 2023, 49(6): 107-114.

https://www.ecice06.com/CN/Y2023/V49/I6/107

图/表 14

20230615164820

20230615164825

20230615164828

20230615164832

20230615164837

20230615164840

20230615164844

20230615164904

20230615164907

20230615164911

20230615164915

20230615164918

20230615164921

20230615164925

参考文献

[1] ZHONG Z,CHEN D.A frustratingly easy approach for entity and relation extraction[EB/OL].[2022-06-10].https://arxiv.org/abs/2010.12812.
[2] ZHANG Q Q,CHEN M D,LIU L Z.A review on entity relation extraction[C]//Proceedings of the 2nd International Conference on Mechanical,Control and Computer Engineering.Washington D.C.,USA:IEEE Press,2018:178-183.
[3] 鄂海红,张文静,肖思琪,等.深度学习实体关系抽取研究综述[J].软件学报,2019,30(6):1793-1818.E H H,ZHANG W J,XIAO S Q,et al.Survey of entity relationship extraction based on deep learning[J].Journal of Software,2019,30(6):1793-1818.(in Chinese)
[4] GAO C,ZHANG X,LIU H,et al.A joint extraction model of entities and relations based on relation decomposition[J].International Journal of Machine Learning and Cybernetics,2022,13(7):1833-1845.
[5] ZENG X,ZENG D,HE S,et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Melbourne,Australia:ACL,2018:506-514.
[6] NAYAK T,NG H T.Effective attention modeling for neural relation extraction[EB/OL].[2022-06-10].https://arxiv.org/abs/1912.03832.
[7] ZENG D J,LIU K,LAI S W,et al.Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics.Washington D.C.,USA:IEEE Press,2014:2335-2344.
[8] GUO X Y,ZHANG H,YANG H J,et al.A single attention-based combination of CNN and RNN for relation classification[J].IEEE Access,2019,7:12467-12475.
[9] GUO Z,ZHANG Y,LU W.Attention guided graph convolutional networks for relation extraction[EB/OL].[2022-06-10].https://arxiv.org/abs/1906.07510.
[10] MIWA M,BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[EB/OL].[2022-06-10].https://arxiv.org/abs/1601.00770.
[11] CHEN D R,CAI J J,LI C Z.A joint entity-relation extraction method with sparse parameter sharing architecture[C]//Proceedings of the 7th International Conference on Computer and Communications.Washington D.C.,USA:IEEE Press,2022:1705-1709.
[12] KATIYAR A,CARDIE C.Going out on a limb:joint extraction of entity mentions and relations without dependency trees[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Washington D.C.,USA:IEEE Press,2017:917-928.
[13] WANG Y,SUN C,WU Y,et al.UniRE:a unified label space for entity relation extraction[EB/OL].[2022-06-10].https://arxiv.org/abs/2107.04292.
[14] 禹克强,黄芳,吴琪,等.基于双向语义的中文实体关系联合抽取方法[J].计算机工程,2023,49(1):92-99,112.YU K Q,HUANG F,WU Q,et al.Joint extraction method for Chinese entity relationship based on bidirectional semantics[J].Computer Engineering,2023,49(1):92-99,112.(in Chinese)
[15] CHEN Z,GUO C.A pattern-first pipeline approach for entity and relation extraction[J].Neurocomputing,2022,494:182-191.
[16] ZHENG S,WANG F,BAO H,et al.Joint extraction of entities and relations based on a novel tagging scheme[EB/OL].[2022-06-10].https://arxiv.org/abs/1706.05075.
[17] HANG T,FENG J,WU Y.Joint extraction of entities and overlapping relations using source-target entity labeling[J].Expert Systems with Applications,2021,177:114853.
[18] EBERTS M,ULGES A.Span-based joint entity and relation extraction with transformer pre-training[EB/OL].[2022-06-10].https://arxiv.org/abs/1909.07755.
[19] 陈仁杰,郑小盈,祝永新.融合实体类别信息的实体关系联合抽取[J].计算机工程,2022,48(3):46-53.CHEN R J,ZHENG X Y,ZHU Y X.Joint entity and relation extraction fusing entity type information[J].Computer Engineering,2022,48(3):46-53.(in Chinese)
[20] ROTH D,YIH W.A linear programming for mulation for global inference in natural language tasks[D].Champaign,USA:University of Illinois at Urbana-Champaign,2004.
[21] LUAN Y,HE L,OSTENDORF M,et al.Multi-task identification of entities,relations,and coreference for scientific knowledge graph construction[EB/OL].[2022-06-10].https://arxiv.org/abs/1808.09602.
[22] LUAN Y,WADDEN D,HE L,et al.A general framework for information extraction using dynamic span graphs[EB/OL].[2022-06-10].https://arxiv.org/abs/1904.03296.
[23] BEKOULIS G,DELEU J,DEMEESTER T,et al.Joint entity recognition and relation extraction as a multi-head selection problem[J].Expert Systems with Applications,2018,114:34-45.
[24] FU T J,LI P H,MA W Y.Graphrel:modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Washington D.C.,USA:IEEE Press,2019:1409-1418.

选择文件类型/文献管理软件名称

选择包含的内容