基于Mogrifier LSTM的序列标注关系抽取方法

doi:10.19678/j.issn.1000-3428.0063198

摘要/Abstract

摘要： 对文本中的上下文信息进行充分利用能够提高关系抽取性能，但当前多数基于深度学习的关系抽取方法仅捕获文本序列中的浅层特征信息，在长序列文本中易丢失上下文信息。针对传统LSTM中输入和隐藏状态之间相互独立且缺少信息交互的问题，建立一种基于Mogrifier LSTM的序列标注关系抽取模型。将由词嵌入、字符嵌入和位置嵌入构成的嵌入层结果输入Mogrifier LSTM层，该层通过在传统LSTM计算之前交替地让当前输入与之前隐藏状态进行多轮运算，以增强上下文交互能力。使用自注意力机制提高模型对重要特征的关注度，同时采用基于关系的注意力机制获取特定关系下的句子表示，从而解决实体关系重叠问题。在关系分类模块，利用Bi-LSTM进行序列标注，将句子中的每个词汇映射为相应的标签。实验结果表明，在NYT数据集上该模型的F1值达到0.841，优于HRL、OrderRL等模型，在存在SEO重叠类型的关系抽取中，F1值仍能达到0.745，所提模型能够在一定程度上解决关系重叠问题同时有效提升关系抽取性能。

关键词: 关系抽取, Mogrifier LSTM模型, 上下文交互, 注意力机制, 关系重叠

Abstract: Making full use of the context information in the text can improve the relationship extraction performance.However, most current methods for deep-learning-based relationship extraction only capture the shallow feature information in the text sequence, and it is easy to lose the context information in the long sequence of text.A sequence tagging relationship extraction model based on Mogrifier Long Short-Term Memory(LSTM) is established to solve the independent input and hidden states and lack of information interaction in the traditional LSTM.The results of the embedding layer composed of word embedding, character embedding, and position embedding are input into the Mogrifier LSTM layer, which alternately performs multiple operations between the current input and the previous hidden states before the traditional LSTM calculation to improve the context interaction ability.The self-attention mechanism is used to improve the attention of the model to the critical features, and the relationship-based attention mechanism is used to obtain the sentence representation in the specific relationship to solve the problem of entity relationship overlap.In the relationship classification module, Bi-LSTM is used for sequence tagging to map each word in a sentence to a corresponding label.The experimental results show that the F1 value of the proposed model on the NYT dataset reaches 0.841, which is better than those of the HRL and OrderRL models.In relationship extraction with SEO overlap types, the F1 value reaches 0.745.The proposed model can reasonably solve the relationship overlap problem and effectively improve relationship extraction performance.

Key words: relationship extraction, Mogrifier LSTM model, context interaction, attention mechanism, relationship overlap

中图分类号:

TP391

方义秋, 刘飞, 葛君伟. 基于Mogrifier LSTM的序列标注关系抽取方法[J]. 计算机工程, 2022, 48(10): 81-87,94.

FANG Yiqiu, LIU Fei, GE Junwei. Method for Sequence Tagging Relationship Extraction Based on Mogrifier LSTM[J]. Computer Engineering, 2022, 48(10): 81-87,94.

http://www.ecice06.com/CN/Y2022/V48/I10/81

图/表 9

参考文献

[1] 石赫, 杨群, 刘绍翰, 等.基于深度学习的电网故障预案信息抽取研究[J].计算机科学, 2020, 47(S2):52-56. SHI H, YANG Q, LIU S H, et al.Study on information extraction of power grid fault emergency pre-plans based on deep learning[J].Computer Science, 2020, 47(S2):52-56.(in Chinese)
[2] 邢毅雪, 朱永华, 高海燕, 等.基于注意力机制的远程监督实体关系抽取[J].上海大学学报(自然科学版), 2021, 27(5):983-992. XING Y X, ZHU Y H, GAO H Y, et al.Distant supervision for relation extraction via attention CNNs[J].Journal of Shanghai University(Natural Science Edition), 2021, 27(5):983-992.(in Chinese)
[3] ZHONG Z X, CHEN D Q.A frustratingly easy approach for entity and relation extraction[EB/OL].[2021-10-05].https://arxiv.org/abs/2010.12812.
[4] WANG J, LU W.Two are better than one:joint entity and relation extraction with table-sequence encoders[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing.Washington D.C., USA:IEEE Press, 2020:125-136.
[5] 唐敏.基于深度学习的中文实体关系抽取方法研究[D].成都:西南交通大学, 2018. TANG M.Research on Chinese entity relation extraction based on deep learning[D].Chengdu:Southwest Jiaotong University, 2018.(in Chinese)
[6] KAMBHATLA N.Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations[EB/OL].[2021-10-05].https://aclanthology.org/P04-3022.pdf.
[7] MINTZ M, BILLS S, SNOW R, et al.Distant supervision for relation extraction without labeled data[EB/OL].[2021-10-05].https://web.stanford.edu/~jurafsky/mintz.pdf.
[8] HU Z K, CAO Y X, HUANG L F, et al.How knowledge graph and attention help? A quantitative analysis into bag-level relation extraction[EB/OL].[2021-10-05].https://arxiv.org/abs/2107.12064.
[9] RIEDEL S, YAO L M, MCCALLUM A.Modeling relations and their mentions without labeled text[EB/OL].[2021-10-05].https://maroo.cs.umass.edu/getpdf.php?id=931.
[10] 余小康, 陈岭, 郭敬, 等.结合从句级远程监督与半监督集成学习的关系抽取方法[J].模式识别与人工智能, 2017, 30(1):54-63. YU X K, CHEN L, GUO J, et al.Relation extraction method combining clause level distant supervision and semi-supervised ensemble learning[J].Pattern Recognition and Artificial Intelligence, 2017, 30(1):54-63.(in Chinese)
[11] HASEGAWA T, SEKINE S, GRISHMAN R.Discovering relations among named entities from large corpora[C]//Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics.[S.l.]:ACL, 2004:415-422.
[12] WEI Z P, SU J L, WANG Y, et al.A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:ACL, 2020:1476-1488.
[13] HANG T T, FENG J, WU Y R, et al.Joint extraction of entities and overlapping relations using source-target entity labeling[J].Expert Systems with Applications, 2021, 177:114853.
[14] MELIS G, KOČISKÝ T, BLUNSOM P.Mogrifier LSTM[EB/OL].[2021-10-05].https://arxiv.org/abs/1909.01792.
[15] ZENG D, LIU K, LAI S, et al.Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics:Technical Papers.Dublin, Ireland:COLING Press, 2014:2335-2344.
[16] GENG Z Q, CHEN G F, HAN Y M, et al.Semantic relation extraction using sequential and tree-structured LSTM with attention[J].Information Sciences, 2020, 509:183-192.
[17] 闫雄, 段跃兴, 张泽华.采用自注意力机制和CNN融合的实体关系抽取[J].计算机工程与科学, 2020, 42(11):2059-2066. YAN X, DUAN Y X, ZHANG Z H.Entity relationship extraction fusing self-attention mechanism and CNN[J].Computer Engineering & Science, 2020, 42(11):2059-2066.(in Chinese)
[18] YUAN Y, ZHOU X F, PAN S R, et al.A relation-specific attention network for joint entity and relation extraction[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence.Washington D.C., USA:IEEE Press, 2020:4054-4060.
[19] LIN Y K, SHEN S Q, LIU Z Y, et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:ACL, 2016:2124-2133.
[20] ZHENG S C, WANG F, BAO H Y, et al.Joint extraction of entities and relations based on a novel tagging scheme[EB/OL].[2021-10-05].https://arxiv.org/abs/1706.05075.
[21] ZENG X R, ZENG D J, HE S Z, et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.[S.l.]:ACL, 2018:506-514.
[22] BEKOULIS G, DELEU J, DEMEESTER T, et al.Joint entity recognition and relation extraction as a multi-head selection problem[J].Expert Systems with Applications, 2018, 114:34-45.
[23] ZENG D J, ZHANG H R, LIU Q Y.CopyMTL:copy mechanism for joint extraction of entities and relations with multi-task learning[J].Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5):9507-9514.
[24] ZENG X R, HE S Z, ZENG D J, et al.Learning the extraction order of multiple relational facts in a sentence with reinforcement learning[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.[S.l.]:ACL, 2019:367-377.
[25] TAKANOBU R, ZHANG T Y, LIU J X, et al.A hierarchical framework for relation extraction with reinforcement learning[J].Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33:7072-7079.
[26] YU B, ZHANG Z, SHU X, et al.Joint extraction of entities and relations based on a novel decomposition strategy[C]//Proceedings of the 24th European Conference on Artificial Intelligence.Berlin, Germany:Springer, 2020:2282-2289.

选择文件类型/文献管理软件名称

选择包含的内容