面向地质领域的实体关系联合抽取研究

doi:10.19678/j.issn.1000-3428.0064077

摘要/Abstract

摘要： 构建地质领域的知识图谱有助于便捷高效地共享和应用多源地质知识，而地质关系三元组抽取对地质领域知识图谱构建具有重要意义。针对现有实体关系联合抽取模型无法有效识别重叠三元组的问题，考虑地质领域专业知识的特殊性，基于预训练语言模型BERT建立一种用于地质领域关系三元组抽取的层级标注模型HtERT。采用中文预训练语言模型BERT-wwm替代原始的BERT模型作为底层编码器，以提高模型对中文的编码能力。在实体识别阶段，引入关于实体起始位置的嵌入表示来限制实体的抽取长度，从而提升实体识别的准确度。引入全局上下文信息和BiLSTM网络使得模型抽取到的特征能更精确地表示地质样本信息，增强模型对地质关系三元组以及重叠三元组的抽取能力。在地质领域数据集上的实验结果表明，HtERT模型相比于PCNN、BiLSTM、PCNN+ATT、CASREL等基线模型具有明显优势，精确率、召回率以及F1值分别平均提升15.24、10.96和13.20个百分点，验证了该模型在地质领域实体关系联合抽取任务中的有效性。

关键词: 实体关系抽取, 联合抽取, 重叠三元组, 地质领域, 预训练模型BERT

Abstract: Constructing a knowledge graph in the geological domain facilitates convenient and efficient sharing and utilization of multi-source geological knowledge, with the extraction of geological relation triples being of paramount importance.Aiming to address the inadequacy of existing entity and relationship joint extraction models in effectively identifying overlapping triples, this paper proposes a hierarchical annotation model, HtERT, for geological domain relationship triplet extraction, leveraging the specificities of geological domain expertise and the pretrained language model BERT.The original BERT model is replaced with the Chinese pretraining language model BERT-wwm as the underlying encoder, thereby enhancing the model's coding capacity for Chinese.During entity recognition, the embedded representation of the initial position of the entity is introduced to limit the extraction length of the entity and to improve the accuracy of entity recognition.Introducing global context information and a BiLSTM network enables the features extracted from the model to more accurately represent the geological sample information, and enhances the ability of the model to extract geological relation triples and overlapping triples.HtERT outperforms baseline models, such as PCNN, BiLSTM, PCNN+ATT, and CASREL, on the geological domain dataset.The Precision, Recall, and F1 values increase by 15.24, 10.96, and 13.20 percentage points, respectively, which verifies the effectiveness of the model in the joint extraction task of entity and relation in the geological domain.

Key words: entity and relation extraction, joint extraction, overlapping triple, geological domain, pre-training model BERT

中图分类号:

TP18

吴雪莹, 段友祥, 昌伦杰, 李世银, 孙歧峰. 面向地质领域的实体关系联合抽取研究[J]. 计算机工程, 2023, 49(3): 121-127.

WU Xueying, DUAN Youxiang, CHANG Lunjie, LI Shiyin, SUN Qifeng. Research on Entity and Relation Joint Extraction for Geological Domain[J]. Computer Engineering, 2023, 49(3): 121-127.

http://www.ecice06.com/CN/Y2023/V49/I3/121

图/表 9

20230314185132

20230314185137

20230314185140

20230314185143

20230314185147

20230314185150

20230314185153

20230314185156

20230314185159

参考文献

[1] DEVLIN J, CHANG M W, LEE K, et al.BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL].[2022-02-05].https://arxiv.org/abs/1810.04805.
[2] ZELENKO D, AONE C, RICHARDELLA A.Kernel methods for relation extraction[J].Journal of Machine Learning Research, 2003, 3(6):1083-1106.
[3] MINTZ M, BILLS S, SNOW R, et al.Distant supervision for relation extraction without labeled data[C]//Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing.New York, USA:ACM Press, 2009:1003-1011.
[4] GORMLEY M R, YU M, DREDZE M.Improved relation extraction with feature-rich compositional embedding models[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2015:156-160.
[5] LI Q, JI H.Incremental joint extraction of entity mentions and relations[EB/OL].[2022-02-05].https://aclanthology.org/P14-1038.pdf.
[6] MIWA M, SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2014:17-23.
[7] REN X, WU Z Q, HE W Q, et al.CoType:joint extraction of typed entities and relations with knowledge bases[EB/OL].[2022-02-05].https://arxiv.org/abs/1610.08763.
[8] SANTOS C N D, XIANG B, ZHOU B W.Classifying relations by ranking with convolutional neural networks[EB/OL].[2022-02-05].https://arxiv.org/abs/1504.06580.
[9] NGUYEN T H, GRISHMAN R.Combining neural networks and log-linear models to improve relation extraction[EB/OL].[2022-02-05].https://arxiv.org/abs/1511.05926.
[10] 张军莲, 张一帆, 汪鸣泉, 等.基于图卷积神经网络的中文实体关系联合抽取[J].计算机工程, 2021, 47(12):103-111. ZHANG J L, ZHANG Y F, WANG M Q, et al.Joint extraction of Chinese entity relations based on graph convolutional neural network[J].Computer Engineering, 2021, 47(12):103-111.(in Chinese)
[11] MIWA M, BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:18-25.
[12] ZHENG S C, HAO Y X, LU D Y, et al.Joint entity and relation extraction based on a hybrid neural network[J].Neurocomputing, 2017, 257:59-66.
[13] ZHENG S C, WANG F, BAO H Y, et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2017:122-135.
[14] ZENG X R, ZENG D J, HE S Z, et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2018:102-110.
[15] FU T J, LI P H, MA W Y.GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2019:1409-1418.
[16] WEI Z P, SU J L, WANG Y, et al.A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2020:1476-1488.
[17] 黄徐胜, 朱月琴, 付立军, 等.基于BERT的金矿地质实体关系抽取模型研究[J].地质力学学报, 2021, 27(3):391-399. HUANG X S, ZHU Y Q, FU L J, et al.Research on a geological entity relation extraction model for gold mine based on BERT[J].Journal of Geomechanics, 2021, 27(3):391-399.(in Chinese)
[18] LUO X, ZHOU W W, WANG W P, et al.Attention-based relation extraction with bidirectional gated recurrent unit and highway network in the analysis of geological data[J].IEEE Access, 2017, 6:5705-5715.
[19] 王智广, 文红英, 鲁强, 等.地质领域开放式实体关系联合抽取[J].计算机工程与设计, 2021, 42(4):996-1005. WANG Z G, WEN H Y, LU Q, et al.Joint extraction of open entity relation in geological field[J].Computer Engineering and Design, 2021, 42(4):996-1005.(in Chinese)
[20] 张雪英, 叶鹏, 王曙, 等.基于深度信念网络的地质实体识别方法[J].岩石学报, 2018, 34(2):343-351. ZHANG X Y, YE P, WANG S, et al.Geological entity recognition method based on deep belief networks[J].Acta Petrologica Sinica, 2018, 34(2):343-351.(in Chinese)
[21] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2017:6000-6010.
[22] CUI Y M, CHE W X, LIU T, et al.Pre-training with whole word masking for Chinese BERT[J].IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29:3504-3514.
[23] CHE W X, LI Z H, LIU T.LTP:a Chinese language technology platform[C]//Proceedings of the 23rd International Conference on Computational Linguistics:Demonstrations.New York, USA:ACM Press, 2010:13-16.
[24] LIN Y K, SHEN S Q, LIU Z Y, et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:2124-2133.
[25] ZHOU P, SHI W, TIAN J, et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:207-212.

选择文件类型/文献管理软件名称

选择包含的内容