面向地质领域的实体关系联合抽取研究

doi:10.19678/j.issn.1000-3428.0064077

计算机工程 ›› 2023, Vol. 49 ›› Issue (3): 121-127. doi: 10.19678/j.issn.1000-3428.0064077

面向地质领域的实体关系联合抽取研究

吴雪莹¹, 段友祥¹, 昌伦杰², 李世银², 孙歧峰¹

1. 中国石油大学(华东) 计算机科学与技术学院, 山东青岛 266580;
2. 中国石油塔里木油田分公司勘探开发研究院, 新疆库尔勒 841000

收稿日期:2022-03-02 修回日期:2022-05-05 发布日期:2022-05-25
作者简介:吴雪莹(1998—),女,硕士研究生,主研方向为自然语言处理;段友祥,教授、博士;昌伦杰,教授级高级工程师;李世银,高级工程师;孙歧峰,讲师、博士。
基金资助:
中石油重大科技项目（ZD2019-183-006）；中央高校基本科研业务费专项资金（20CX05017A）。

Research on Entity and Relation Joint Extraction for Geological Domain

WU Xueying¹, DUAN Youxiang¹, CHANG Lunjie², LI Shiyin², SUN Qifeng¹

1. College of Computer Science and Technology, China University of Petroleum(East China), Qingdao 266580, Shandong, China;
2. Research Institute of Exploration & Development, PetroChina Tarim Oilfield Company, Korla 841000, Xinjiang, China

Received:2022-03-02 Revised:2022-05-05 Published:2022-05-25

摘要/Abstract

摘要： 构建地质领域的知识图谱有助于便捷高效地共享和应用多源地质知识，而地质关系三元组抽取对地质领域知识图谱构建具有重要意义。针对现有实体关系联合抽取模型无法有效识别重叠三元组的问题，考虑地质领域专业知识的特殊性，基于预训练语言模型BERT建立一种用于地质领域关系三元组抽取的层级标注模型HtERT。采用中文预训练语言模型BERT-wwm替代原始的BERT模型作为底层编码器，以提高模型对中文的编码能力。在实体识别阶段，引入关于实体起始位置的嵌入表示来限制实体的抽取长度，从而提升实体识别的准确度。引入全局上下文信息和BiLSTM网络使得模型抽取到的特征能更精确地表示地质样本信息，增强模型对地质关系三元组以及重叠三元组的抽取能力。在地质领域数据集上的实验结果表明，HtERT模型相比于PCNN、BiLSTM、PCNN+ATT、CASREL等基线模型具有明显优势，精确率、召回率以及F1值分别平均提升15.24、10.96和13.20个百分点，验证了该模型在地质领域实体关系联合抽取任务中的有效性。

关键词: 实体关系抽取, 联合抽取, 重叠三元组, 地质领域, 预训练模型BERT

Abstract: Constructing a knowledge graph in the geological domain facilitates convenient and efficient sharing and utilization of multi-source geological knowledge, with the extraction of geological relation triples being of paramount importance.Aiming to address the inadequacy of existing entity and relationship joint extraction models in effectively identifying overlapping triples, this paper proposes a hierarchical annotation model, HtERT, for geological domain relationship triplet extraction, leveraging the specificities of geological domain expertise and the pretrained language model BERT.The original BERT model is replaced with the Chinese pretraining language model BERT-wwm as the underlying encoder, thereby enhancing the model's coding capacity for Chinese.During entity recognition, the embedded representation of the initial position of the entity is introduced to limit the extraction length of the entity and to improve the accuracy of entity recognition.Introducing global context information and a BiLSTM network enables the features extracted from the model to more accurately represent the geological sample information, and enhances the ability of the model to extract geological relation triples and overlapping triples.HtERT outperforms baseline models, such as PCNN, BiLSTM, PCNN+ATT, and CASREL, on the geological domain dataset.The Precision, Recall, and F1 values increase by 15.24, 10.96, and 13.20 percentage points, respectively, which verifies the effectiveness of the model in the joint extraction task of entity and relation in the geological domain.

Key words: entity and relation extraction, joint extraction, overlapping triple, geological domain, pre-training model BERT

中图分类号:

TP18

吴雪莹, 段友祥, 昌伦杰, 李世银, 孙歧峰. 面向地质领域的实体关系联合抽取研究[J]. 计算机工程, 2023, 49(3): 121-127.

WU Xueying, DUAN Youxiang, CHANG Lunjie, LI Shiyin, SUN Qifeng. Research on Entity and Relation Joint Extraction for Geological Domain[J]. Computer Engineering, 2023, 49(3): 121-127.

https://www.ecice06.com/CN/Y2023/V49/I3/121

图/表 9

20230314185132

20230314185137

20230314185140

20230314185143

20230314185147

20230314185150

20230314185153

20230314185156

20230314185159

参考文献

[1] DEVLIN J, CHANG M W, LEE K, et al.BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL].[2022-02-05].https://arxiv.org/abs/1810.04805.
[2] ZELENKO D, AONE C, RICHARDELLA A.Kernel methods for relation extraction[J].Journal of Machine Learning Research, 2003, 3(6):1083-1106.
[3] MINTZ M, BILLS S, SNOW R, et al.Distant supervision for relation extraction without labeled data[C]//Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing.New York, USA:ACM Press, 2009:1003-1011.
[4] GORMLEY M R, YU M, DREDZE M.Improved relation extraction with feature-rich compositional embedding models[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2015:156-160.
[5] LI Q, JI H.Incremental joint extraction of entity mentions and relations[EB/OL].[2022-02-05].https://aclanthology.org/P14-1038.pdf.
[6] MIWA M, SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2014:17-23.
[7] REN X, WU Z Q, HE W Q, et al.CoType:joint extraction of typed entities and relations with knowledge bases[EB/OL].[2022-02-05].https://arxiv.org/abs/1610.08763.
[8] SANTOS C N D, XIANG B, ZHOU B W.Classifying relations by ranking with convolutional neural networks[EB/OL].[2022-02-05].https://arxiv.org/abs/1504.06580.
[9] NGUYEN T H, GRISHMAN R.Combining neural networks and log-linear models to improve relation extraction[EB/OL].[2022-02-05].https://arxiv.org/abs/1511.05926.
[10] 张军莲, 张一帆, 汪鸣泉, 等.基于图卷积神经网络的中文实体关系联合抽取[J].计算机工程, 2021, 47(12):103-111. ZHANG J L, ZHANG Y F, WANG M Q, et al.Joint extraction of Chinese entity relations based on graph convolutional neural network[J].Computer Engineering, 2021, 47(12):103-111.(in Chinese)
[11] MIWA M, BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:18-25.
[12] ZHENG S C, HAO Y X, LU D Y, et al.Joint entity and relation extraction based on a hybrid neural network[J].Neurocomputing, 2017, 257:59-66.
[13] ZHENG S C, WANG F, BAO H Y, et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2017:122-135.
[14] ZENG X R, ZENG D J, HE S Z, et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2018:102-110.
[15] FU T J, LI P H, MA W Y.GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2019:1409-1418.
[16] WEI Z P, SU J L, WANG Y, et al.A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2020:1476-1488.
[17] 黄徐胜, 朱月琴, 付立军, 等.基于BERT的金矿地质实体关系抽取模型研究[J].地质力学学报, 2021, 27(3):391-399. HUANG X S, ZHU Y Q, FU L J, et al.Research on a geological entity relation extraction model for gold mine based on BERT[J].Journal of Geomechanics, 2021, 27(3):391-399.(in Chinese)
[18] LUO X, ZHOU W W, WANG W P, et al.Attention-based relation extraction with bidirectional gated recurrent unit and highway network in the analysis of geological data[J].IEEE Access, 2017, 6:5705-5715.
[19] 王智广, 文红英, 鲁强, 等.地质领域开放式实体关系联合抽取[J].计算机工程与设计, 2021, 42(4):996-1005. WANG Z G, WEN H Y, LU Q, et al.Joint extraction of open entity relation in geological field[J].Computer Engineering and Design, 2021, 42(4):996-1005.(in Chinese)
[20] 张雪英, 叶鹏, 王曙, 等.基于深度信念网络的地质实体识别方法[J].岩石学报, 2018, 34(2):343-351. ZHANG X Y, YE P, WANG S, et al.Geological entity recognition method based on deep belief networks[J].Acta Petrologica Sinica, 2018, 34(2):343-351.(in Chinese)
[21] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2017:6000-6010.
[22] CUI Y M, CHE W X, LIU T, et al.Pre-training with whole word masking for Chinese BERT[J].IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29:3504-3514.
[23] CHE W X, LI Z H, LIU T.LTP:a Chinese language technology platform[C]//Proceedings of the 23rd International Conference on Computational Linguistics:Demonstrations.New York, USA:ACM Press, 2010:13-16.
[24] LIN Y K, SHEN S Q, LIU Z Y, et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:2124-2133.
[25] ZHOU P, SHI W, TIAN J, et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:207-212.

选择文件类型/文献管理软件名称

选择包含的内容

面向地质领域的实体关系联合抽取研究

Research on Entity and Relation Joint Extraction for Geological Domain

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 12

编辑推荐

Metrics

本文评价

[1]	廖涛, 孙皓洁, 张顺香. 基于跨度和特征融合的实体关系联合抽取模型[J]. 计算机工程, 2023, 49(6): 107-114.
[2]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[3]	杨红菊, 靳新宇. 一个实体关系与事件抽取的通用模型[J]. 计算机工程, 2023, 49(2): 143-149.
[4]	禹克强, 黄芳, 吴琪, 欧阳洋. 基于双向语义的中文实体关系联合抽取方法[J]. 计算机工程, 2023, 49(1): 92-99,112.
[5]	陈仁杰, 郑小盈, 祝永新. 融合实体类别信息的实体关系联合抽取[J]. 计算机工程, 2022, 48(3): 46-53.
[6]	张军莲, 张一帆, 汪鸣泉, 黄永健. 基于图卷积神经网络的中文实体关系联合抽取[J]. 计算机工程, 2021, 47(12): 103-111.
[7]	张志昌, 周侗, 张瑞芳, 张敏钰. 融合双向GRU与注意力机制的医疗实体关系识别[J]. 计算机工程, 2020, 46(6): 296-302.
[8]	李明耀,杨静. 基于依存分析的开放式中文实体关系抽取方法[J]. 计算机工程, 2016, 42(6): 201-207.
[9]	谭永兴, 罗军勇, 尹美娟. Markov逻辑网及其在信息抽取中的应用[J]. 计算机工程, 2012, 38(18): 162-165.
[10]	姚全珠, 王美君, 李如琼. 基于子树特征的中文实体关系抽取[J]. 计算机工程, 2012, 38(01): 48-50,54.
[11]	李妩可, 郭赛球, 尹艳. 命名实体关系抽取算法的改进[J]. 计算机工程, 2010, 36(24): 289-封三.
[12]	周晶. 基于条件随机域模型的中文实体关系抽取[J]. 计算机工程, 2010, 36(24): 192-194.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

面向地质领域的实体关系联合抽取研究

Research on Entity and Relation Joint Extraction for Geological Domain

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 12

编辑推荐

Metrics

本文评价