Document-Level Relation Extraction Based on Evidential Sentences and Graph Convolutional Network

doi:10.19678/j.issn.1000-3428.0065461

Abstract

Abstract:

In document-level relation extraction models based on a Graph Convolutional Network(GCN), the contribution of neighboring nodes and sentence noise cannot be distinguished. To address this issue, an improved document-level relation extraction model is built, whereby evidential sentences are integrated into the GCN for message propagation. Based on heuristic paths, path information containing evidential sentences is obtained to extract relations. The proportion of sentences in all sample paths is counted, and evidential sentence path information is integrated into heterogeneous graphs for similarity calculations to obtain three evidential sentences related to the samples. The evidential sentence information is subsequently used to assign corresponding weights to different types of edges according to contribution differentiation rules. A graph convolution operation is used to enhance the node information twice, ultimately achieving document-level relation extraction. The experimental results on the Document-level Relation Extraction Dataset(DocRED) show that the F1 value of the model reaches 56.96%, which is 0.42-13.51 percentage points higher than those of models such as Paths and Hin-Glove. This verifies the effectiveness of incorporating evidential sentence information into document graphs to improve the performance of document-level relation extraction models.

Key words: document-level relation extraction, Graph Convolutional Network(GCN), evidential sentence, heterogeneous graph, weight

摘要：

针对基于图卷积网络的文档级关系抽取模型存在未对邻居节点贡献度加以区分及句子噪声的问题，在将证据句融入图卷积网络进行消息传播的基础上，构建一种改进的文档级关系抽取模型。基于启发式路径得到包含证据句的路径信息，在包含证据句的路径信息基础上进行关系抽取，统计所有样本路径中的句子占比，并在异构图中融入证据句路径信息进行相似度计算，得到与样本相关的3句证据句。在证据句信息的基础上对不同类型的边根据贡献度区分规则赋予相应权重，并使用图卷积操作对节点信息进行二次增强，最终实现文档级关系抽取。在DocRED数据集上的实验结果表明，该模型的F1值达到56.96%，相比于Paths、Hin-Glove等基线模型提升了0.42~13.51个百分点，验证了在文档图中融入证据句信息对于提升文档级关系抽取模型性能的有效性。

关键词: 文档级关系抽取, 图卷积网络, 证据句, 异构图, 权重

Jianhong MA, Tian GONG, Shuang YAO. Document-Level Relation Extraction Based on Evidential Sentences and Graph Convolutional Network[J]. Computer Engineering, 2023, 49(8): 104-110.

马建红, 龚天, 姚爽. 基于证据句与图卷积网络的文档级关系抽取[J]. 计算机工程, 2023, 49(8): 104-110.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0065461

https://www.ecice06.com/EN/Y2023/V49/I8/104

Figures/Tables 6

References 25

1	ZENG D, LIU K, LAI S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics. Dublin, Republic of Ireland: [s. n.], 2014: 2335-2344.
2	ZHANG S, ZHENG D, HU X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language. New York, USA: ACM Press, 2015: 73-78.
3	ZHANG Y H, QI P, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2018: 2205-2215.
4	李明耀, 杨静. 基于依存分析的开放式中文实体关系抽取方法. 计算机工程, 2016, 42 (6): 201- 207. URL
	LI M Y, YANG J. Open Chinese entity relation extraction method based on dependency parsing. Computer Engineering, 2016, 42 (6): 201- 207. URL
5	YAO Y A, YE D M, LI P, et al. DocRED: a large-scale document-level relation extraction dataset[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2019: 764-777.
6	VERGA P, STRUBELL E, MCCALLUM A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction[C]//Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: Association for Computational Linguistics, 2018: 872-884.
7	CHRISTOPOULOU F, MIWA M, ANANIADOU S. Connecting the dots: document-level neural relation extraction with edge-oriented graphs[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2019: 4925-4936.
8	SAHU S K, CHRISTOPOULOU F, MIWA M, et al. Inter-sentence relation extraction with document-level graph convolutional neural network[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2019: 4309-4316.
9	NAN G S, GUO Z J, SEKULIC I, et al. Reasoning with latent structure refinement for document-level relation extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2020: 1546-1557.
10	ZENG S A, XU R X, CHANG B B, et al. Double graph based reasoning for document-level relation extraction[C]//Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2020: 1630-1640.
11	PENNINGTON J, SOCHER R, MANNING C. Glove: global vectors for word representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2014: 1532-1543.
12	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. [2022-07-14]. https://arxiv.org/abs/1810.04805.
13	FU S C, LIU W F, ZHANG K, et al. Semi-supervised classification by graph p-Laplacian convolutional networks. Information Sciences, 2021, 560, 92- 106. doi: 10.1016/j.ins.2021.01.075
14	QUIRK C, POON H. Distant supervision for relation extraction beyond the sentence boundary[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2017: 1171-1182.
15	XU W, CHEN K H, ZHAO T J. Document-level relation extraction with reconstruction[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 14167-14175.
16	VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. [2022-07-11]. http://arxiv.org/abs/1710.10903.
17	GUO Z J, ZHANG Y, LU W. Attention guided graph convolutional networks for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2019: 241-251.
18	吴婷, 孔芳. 基于图注意力卷积神经网络的文档级关系抽取. 中文信息学报, 2021, 35 (10): 73- 80. URL
	WU T, KONG F. Document-level relation extraction based on graph attention convolutional neural network. Journal of Chinese Information Processing, 2021, 35 (10): 73- 80. URL
19	ZHANG Z Y, YU B W, SHU X B, et al. Document-level relation extraction with dual-tier heterogeneous graph[C]//Proceedings of the 28th International Conference on Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2020: 1630-1641.
20	ZHOU W X, HUANG K, MA T Y, et al. Document-level relation extraction with adaptive thresholding and localized context pooling[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 14612-14620.
21	HUANG Q Z, ZHU S Q, FENG Y S, et al. Three sentences are all you need: local path enhanced document relation extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2021: 998-1004.
22	WU Y, LUO R B, LEUNG H C M, et al. RENET: a deep learning approach for extracting gene-disease associations from literature[C]//Proceedings of International Conference on Research in Computational Molecular Biology. Berlin, Germany: Springer, 2019: 272-284.
23	LI J, SUN Y P, JOHNSON R J, et al. BioCreative V CDR task corpus: a resource for chemical disease relation extraction[EB/OL]. [2022-07-11]. https://pubmed.ncbi.nlm.nih.gov/27161011/.
24	TANG H Z, CAO Y N, ZHANG Z Y, et al. Hin: hierarchical inference network for document-level relation extraction[C]//Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining. Berlin, Germany: Springer, 2020: 197-209.
25	SOROKIN D, GUREVYCH I. Context-aware representations for knowledge base relation extraction[C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2017: 1784-1789.

实验序号	GCN层数	训练集中的F1值/%	验证集中的F1值/%
1	1和1	67.04	56.96
2	1和2	62.74	53.48
3	2和1	60.45	52.27
4	2和2	58.43	51.24

实验序号	GCN层数	训练集中的F1值/%	验证集中的F1值/%
1	1和1	67.04	56.96
2	1和2	62.74	53.48
3	2和1	60.45	52.27
4	2和2	58.43	51.24

实验序号	实体对表征	维度	验证集中的F1值/%
1	$ \left[{\boldsymbol{l}}_{1};{\boldsymbol{l}}_{2};\boldsymbol{g}\right] $	512	52.55
2	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};{\boldsymbol{l}}_{1}\right] $	512	55.98
3	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};\left({\boldsymbol{l}}_{1}+\boldsymbol{g}\right)/2\right] $	512	56.50
4	$ \left[{\boldsymbol{l}}_{1};{\boldsymbol{l}}_{2}\right] $	512	56.49
5	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};\left({\boldsymbol{l}}_{1}+\boldsymbol{g}\right)/2\right] $	768	56.96
6	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};{\boldsymbol{l}}_{1}\right] $	768	56.23

实验序号	实体对表征	维度	验证集中的F1值/%
1	$ \left[{\boldsymbol{l}}_{1};{\boldsymbol{l}}_{2};\boldsymbol{g}\right] $	512	52.55
2	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};{\boldsymbol{l}}_{1}\right] $	512	55.98
3	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};\left({\boldsymbol{l}}_{1}+\boldsymbol{g}\right)/2\right] $	512	56.50
4	$ \left[{\boldsymbol{l}}_{1};{\boldsymbol{l}}_{2}\right] $	512	56.49
5	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};\left({\boldsymbol{l}}_{1}+\boldsymbol{g}\right)/2\right] $	768	56.96
6	$ \left[{\boldsymbol{l}}_{0};{\boldsymbol{l}}_{2};{\boldsymbol{l}}_{1}\right] $	768	56.23

实验序号	权重		验证集中的1值/%
实验序号	M-M	S-S	验证集中的1值/%
1	0	0.5或$ 1+{s}_{\mathrm{s}\mathrm{i}\mathrm{m}} $	56.34
2	0或1	0.5或$ 1+{s}_{\mathrm{s}\mathrm{i}\mathrm{m}} $或1	56.96
3	0或1	$ 1+{s}_{\mathrm{s}\mathrm{i}\mathrm{m}} $或1	55.87

Please choose a citation manager

Content to export