动态聚合实体和关系上下文的知识图谱补全

doi:10.19678/j.issn.1000-3428.0065410

摘要/Abstract

摘要：

目前关于知识图谱补全的研究通常只进行实体嵌入增强，弱化了关系附加信息对链路预测的影响，并忽略了知识图谱中实体和关系的结构关联关系。为解决上述问题，提出一种动态聚合实体和关系上下文的知识图谱补全模型(DAERC)。采用编码器-解码器结构，获取并筛选最优上下文来增强实体和关系表示。编码器使用改进的图注意力机制设计关系和实体聚合器，以累乘的方式动态聚合上下文信息，并将关系聚合器的输出应用于实体聚合过程，缓解了因独立聚合造成的实体关系交互性较弱的问题。解码器通过不同的评分函数获取每个候选三元组的得分，验证三元组的合理性。实验结果表明，DAERC有效地增强了TransE、ConvE、RotatE这3类知识图谱嵌入模型的实体和关系嵌入表示能力，在FB15k-237和WN18RR数据集上Hits@10指标相较于表现最好的对照CoNE模型分别提升约5.2%、2.1%、1.7%和7.2%、2.3%、2.2%。

关键词: 知识图谱补全, 图结构, 上下文, 注意力系数, 动态聚合

Abstract:

Most of the recent research on Knowledge Graph Completion(KGC) usually enhances entity embedding alone, which weakens the impact of additional information about relation on link prediction and ignores the structural association relation between entities and those in the knowledge graph.To address these issues, a KGC model with dynamic aggregation of the contexts of entities and relations, called DAERC, is proposed. The model is an encoder-decoder architecture; it obtains and filters the optimal contexts to enhance entity and relation embedding representation. The encoder uses an improved graph attention mechanism to design a relation and an entity aggregator that dynamically aggregates context information by multiplying. Furthermore, it applies the output of the relation aggregator to the entity aggregator, and alleviates the problem of weak entity-relation interactivity caused by independent aggregation.The decoder obtains the score of each candidate triplet through different score functions to predict the plausibility for each triplet. The results reveal that the DAERC effectively enhances the ability to embed and represent entities and relations for three different Knowledge Graph Embedding(KGE) model: TransE, ConvE, and RotatE. On the FB15k-237 dataset, compared to the best performing CoNE model applied to TransE, ConvE, and RotatE, DAERC's Hits@10 is increased by approximately 5.2%, 2.1%, and 1.7%, and improved by approximately 7.2%, 2.3%, and 2.2% on the WN18RR dataset.

Key words: Knowledge Graph Completion(KGC), graph structure, context, attention coefficient, dynamic aggregation

马坤, 安敬民, 李冠宇. 动态聚合实体和关系上下文的知识图谱补全[J]. 计算机工程, 2023, 49(8): 77-84, 95.

Kun MA, Jingmin AN, Guanyu LI. Knowledge Graph Completion with Dynamically Aggregating Context of Entity and Relation[J]. Computer Engineering, 2023, 49(8): 77-84, 95.

http://www.ecice06.com/CN/Y2023/V49/I8/77

图/表 10

参考文献 32

1	李涓子, 侯磊. 知识图谱研究综述. 山西大学学报(自然科学版), 2017, 40 (3): 454- 459. URL
	LI J Z, HOU L. Reviews on knowledge graph research. Journal of Shanxi University(Natural Science Edition), 2017, 40 (3): 454- 459. URL
2	CHEN Z, WANG Y H, ZHAO B, et al. Knowledge graph completion: a review. IEEE Access, 2020, 8, 192435- 192456. doi: 10.1109/ACCESS.2020.3030076
3	JIANG K, WU D, JIANG H. FreebaseQA: a new factoid QA data set matching trivia-style question-answer pairs with Freebase[C]//Proceedings of North American Chapter of the Association for Computational Linguistics. Philadelphia, USA: Association for Computational Linguistics, 2019: 318-323.
4	AUER S, BIZER C, KOBILAROV G, et al. DBpedia: a nucleus for a Web of open data[C]//Proceedings of International Semantic Web Conference, Asian Semantic Web Conference. Berlin, Germany: Springer, 2007: 722-735.
5	BOIŃSKI T, SZYMANSKI J, DUDEK B, et al. NLP questions answering using DBpedia and YAGO. Vietnam Journal of Computer Science, 2020, 7 (4): 339- 354. doi: 10.1142/S2196888820500190
6	BORDES A, CHOPRA S, WESTON J. Question answering with subgraph embeddings[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2014: 615-620.
7	WANG H W, ZHANG F Z, ZHANG M D, et al. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2019: 968-977.
8	WANG Q, MAO Z D, WANG B, et al. Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 2017, 29 (12): 2724- 2743. doi: 10.1109/TKDE.2017.2754499
9	WANG K, LIU Y, XU X J, et al. Enhancing knowledge graph embedding by composite neighbors for link prediction. Computing, 2020, 102 (12): 2587- 2606. doi: 10.1007/s00607-020-00842-5
10	BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2013: 2787-2795.
11	DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[EB/OL]. [2022-07-12]. https://arxiv.org/abs/1707.01476v6.
12	SUN Z Q, DENG Z H, NIE J Y, et al. RotatE: knowledge graph embedding by relational rotation in complex space[EB/OL]. [2022-07-12]. https://arxiv.org/abs/1902.10197.
13	WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2014: 1-10.
14	LIN Y K, LIU Z Y, SUN M S, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2015: 2181-2187.
15	JI G L, LIU K, HE S Z, et al. Knowledge graph completion with adaptive sparse transfer matrix[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 985-991.
16	NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning. New York, USA: ACM Press, 2011: 809-816.
17	NICKEL M, ROSASCO L, POGGIO T. Holographic embeddings of knowledge graphs[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2016: 1955-1961.
18	YANG B S, YIH W T, HE X D, et al. Embedding entities and relations for learning and inference in knowledge bases[EB/OL]. [2022-07-12]. https://arxiv.org/abs/1412.6575.
19	TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning. New York, USA: ACM Press, 2016: 2071-2080.
20	JIANG X, WANG Q, WANG B. Adaptive convolution for multi-relational learning[EB/OL]. [2022-07-12]. https://www.xueshufan.com/publication/2946476638.
21	NGUYEN D Q, NGUYEN T D. A novel embedding model for knowledge base completion based on convolutional neural network[EB/OL]. [2022-07-12]. https://arxiv.org/abs/1712.02121.
22	NGUYEN D Q, VU T, NGUYEN T D, et al. A capsule network-based embedding model for knowledge graph completion and search personalization[C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2019: 2180-2189.
23	付林, 刘钊, 邱晨, 等. 基于特征联合与多注意力的实体关系链接. 计算机工程, 2022, 48 (8): 53- 61. URL
	FU L, LIU Z, QIU C, et al. Entity relation linking based on feature joint and multi-attention. Computer Engineering, 2022, 48 (8): 53- 61. URL
24	TOUTANOVA K, CHEN D Q, PANTEL P, et al. Representing text for joint embedding of text and knowledge bases[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2015: 1499-1509.
25	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. [2022-07-12]. https://arxiv.org/abs/1710.10903.
26	ZHANG Z, ZHUANG F Z, ZHU H S, et al. Relational graph neural network with hierarchical attention for knowledge graph completion[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2020: 9612-9619.
27	WANG H W, REN H Y, LESKOVEC J. Relational message passing for knowledge graph completion[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2021: 1697-1707.
28	QIAO Z Y, NING Z Y, DU Y, et al. Context-enhanced entity and relation embedding for knowledge graph completion(student abstract)[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 15871-15872.
29	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2022-07-12]. https://arxiv.org/abs/1412.6980.
30	BENGIO Y, DUCHARME R, VINCENT P. A neural probabilistic language model[C]//Proceedings of the 13th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2000: 932-938.
31	ZHANG Z Q, CAI J Y, ZHANG Y D, et al. Learning hierarchy-aware knowledge graph embeddings for link prediction[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2020: 3065-3072.
32	陈恒, 王思懿, 李冠宇, 等. 基于四元数胶囊网络的知识图谱补全模型. 计算机工程, 2022, 48 (2): 40-46, 64 URL
	CHEN H, WANG S Y, LI G Y, et al. Knowledge graph completion model based on quaternion capsule network. Computer Engineering, 2022, 48 (2): 40-46, 64 URL

模型	评分函数	损失函数
TransE	$ {F}_{\mathrm{T}}=\mathrm{ }-‖{\boldsymbol{e}}_{\mathrm{h}}+\boldsymbol{r}-{\boldsymbol{e}}_{\mathrm{t}}‖ $	$ {L}_{\mathrm{T}}=\sum\limits_{t\in T}\sum\limits_{{t}^{\text{'}}\in {T}^{\text{'}}}\mathrm{m}\mathrm{a}\mathrm{x}(0, \gamma -F(t)+F({t}^{\text{'}}\left)\right) $
ConvE	$ {F}_{\mathrm{C}}=f\left(\mathrm{v}\mathrm{e}\mathrm{c}\right(f\left(\right[{\stackrel{-}{\boldsymbol{e}}}_{\mathrm{h}};\stackrel{-}{\boldsymbol{r}}\left]\mathrm{*}\omega \right)\left)\boldsymbol{W}\right){\boldsymbol{e}}_{\mathrm{t}} $	$ {L}_{\mathrm{C}}=\mathrm{ }-\frac{1}{N}\sum\limits_{t\in T}\ln F\left(t\right)+\sum\limits_{{t}^{\text{'}}\in {T}^{\text{'}}}\ln(1-F({t}^{\text{'}}\left)\right) $
RotatE	$ {F}_{\mathrm{R}}=-‖{\boldsymbol{e}}_{\mathrm{h}, \mathrm{c}}\circ {\boldsymbol{r}}_{\mathrm{c}}-{\boldsymbol{e}}_{\mathrm{t}, \mathrm{c}}‖ $	$ {L}_{\mathrm{R}}=\mathrm{ }-\sum\limits_{t\in T}\ln\sigma (\gamma -F(t\left)\right)-\frac{1}{{N}_{{T}^{\text{'}}}}\sum\limits_{{t}^{\text{'}}\in {T}^{\text{'}}}\ln\sigma \left(F\right({t}^{\text{'}})-\gamma \left.\right) $

模型	评分函数	损失函数
TransE	$ {F}_{\mathrm{T}}=\mathrm{ }-‖{\boldsymbol{e}}_{\mathrm{h}}+\boldsymbol{r}-{\boldsymbol{e}}_{\mathrm{t}}‖ $	$ {L}_{\mathrm{T}}=\sum\limits_{t\in T}\sum\limits_{{t}^{\text{'}}\in {T}^{\text{'}}}\mathrm{m}\mathrm{a}\mathrm{x}(0, \gamma -F(t)+F({t}^{\text{'}}\left)\right) $
ConvE	$ {F}_{\mathrm{C}}=f\left(\mathrm{v}\mathrm{e}\mathrm{c}\right(f\left(\right[{\stackrel{-}{\boldsymbol{e}}}_{\mathrm{h}};\stackrel{-}{\boldsymbol{r}}\left]\mathrm{*}\omega \right)\left)\boldsymbol{W}\right){\boldsymbol{e}}_{\mathrm{t}} $	$ {L}_{\mathrm{C}}=\mathrm{ }-\frac{1}{N}\sum\limits_{t\in T}\ln F\left(t\right)+\sum\limits_{{t}^{\text{'}}\in {T}^{\text{'}}}\ln(1-F({t}^{\text{'}}\left)\right) $
RotatE	$ {F}_{\mathrm{R}}=-‖{\boldsymbol{e}}_{\mathrm{h}, \mathrm{c}}\circ {\boldsymbol{r}}_{\mathrm{c}}-{\boldsymbol{e}}_{\mathrm{t}, \mathrm{c}}‖ $	$ {L}_{\mathrm{R}}=\mathrm{ }-\sum\limits_{t\in T}\ln\sigma (\gamma -F(t\left)\right)-\frac{1}{{N}_{{T}^{\text{'}}}}\sum\limits_{{t}^{\text{'}}\in {T}^{\text{'}}}\ln\sigma \left(F\right({t}^{\text{'}})-\gamma \left.\right) $

数据集	#Relation	#Entity	#Train	#Valid	#Test
FB15k-237	237	14 541	272 115	17 535	20 466
WN18RR	11	40 943	86 835	3 034	3 134

数据集	#Relation	#Entity	#Train	#Valid	#Test
FB15k-237	237	14 541	272 115	17 535	20 466
WN18RR	11	40 943	86 835	3 034	3 134

模型	FB15k-237				WN18RR
模型	MR	MRR	Hits@10/%	Hits@1/%	MR	MRR	Hits@10/%	Hits@1/%
TransE	221	24.2	42.4	16.6	379 3	18.0	45.7	1.3
DistMult	309	22.4	38.3	14.6	511 0	43.0	49.0	39.0
ComplEx	289	22.7	39.1	14.5	526 1	44.0	51.0	41.0
ANALOGY	309	22.7	38.7	14.8	898 2	39.2	41.0	38.4
ConvE	244	31.6	50.1	23.7	418 7	43.0	52.0	40.0
RotatE	177	33.8	53.3	24.1	334 0	47.6	57.1	42.8
CoNE-TransE	211	30.6	48.5	21.7	354 2	22.7	50.2	6.4
CoNE-ConvE	187	33.4	51.5	24.6	416 5	45.5	52.7	42.3
CoNE-RotatE	147	35.0	53.6	25.8	277 6	49.4	58.2	45.1
DAERC-TransE	167	32.2	50.9	22.9	232 4	28.1	53.8	11.3
DAERC-ConvE	179	33.8	52.6	24.9	415 4	46.0	53.9	43.0
DAERC-RotatE	162	35.4	54.5	26.2	275 2	50.5	59.5	46.1

选择文件类型/文献管理软件名称

选择包含的内容