Knowledge Graph Completion Based on Logical Rules and Graph Neural Network

doi:10.19678/j.issn.1000-3428.0069129

Abstract

Abstract:

Knowledge Graph Completion(KGC) aims to utilize the existing knowledge in a Knowledge Graph(KG) to derive new facts. This process is pivotal in several tasks and domains and has garnered increasing attention from researchers. However, most existing KGC methods focus on modeling fact triples in a KG without fully considering deep semantics or associations between entities and relationships in the KG. To address this issue, logical rules can be used to reflect the implicit associations between relationships in a KG. The semantic nature of the KG implies that higher-order neighborhoods around fact triples contain deep semantic information. Therefore, to fully explore the inherent semantics and correlations of entities and relationships in KGs, this study proposes a novel model for a KGC based on logical rules and Graph Neural Networks(GNNs). The framework first employs an automatic rule learning process based on the efficient Expectation-Maximization(EM) iterative optimization algorithm. It then performs joint embedding training on the obtained high-quality logical rules, entities, and relationships in the KG to model the complex relationship patterns in the KG and improve the generalization of the embedding representation. Subsequently, attention embedding propagation is performed by simultaneously considering the importance of logic rules and triples to aggregate higher-order neighborhood information, and entity and relation embedding representations incorporating deep semantics and associations are obtained for the KGC. In this study, extensive experiments are conducted on four public datasets for link prediction, and the results demonstrate the effectiveness of the proposed model.

Key words: Knowledge Graph(KG), Knowledge Graph Completion(KGC), logical rules, Graph Neural Network(GNN), link prediction

摘要：

知识图谱补全旨在利用知识图谱中已有的知识推导出新的事实, 在多个任务和领域发挥重要作用, 引起研究者越来越多的关注。然而现有的知识图谱补全方法大多专注于对知识图谱中的事实三元组进行建模, 未充分考虑知识图谱中实体和关系之间所蕴涵的深层语义和关联。为了解决这个问题, 使用逻辑规则反映知识图谱中关系之间的隐含关联, 而知识图谱语义网络的本质决定了事实三元组周围高阶邻域中包含着深层语义信息。因此, 为了挖掘知识图谱中实体和关系的内在语义和关联, 提出一种基于逻辑规则和图神经网络(GNN)进行知识图谱补全模型。首先基于高效期望最大化(EM)迭代优化算法进行规则自动学习, 将得到的高质量逻辑规则与知识图谱中的实体和关系进行联合嵌入训练, 以实现对知识图谱中复杂关系模式的建模, 并提高嵌入表示的泛化性。同时, 考虑逻辑规则和三元组的重要性进行注意力嵌入传播, 以聚合高阶邻域信息, 最终得到融合深层语义和关联的实体、关系嵌入表示用于知识图谱补全。在4个公开数据集上针对链接预测任务进行实验, 实验结果证明了所提出模型的有效性。

关键词: 知识图谱, 知识图谱补全, 逻辑规则, 图神经网络, 链接预测

LIU Chunyu, CHEN Qingfeng, MO Shaocong, XIE Ze. Knowledge Graph Completion Based on Logical Rules and Graph Neural Network[J]. Computer Engineering, 2025, 51(3): 131-143.

刘春雨, 陈庆锋, 莫少聪, 谢泽. 基于逻辑规则和图神经网络的知识图谱补全[J]. 计算机工程, 2025, 51(3): 131-143.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069129

https://www.ecice06.com/EN/Y2025/V51/I3/131

Figures/Tables 10

Fig.1 The overall framework of the SLRGNN model

Fig.2 Schematic diagram of knowledge graph completion

Fig.3 Experimental results of parameters on the FB15k-237 dataset

Fig.4 Experimental results of parameters on the WN18RR dataset

Fig.5 The impact of knowledge graph sparsity on MRR

References 59

1	SUCHANEK F M, KASNECI G, WEIKUM G. Yago: a core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web. New York, USA: ACM Press, 2007: 697-706.
2	BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York, USA: ACM Press, 2008: 1247-1250.
3	AUER S , BIZER C , KOBILAROV G , et al. DBpedia: a nucleus for a Web of open data. Berlin, Germany: Springer, 2007.
4	VRANDEČIĆ D , KRÖTZSCH M . Wikidata. Communications of the ACM, 2014, 57 (10): 78- 85. doi: 10.1145/2629489
5	CARLSON A , BETTERIDGE J , KISIEL B , et al. Toward an architecture for never-ending language learning. Artificial Intelligence, 2010, 24 (1): 1306- 1313. URL
6	ALHUSSIEN I, CAMBRIA E, ZHANG N S. Semantically enhanced models for commonsense knowledge acquisition[C]//Proceedings of IEEE International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2018: 1014-1021.
7	王萌, 王昊奋, 李博涵, 等. 新一代知识图谱关键技术综述. 计算机研究与发展, 2022, 59 (9): 1947- 1965.
	WANG M , WANG H F , LI B H , et al. Survey on key technologies of new generation knowledge graph. Journal of Computer Research and Development, 2022, 59 (9): 1947- 1965.
8	EHRLINGER L, WÖSS W. Towards a definition of knowledge graphs[C]//Proceedings of the 12th International Conference on Semantic Systems. Washington D. C., USA: IEEE Press, 2016: 456-467.
9	ZOU Y Y , FININ T , CHEN H . F-OWL: an inference engine for semantic Web. Berlin, Germany: Springer, 2004.
10	BVHMANN L , LEHMANN J . Pattern based knowledge base enrichment. Berlin, Germany: Springer, 2013.
11	JIANG S P, LOWD D, DOU D J. Learning to refine an automatically extracted knowledge base using Markov logic[C]//Proceedings of the 12th IEEE International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2012: 912-917.
12	GALÁRRAGA L A, TEFLIOUDI C, HOSE K, et al. AMIE: association rule mining under incomplete evidence in ontological knowledge bases[C]//Proceedings of the 22nd International Conference on World Wide Web. Washington D. C., USA: IEEE Press, 2013: 413-422.
13	GALÁRRAGA L , TEFLIOUDI C , HOSE K , et al. Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal, 2015, 24 (6): 707- 730. doi: 10.1007/s00778-015-0394-1
14	KIMMIG A, BACH S, BROECHELER M, et al. A short introduction to probabilistic soft logic[C]//Proceedings of NIPS'12. Cambridge, USA: MIT Press, 2012: 1-4.
15	QU M, CHEN J, XHONNEUX L P, et al. RNNLogic: learning logic rules for reasoning on knowledge graphs[C]//Proceedings of 2021 International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2021: 762-775.
16	GARDNER M, TALUKDAR P P, KISIEL B, et al. Improving learning and inference in a large knowledge-base using latent syntactic cues[C]//Proceedings of 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2013: 833-838.
17	GARDNER M, MITCHELL T. Efficient and expressive knowledge base completion using subgraph feature extraction[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2015: 1488-1498.
18	MEILICKE C, CHEKOL M, FINK M, et al. Reinforced anytime bottom up rule learning for knowledge graph completion[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/2004.04412.
19	BORDES A, USUNIER N, GARCIA-DURÁN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2013: 246-257.
20	WANG Z , ZHANG J W , FENG J L , et al. Knowledge graph embedding by translating on hyperplanes. Artificial Intelligence, 2014, 28 (1): 468- 477. URL
21	SUN Z, DENG Z H, NIE J Y, et al. RotatE: knowledge graph embedding by relational rotation in complex space[C]//Proceedings of the 7th International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2019: 332-345.
22	ZHANG S, TAY Y, YAO L, et al. Quaternion knowledge graph embeddings[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 323-335.
23	NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2011: 809-816.
24	YANG B S, YIH W T, HE X D, et al. Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of the 3rd International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2015: 532-541.
25	TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2016: 3021-3032.
26	BALAZEVIC I, ALLEN C, HOSPEDALES T M. TuckER: tensor factorization for knowledge graph completion[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/1901.09590.
27	DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/1707.01476.
28	NGUYEN D Q, VU T, NGUYEN T D, et al. A capsule network-based embedding model for knowledge graph completion and search personalization[C]//Proceedings of NAACL'19. Stroudsburg, USA: Association for Computational Linguistics, 2019: 2180-2189.
29	VASHISHTH S , SANYAL S , NITIN V , et al. InteractE: improving convolution-based knowledge graph embeddings by increasing feature interactions. Artificial Intelligence, 2020, 34 (3): 3009- 3016. URL
30	DAS R, NEELAKANTAN A, BELANGER D, et al. Chains of reasoning over entities, relations, and text using recurrent neural networks[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2017: 132-141.
31	GUO L, SUN Z, HU W. Learning to exploit long-term relational dependencies in knowledge graphs[C]//Proceedings of the 36th International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2019: 2505-2514.
32	SCHLICHTKRULL M , KIPF T N , BLOEM P , et al. Modeling relational data with graph convolutional networks. Berlin, Germany: Springer, 2018: 593- 607.
33	SHANG C , TANG Y , HUANG J , et al. End-to-end structure-aware convolutional networks for knowledge base completion. Artificial Intelligence, 2019, 33 (1): 3060- 3067. URL
34	VASHISHTH S, SANYAL S, NITIN V, et al. Composition-based multi-relational graph convolutional networks[C]//Proceedings of International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2020: 682-693.
35	YAO L, MAO C, LUO Y. KG-BERT: Bert for knowledge graph completion[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/1909.03193.
36	KIM B, HONG T, KO Y, et al. Multi-task learning for knowledge graph completion with pre-trained language models[C]//Proceedings of the 28th International Conference on Computational Linguistics. Stroudsburg, USA: International Committee on Computational Linguistics, 2020: 1737-1743.
37	LI D, YI M, HE Y. Lp-Bert: multi-task pre-training knowledge graph Bert for link prediction[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/2201.04843.
38	XIONG W H, HOANG T, WANG W Y. DeepPath: a reinforcement learning method for knowledge graph reasoning[C]//Proceedings of International Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2017: 332-345.
39	DAS R, DHULIAWALA S, ZAHEER M, et al. Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/1711.05851.
40	LIN X V, SOCHER R, XIONG C M. Multi-hop knowledge graph reasoning with reward shaping[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2018: 3243-3253.
41	MEILICKE C , FINK M , WANG Y J , et al. Fine-grained evaluation of rule- and embedding-based systems for knowledge graph completion. Berlin, Germany: Springer, 2018.
42	GUO S, WANG Q, WANG L H, et al. Jointly embedding knowledge graphs and logical rules[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2016: 192-202.
43	GUO S , WANG Q , WANG L H , et al. Knowledge graph embedding with iterative guidance from soft rules. Artificial Intelligence, 2018, 32 (1): 335- 346. URL
44	ZHANG W, PAUDEL B, WANG L, et al. Iteratively learning embeddings and rules for knowledge graph reasoning[C]//Proceedings of World Wide Web Conference. New York, USA: ACM Press, 2019: 2366-2377.
45	QU M, TANG J. Probabilistic logic neural networks for reasoning[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 542-551.
46	EVANS R , GREFENSTETTE E . Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 2018, 61, 51- 64. URL
47	YANG F, YANG Z, COHEN W W. Differentiable learning of logical rules for knowledge base reasoning[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 465-476.
48	TANG X, ZHU S C, LIANG Y, et al. RulE: neural-symbolic knowledge graph reasoning with rule embedding[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/2210.14905.
49	WEI Z Y, ZHAO J, LIU K, et al. Large-scale knowledge base completion: inferring via grounding network sampling over selected instances[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. New York, USA: ACM Press, 2015: 1331-1340.
50	ROCKTÄSCHEL T, SINGH S, RIEDEL S. Injecting logical background knowledge into embeddings for relation extraction[C]//Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, USA: Association for Computational Linguistics, 2015: 1119-1129.
51	WANG P, DOU D, WU F Z, et al. Logic rules powered knowledge graph embedding[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/1903.03772.
52	HU Z T, MA X Z, LIU Z Z, et al. Harnessing deep neural networks with logic rules[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2016: 2410-2420.
53	TOUTANOVA K, CHEN D Q. Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. Stroudsburg, USA: Association for Computational Linguistics, 2015: 57-66.
54	KOK S, DOMINGOS P. Statistical predicate invention[C]//Proceedings of the 24th International Conference on Machine Learning. New York, USA: ACM Press, 2007: 433-440.
55	MILLER G A . WordNet. Communications of the ACM, 1995, 38 (11): 39- 41. doi: 10.1145/219717.219748
56	LAO N , COHEN W W . Relational retrieval using a combination of path-constrained random walks. Machine Learning, 2010, 81 (1): 53- 67. doi: 10.1007/s10994-010-5205-8
57	CHENG K W, LIU J H, WANG W, et al. RLogic: recursive logical rule learning from knowledge graphs[EB/OL]. [2023-11-20]. https://dl.acm.org/doi/pdf/10.1145/3534678.3539421.
58	SADEGHIAN A, ARMANDPOUR M, DING P, et al. DRUM: end-to-end differentiable rule mining on knowledge graphs[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 135-143.
59	CHENG K W, AHMED N, SUN Y Z. Neural compositional rule learning for knowledge graph reasoning[EB/OL]. [2023-11-20]. https://arxiv.org/pdf/2303.03581.

[1]	CAI Ruichu, XU Zunhong, CHEN Daoxin, YANG Zhenhui, LI Zijian, HAO Zhifeng. Causal Mechanism-Based Molecular Property Prediction [J]. Computer Engineering, 2025, 51(3): 105-112.
[2]	ZHANG Heping, ZHANG Hegui, XIE Xiaoyao, ZHANG Taihua, ZHANG Sicong, YU Guojun. Network Embedding Based on k-core Decomposition [J]. Computer Engineering, 2025, 51(2): 139-148.
[3]	MA Hengzhi, QIAN Yurong, LENG Hongyong, WU Haipeng, TAO Wenbin, ZHANG Yiyang. Review of Research Progress on Knowledge Graph Embedding [J]. Computer Engineering, 2025, 51(2): 18-34.
[4]	LI Zelin, LÜ Zhaofeng, CHEN Fuqiang, LI Ke. Entity Alignment Model Based on Multi-Hop Information Fusion [J]. Computer Engineering, 2024, 50(9): 142-152.
[5]	TANG Zhikang, WU Yuqi, LI Chunying, TANG Yong. Recommendation of Learning Resource Based on Knowledge Graph Convolutional Network [J]. Computer Engineering, 2024, 50(9): 153-160.
[6]	Huayu LI, Zhikang ZHANG, Yang YAN, Yang YUE. Enhanced Domain Multi-modal Entity Recognition Based on Knowledge Graph [J]. Computer Engineering, 2024, 50(8): 31-39.
[7]	Xingyu HE, Yixin ZHOU, Dongxu LUO, Guisong YANG. Education Resource Recommendation Based on Graph Neural Network and Multi-Subject Rating [J]. Computer Engineering, 2024, 50(7): 13-22.
[8]	Juan LIU, Youxiang DUAN, Yuxi LU, Lu ZHANG. Knowledge Graph Completion with Knowledge Enhancement and Contrastive Learning [J]. Computer Engineering, 2024, 50(7): 112-122.
[9]	CHEN Jiayu, WANG Yuanlong, ZHANG Hu. Question Generation Model Based on Text Knowledge Enhancement [J]. Computer Engineering, 2024, 50(6): 86-93.
[10]	ZHANG Baoxin, YANG Dan, NIE Tiezheng, KOU Yue. Recommendation Method Based on Self-supervised Multi-view Graph Collaborative Filtering [J]. Computer Engineering, 2024, 50(5): 100-110.
[11]	YOU Ben, LI Xiaohong, YAO Jin, FENG Shaojie. Semi-supervised Classification for Short Text Based on Multi-grained Graphs and Attention Mechanism [J]. Computer Engineering, 2024, 50(5): 83-90.
[12]	SUN Wenjie, LI Zongmin, SUN Haomiao. Multi-Agent Reinforcement Learning Value Function Factorization Approach Based on Graph Neural Network [J]. Computer Engineering, 2024, 50(5): 62-70.
[13]	ZHANG Hongchen, LI Linyu, YANG Li, SAN Chenjun, YIN Chunlin, YAN Bing, YU Hong, ZHANG Xuan. Knowledge Graph Completion Based on Contrastive Learning and Language Model-Enhanced Embedding [J]. Computer Engineering, 2024, 50(4): 168-176.
[14]	Jun WANG, Huixia LAI, Yue WAN, Shi ZHANG. Angle-based Graph Neural Network Method for Anomaly Detection in High Dimensional Data [J]. Computer Engineering, 2024, 50(3): 156-165.
[15]	Bohan WANG, Xiaoyan JIANG, Liuyi FAN. Semantic Segmentation Improvement Method Based on Deep Supervision for the Construction of Latent Space [J]. Computer Engineering, 2024, 50(3): 191-199.

Please choose a citation manager

Content to export