融合实体邻域信息的知识图谱嵌入负采样方法

doi:10.19678/j.issn.1000-3428.0064167

摘要/Abstract

摘要： 知识图谱嵌入的主要任务是将实体与关系嵌入低维、连续的向量空间。在模型训练过程中，必须同时提供正负三元组。已有的负采样方法多使用均匀随机采样方法构造负样本，通过这种方式获得的负样本对于模型的训练贡献很小。基于生成对抗网络，生成器能够采样更多可信的负三元组，增强嵌入模型性能。然而，离散数据在使用遗传算法时存在梯度消失的问题。针对以上问题，提出一种融合实体邻域信息的知识图谱嵌入负采样方法。该方法基于生成对抗网络的框架，通过图卷积神经网络聚合实体在不同关系路径上的邻域信息，用以辅助生成器产生高质量的负样本，提高鉴别器的性能。同时，在鉴别器部分引入Wasserstein距离代替传统的散度，解决梯度消失问题，加速模型收敛。在链接预测任务和三元组分类任务上对所提方法的有效性进行验证，结果表明，该方法在链接预测任务中MR、MRR、Hits@10较基线模型分别平均提升4.18、9.19、10.18个百分点，在三元组分类任务中准确率平均提升4.50个百分点，充分证明实体邻域信息的融入能够进一步提升负样本质量，显著提升模型性能。

关键词: 知识图谱嵌入, 生成对抗网络, 邻域信息, 图卷积神经网络, Wasserstein距离

Abstract: Knowledge Graph Embedding(KGE) embeds entities and relations into low-dimensional and continuous vector space.During model training, both positive and negative triples must be provided.Most of the existing negative sampling methods use uniform random sampling to construct negative samples, which have little contribution toward the training of the model.Inspired by Generative Adversarial Network(GAN), the generator can sample more plausible negative triples, which enhances the embedding model performance.However, discrete data exhibit vanishing gradients when using genetic algorithms.Therefore, this paper proposes a KGE negative sampling method fused with entity neighborhood information and uses the graph CNN to aggregate the neighborhood information of entities on different relation paths to generate high-quality negative samples and improve the performance of the discriminator.The Wasserstein distance is introduced to replace the traditional divergence to solve the gradient disappearance problem and accelerate the model convergence.Furthermore, the proposed method is evaluated on the link prediction task and triplet classification task.The results show that MR, MRR, and Hits@10 obtained by the proposed method are better compared to other baseline models in the link prediction task, with an average improvement of 4.18, 9.19, and 10.18 percentage points, respectively.The accuracy rate in the triplet classification task increased by 4.50 percentage points on average, thereby confirming that the integration of entity neighborhood information can improve the quality of negative samples and the model performance.

Key words: Knowledge Graph Embedding(KGE), Generative Adversarial Network(GAN), neighborhood information, graph Convolution Neural Network(CNN), Wasserstein distance

中图分类号:

TP18

翟社平, 张宇航, 柏晓夏. 融合实体邻域信息的知识图谱嵌入负采样方法[J]. 计算机工程, 2023, 49(3): 95-104.

ZHAI Sheping, ZHANG Yuhang, BAI Xiaoxia. Knowledge Graph Embedding Negative Sampling Method Fused with Entity Neighborhood Information[J]. Computer Engineering, 2023, 49(3): 95-104.

http://www.ecice06.com/CN/Y2023/V49/I3/95

参考文献

[1] CHEN X, JIA S, XIANG Y.A review:knowledge reasoning over knowledge graph[J].Expert Systems with Applications, 2020, 141(5):1-21.
[2] SUCHANEK F M, KASNECI G, WEIKUM G.YAGO:a large ontology from Wikipedia and WordNet[J].Journal of Web Semantics, 2008, 6(3):203-217.
[3] BOLLACKER K, EVANS C, PARITOSH P, et al.Freebase:a collaboratively created graph database for structuring human knowledge[C]//Proceedings of 2008 ACM SIGMOD International Conference on Management of Data.New York, USA:ACM Press, 2008:1247-1250.
[4] AMILLER G.WordNet:a lexical database for English[J].Communications of the ACM, 1995, 38(11):39-41.
[5] LI F Y, LI Y, SHANG C J, et al.Fuzzy knowledge-based prediction through weighted rule interpolation[J].IEEE Transactions on Cybernetics, 2020, 50(10):4508-4517.
[6] WU C, LIU S, ZENG Z Y, et al.Knowledge graph-based multi-context-aware recommendation algorithm[J].Information Sciences, 2022, 595:179-194.
[7] 刘知远, 孙茂松, 林衍凯, 等.知识表示学习研究进展[J].计算机研究与发展, 2016, 53(2):247-261. LIU Z Y, SUN M S, LIN Y K, et al.Knowledge representation learning:a review[J].Journal of Computer Research and Development, 2016, 53(2):247-261.(in Chinese)
[8] NEELAKANTAN A, ROTH B, MCCALLUM A.Compositional vector space models for knowledge base completion[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2015:156-166.
[9] NGUYEN D Q, NGUYEN T D, NGUYEN D Q, et al.A novel embedding model for knowledge base completion based on convolutional neural network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 2(Short Papers). Stroudsburg, USA:Association for Computational Linguistics, 2018:327-333.
[10] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2018:1811-1818.
[11] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge, USA:MIT Press, 2014:2672-2680.
[12] WANG P F, LI S Y, PAN R.Incorporating GAN for negative sampling in knowledge representation learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2018:1-10.
[13] CAI L W, WANG W Y.KBGAN:adversarial learning for knowledge graph embeddings[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long Papers). Stroudsburg, USA:Association for Computational Linguistics, 2018:1470-1480.
[14] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of NIPS'13.Cambridge, USA:MIT Press, 2013:2787-2795.
[15] LIN Y, LIU Z, LUAN H, et al.Modeling relation paths for representation learning of knowledge bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, USA:Association for Computational Linguistics, 2015:705-714.
[16] MIKOLOV T, SUTSKEVER I, CHEN K, et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Nedava, USA:NIPS Press, 2013:3111-3119.
[17] WANG Z, ZHANG J, FENG J, et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA:AAAI Press, 2014:1112-1119.
[18] LIN Y, LIU Z, SUN M, et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2015:2181-2187.
[19] JI G L, HE S Z, XU L H, et al.Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Volume 1(Long Papers). Stroudsburg, USA:Association for Computational Linguistics, 2015:687-696.
[20] JIA Y, WANG Y, LIN H, et al.Locally adaptive translation for knowledge graph embedding[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2016:992-998.
[21] HE S Z, LIU K, JI G L, et al.Learning to represent knowledge graphs with Gaussian embedding[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management.New York, USA:ACM Press, 2015:623-632.
[22] XIAO H, HUANG M L, ZHU X Y.TransG:a generative model for knowledge graph embedding[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Volume 1(Long Papers). Stroudsburg, USA:Association for Computational Linguistics, 2016:2316-2325.
[23] NICKEL M, TRESP V, KRIEGEL H P.A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on International Conference on Machine Learning.New York, USA:ICML Press, 2011:809-816.
[24] YANG B, YIH S W, HE X, et al.Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of International Conference on Learning Representations.San Diego, USA:ICLR Press, 2015:12-24.
[25] TROUILLON T, WELBL J, RIEDEL S, et al.Complex embeddings for simple link prediction[C]//Proceedings of International Conference on Machine Learning.New York, USA:ICML Press, 2016:2071-2080.
[26] LIU Q, JIANG H, EVDOKIMOV A, et al.Probabilistic reasoning via deep learning:neural association models[EB/OL].[2022-02-10].https://arxiv.org/abs/1603.07704.
[27] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA:MIT Press, 2017:6000-6010.
[28] WU Z H, PAN S R, CHEN F W, et al.A comprehensive survey on graph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1):4-24.
[29] BA J L, KIROS J R, HINTON G E.Layer normalization[EB/OL].[2022-02-10].https://arxiv.org/abs/1607.06450.
[30] ARJOVSKY M, CHINTALA S, BOTTOU L.Wasserstein generative adversarial networks[C]//Proceedings of International Conference on Machine Learning.New York, USA:ICML Press, 2017:214-223.
[31] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of Wasserstein GANs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York, USA:ACM Press:2017:5769-5779.
[32] TOUTANOVA K, CHEN D Q.Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality.Stroudsburg, USA:Association for Computational Linguistics, 2015:57-66.
[33] AN B, CHEN B, HAN X P, et al.Accurate text-enhanced knowledge graph representation learning[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1(Long Papers). Stroudsburg, USA:Association for Computational Linguistics, 2018:745-755.
[34] SUN Z, DENG Z H, NIE J Y, et al.RotatE:knowledge graph embedding by relational rotation in complex space[C]//Proceedings of International Conference on Learning Representations.San Diego, USA:ICLR Press, 2019:1-18.
[35] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of European Semantic Web Conference. Berlin, Germany:Springer, 2018:593-607.
[36] BANSAL T, JUAN D C, RAVI S, et al.A2N:attending to neighbors for knowledge graph inference[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2019:4387-4392.
[37] VASHISHTH S, SANYAL S, NITIN V, et al.Composition-based multi-relational graph convolutional networks[C]//Proceedings of International Conference on Learning Representations.San Diego, USA:ICLR Press, 2019:1-19.
[38] KINGMA D P, BA J.Adam:a method for stochastic optimization[C]//Proceedings of International Conference on Learning Representations.San Diego, USA:ICLR Press, 2015:1-15.

选择文件类型/文献管理软件名称

选择包含的内容