基于位置和注意力联合表示的知识图谱问答

doi:10.19678/j.issn.1000-3428.0062103

摘要/Abstract

摘要： 知识图谱是人工智能的重要组成部分，其以结构化的方式描述客观世界中的概念、实体及关系，提供了一种更优的组织、管理和理解互联网海量信息的能力。随着深度学习技术的发展，基于表示学习的知识图谱问答方法陆续出现。利用表示学习的方法实现知识图谱问答的核心目标是将问题嵌入到与三元组相同维度的表示向量空间中，通过合适的答案预测方法来匹配问题与答案。参考复数域编码的思路，构建一种基于位置和注意力联合表示的三元组表示模型Pos-Att-complex。在三元组表示部分，将词本身的特征和位置特征联合编码，并通过解码器网络进一步挖掘深层次特征，从而对三元组进行打分。在知识图谱问答部分，将问题通过RoBERTa嵌入到与三元组向量相同维度的向量空间中，并与通过关系筛选的关系集合进行向量融合。在此基础上，通过联合表示解码器为候选答案打分，以筛选出问题的答案。实验结果表明，该模型在三元组分类和多跳问答基准数据集上均能取得良好的测试结果，准确率优于GraftNet、VRN等模型。

关键词: 表示学习, 知识图谱问答, 复数域编码, 联合表示, 向量融合

Abstract: Knowledge graph is an important part of artificial intelligence.It describes the concepts, entities, and relationships in the objective world in a structured way and provides a better ability to organize, manage, and understand the massive amount of information available on the Internet.With the development of deep learning technology, representation-learning-based knowledge graph question-answering methods have emerged.The core goal of such methods is to embed the question into the representation vector space with the same dimension as triples and match the questions and answers through appropriate answer prediction methods.Referring to the idea of complex field coding, this paper presents a triple-representation model, Pos-Att-complex, based on joint location and attention representation.In the triplet representation part, the features of the word itself and the location features are jointly encoded.The deep-seated features are further mined through the decoder network, so as to score the triplet.In the question and answer part of the knowledge graph, the question is embedded into the vector space with the same dimension as the triplet vector through RoBERTa, and the vector is fused with the relationship set filtered through the relationship.On this basis, the candidate answers are scored by the joint representation decoder to screen them out.Experimental results show that the model can achieve good test results on triple classification and multi hop question and answer benchmark datasets.Furthermore, it outperforms GraftNet, VRN, and other existing models.

Key words: representation learning, knowledge graph question-answering, complex field coding, joint representation, vector fusion

中图分类号:

TP391

吴天波, 周欣, 程军军, 朱晗, 何小海. 基于位置和注意力联合表示的知识图谱问答[J]. 计算机工程, 2022, 48(8): 98-104,112.

WU Tianbo, ZHOU Xin, CHENG Junjun, ZHU Han, HE Xiaohai. Knowledge Graph Question-Answering Based on Joint Location and Attention Representation[J]. Computer Engineering, 2022, 48(8): 98-104,112.

http://www.ecice06.com/CN/Y2022/V48/I8/98

图/表 13

20220825091434

20220825091438

20220825091441

20220825091445

20220825091449

20220825091453

20220825091504

20220825091508

20220825091512

20220825091516

20220825091519

20220825091523

20220825091527

参考文献

[1] BORDES A, USUNIER N, GARCIA-DURÁN A, et al.Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2013:2787-2795.
[2] WANG Z, ZHANG J, FENG J, et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2014:1112-1119.
[3] LIN Y, LIU Z, SUN M, et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2015:2181-2187.
[4] DETTMERS T, MINERVINI P, STENETORP P, et al.Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2018:1811-1818.
[5] MALLEA M D G, MELTZER P, BENTLEY P J.Capsule neural networks for graph classification using explicit tensorial graph representations[EB/OL].[2021-06-05].https://arxiv.org/abs/1902.08399.
[6] LIU W J, ZHOU P, ZHAO Z, et al.K-BERT:enabling language representation with knowledge graph[J].Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(3):2901-2908.
[7] DONG L, WEI F R, ZHOU M, et al.Question answering over freebase with multi-column convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2015:260-269.
[8] YIH W T, CHANG M W, HE X D, et al.Semantic parsing via staged query graph generation:question answering with knowledge base[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2015:1321-1331.
[9] 陈文杰, 文奕, 张鑫, 等.一种改进的基于TransE知识图谱表示方法[J].计算机工程, 2020, 46(5):63-69, 77. CHEN W J, WEN Y, ZHANG X, et al.An improved TransE-based method for knowledge graph representation[J].Computer Engineering, 2020, 46(5):63-69, 77.(in Chinese)
[10] DAI Z H, LI L, XU W.CFO:conditional focused neural question answering with large-scale knowledge bases[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2016:800-810.
[11] SUN H T, BEDRAX-WEISS T, COHEN W.PullNet:open domain question answering with iterative retrieval on knowledge bases and text[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2019:2380-2390.
[12] LAN Y S, JIANG J.Query graph generation for answering multi-hop complex questions from knowledge bases[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2020:969-974.
[13] SAXENA A, TRIPATHI A, TALUKDAR P.Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2020:4498-4507.
[14] 金婧, 万怀宇, 林友芳.融合实体类别信息的知识图谱表示学习[J].计算机工程, 2021, 47(4):77-83. JIN J, WAN H Y, LIN Y F.Knowledge graph representation learning fused with entity category information[J].Computer Engineering, 2021, 47(4):77-83.(in Chinese)
[15] MIKOLOV T, CHEN K, CORRADO G, et al.Efficient estimation of word representations in vector space[EB/OL].[2021-06-05].https://openreview.net/pdf?id=idpCdOWtqXd60.
[16] TROUILLON T, WELBL J, RIEDEL S, et al.Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning.New York, USA:ACM Press, 2016:2071-2080.
[17] SUN Z, DENG Z H, NIE J Y, et al.RotatE:knowledge graph embedding by relational rotation in complex space[EB/OL].[2021-06-05].https://openreview.net/pdf?id=HkgEQnRqYQ.
[18] WANG B, ZHAO D, LIOMA C, et al.Encoding word order in complex embeddings[EB/OL].[2021-06-05].https://openreview.net/pdf?id=Hke-WTVtwr.
[19] SANTORO A, FAULKNER R, RAPOSO D, et al.Relational recurrent neural networks[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.[S.l.]:Curran Associates Inc., 2018:7310-7321.
[20] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[EB/OL].[2021-06-05].https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
[21] VEIT A, WILBER M, BELONGIE S.Residual networks behave like ensembles of relatively shallow networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2016:550-558.
[22] GLOROT X, BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the 13th International Conference on Artificial Intelligence and Statistics.Washington D.C., USA:IEEE Press, 2010:249-256.
[23] LIU Y H, OTT M, GOYAL N, et al.RoBERTa:a robustly optimized BERT pretraining approach[EB/OL].[2021-06-05].https://arxiv.org/abs/1907.11692.
[24] BORDES A, WESTON J, COLLOBERT R, et al.Learning structured embeddings of knowledge bases[C]//Proceedings of the 25th AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2011:301-306.
[25] ZHANG Y Y, DAI H J, KOZAREVA Z, et al.Variational reasoning for question answering with knowledge graph[EB/OL].[2021-06-05].https://arxiv.org/abs/1709.04071.
[26] KINGMA D P, BA J.Adam:a method for stochastic optimization[EB/OL].[2021-06-05].https://openreview.net/pdf?id=8gmWwjFyLj.
[27] PENNINGTON J, SOCHER R, MANNING C.Glove:global vectors for word representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2014:1532-1543.
[28] JI G L, HE S Z, XU L H, et al.Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2015:687-696.
[29] JI G, LIU K, HE S, et al.Knowledge graph completion with adaptive sparse transfer matrix[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2016:985-991.
[30] QIAN W, FU C, ZHU Y, et al.Translating embeddings for knowledge graph completion with relation attention mechanism[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence.Washington D.C., USA:IEEE Press, 2018:4286-4292.
[31] NGUYEN D Q, NGUYEN T, PHUNG D.A relational memory-based embedding model for triple classification and search personalization[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2020:3429-3435.
[32] TROUILLON T, WELBL J, RIEDEL S, et al.Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning.New York, USA:ACM Press, 2016:2071-2080.
[33] MILLER A, FISCH A, DODGE J, et al.Key-value memory networks for directly reading documents[C]//Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2016:1400-1409.
[34] SUN H T, DHINGRA B, ZAHEER M, et al.Open domain question answering using early fusion of knowledge bases and text[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing.[S.l.]:Association for Computational Linguistics, 2018:4231-4242.
[35] ZHANG Y Y, DAI H J, KOZAREVA Z, et al.Variational reasoning for question answering with knowledge graph[EB/OL].[2021-06-05].https://arxiv.org/abs/1709.04071.
[36] SAXENA A, TRIPATHI A, TALUKDAR P.Improving multi-hop question answering over knowledge graphs using knowledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.]:Association for Computational Linguistics, 2020:4498-4507.

选择文件类型/文献管理软件名称

选择包含的内容