基于异质注意力循环神经网络的文本推荐

doi:10.19678/j.issn.1000-3428.0055861

摘要/Abstract

摘要： 针对当前大数据环境下文本推荐不精确的问题，对文本数据和关系网络2种异质数据进行融合，并引入编码器-解码器框架，提出基于异质注意力的循环神经网络模型用于短期文本推荐。使用句子级的分布记忆模型和实体关系表示方法TransR，分别将文本数据和关系网络嵌入到高维向量中作为模型的输入。在编码器阶段，使用双向GRU将用户的短期兴趣引入到推荐模型中，并将注意力机制与解码器相连接，使解码器能动态地选择并线性组合编码器输入序列的不同部分，以建模用户在短期内的偏好。在解码器阶段，将编码器的注意力输出、候选项和当前用户的表示作为输入。通过双向GRU和前馈网络层，计算每个候选项的得分得到推荐结果。实验结果表明，与TF-IDF和ItemKNN等模型相比，该模型在召回率和均值平均精度指标上均有明显提升。

Abstract: To improve the inaccurate text recommendation in the big data environment,this paper merges two kinds of heterogeneous data,text data and relational network,and introduces the encoder-decoder framework.On this basis,a Recurrent Neural Network(RNN) model based on heterogeneous attention is proposed for short-term text recommendation.The sentence-level Distributed Memory Model of Paragraph Vectors(PV-DM) and the representation method for entity relations,TransR,are used to embed text data and relational network into high-dimensional vectors as the input of the model.In the encoding stage,the short-term interests of users are introduced into the recommendation model by using bidirectional GRU,and the attention mechanism is used to connect with the decoder,so that the decoder can dynamically select and linearly combine different parts of the input sequence of the encoder in order to build short-term interests of users.In the decoder stage,the attention output of the encoder,the candidate items,and the representation of current users are taken as inputs.The score of each candidate item is calculated with the bidirectional GRU and the feedforward network layer to obtain the recommendation result.Experimental results show that compared with TF-IDF,ItemKNN and other models,the proposed model significantly improves the recall rate and the average precision of the mean.

Key words: short-term text recommendation, data embedding, heterogeneous data, bidirectional GRU, attention mechanism

中图分类号:

TP18

牛耀强, 孟昱煜, 牛全福. 基于异质注意力循环神经网络的文本推荐[J]. 计算机工程, 2020, 46(10): 52-59.

NIU Yaoqiang, MENG Yuyu, NIU Quanfu. Text Recommendation Based on Heterogeneous Attention Recurrent Neural Network[J]. Computer Engineering, 2020, 46(10): 52-59.

http://www.ecice06.com/CN/Y2020/V46/I10/52

图/表 12

20201023091432

20201023091436

20201023091439

20201023091443

20201023091446

20201023091451

20201023091453

20201023091458

20201023091502

20201023091506

20201023091509

20201023091514

参考文献

[1] RENDLE S,FREUDENTHALER C,GANTNER Z,et al.BPR:Bayesian personalized ranking from implicit feedback[C]//Proceedings of International Conference on Uncertainty in Artificial Intelligence.[S.1.]:AAAI Press,2009:137-149.
[2] XU Lei,YANG Cheng,JIANG Chunxiao,et al.Game analysis of user participation in collaborative filtering systems[J].Chinese Journal of Computers,2016,39(6):1176-1189.(in Chinese)徐蕾,杨成,姜春晓,等.协同过滤推荐系统中的用户博弈[J].计算机学报,2016,39(6):1176-1189.
[3] LANG K.NewsWeeder:learning to filter net-news[C]//Proceedings of the 12th International Conference on Machine Learning.[S.1.]:Morgan Kaufmann Publishers Inc.,1995:457-469.
[4] CHEN J,YING P,ZOU M.Improving music recommendation by incorporating social influence[J].Multimedia Tools and Applications,2019,78(3):2667-2687.
[5] FENG Xingjie,ZENG Yunze.Joint deep modeling of rating matrix and reviews for recommendation[J].Chinese Journal of Computers,2020,43(5):884-900.(in Chinese)冯兴杰,曾云泽.基于评分矩阵与评论文本的深度推荐模型[J].计算机学报,2020,43(5):884-900.
[6] BEIERLE F,AIZAWA A,BEEL J.Exploring choice overload in related-article recommendations in digital libraries[EB/OL].[2019-07-20].https://www.researchgate.net/publication.
[7] XIA F,LIU H,LEE I,et al.Scientific article recommendation:exploiting common author relations and historical preferences[J].IEEE Transactions on Big Data,2016,2(2):101-112.
[8] ELKAhkyHKY A M,YANG S,HE X D.A multi-view deep learning approach for cross domain user modeling in recommendation systems[EB/OL].[2019-07-20].https://www.researchgate.net/publication/.
[9] XU Z,CHEN C,LUKASIEWICZ T,et al.Tag-aware personalized recommendation using a deep-semantic similarity model with negative sampling[C]//Proceedings of ACM Conference on Information and Knowledge Management.New York,USA:ACM Press,2016:443-456.
[10] GONG Yeyun,ZHANG Qi.Hashtag recommendation using attention-based convolutional neural network[C]//Proceedings of IEEE International Joint Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2016:325-337.
[11] SEO S,HUANG J,YANG H,et al,Representation learning of users and items for review rating prediction using attention-based convolutional neural network[C]//Proceedings of the 3rd International Workshop on Machine Learning Methods for Recommender Systems.Huston,USA:[s.n.],2017:224-239.
[12] LI Yang,LIU Ting,JIANG Jing et al.Hashtag recommendation with topical attention-based LSTM[C]//Proceedings of the 26th International Conference on Computational Linguistics.Osaka,Japan:[s.n.],2016:943-952.
[13] HUANG Haoran,ZHANG Qi,HUANG Xuanjing.Mention recommendation for twitter with end-to-end memory network[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.[S.1.]:AAAI Press,2017:334-356.
[14] VERBERT K,MANOUSELIS N,OCHOA X,et al.Context-aware recommender systems for learning:a survey and future challenges[J].IEEE Transactions on Learning Technologies,2012,5(4):318-335.
[15] WANG X,ROSENBLUM D,WANG Y.Context-aware mobile music recommendation for daily activities[C]//Proceedings of the 20th ACM International Conference on Multimedia.New York,USA:ACM Press,2012:99-108.
[16] WANG Dongjing,DENG Shuiguang,ZHANG Xin,et al.Learning to embed music and metadata for context-aware music recommendation[J].World Wide Web-Internet & Web Information Systems,2017(5461):1-25.
[17] WANG Dongjing,DENG Shuiguang,XU Guangdong.Sequence-based context-aware music recommendation[J].Information Retrieval Journal,2018,21(2/3):230-252.
[18] MIKOLOV T,CHEN K,CORRADO G,et al.efficient estimation of word representations in vector space[C]//Proceedings of ICLR'13.Washingtor D.C.,USA:IEEE Press,2013:23-42.
[19] MIKOLOV T,SUTSKERER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[J].Advances in Neural Information Processing Systems,2013,26:3111-3119.
[20] BOGINA V,KUFLI T.Incorporating dwell time in session-based recommendations with recurrent neural networks[C]//Proceedings of ACM Conference on Recommender System.New York,USA:ACM Press,2017:57-59.
[21] QUADRANA M,KARATZOGLOU A,HIDASI B,et al.Personalizing session-based recommendations with hierarchical recurrent neural networks[C]//Proceedings of ACM RecSys'17.New York,USA:ACM Press,2017:432-446.
[22] LE Q,MIKOLOV T.Distributed representations of sentences and documents[EB/OL].[2019-07-20].https://arxiv.org/abs/1405.4053
[23] LIN Yankai,LIU Zhiyuan,SUN Maosong,et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence.[S.1.]:AAAI Press,2015:2181-2187.
[24] BAHDANAU D,CHO K,BENGIO Y.Neural machine Translation by jointly learning to align and translate[EB/OL].[2019-07-20].https://arxiv.org/abs/1409.0473.
[25] BENGIO Y,SIMAND P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,2002,5(2):157-166.
[26] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[27] ZHANG Dongxu,WANG Dong.Relation classification via recurrent neural network[EB/OL].[2019-07-20].https://arxiv.org/abs/1508.01006.
[28] LINDEN G,SMITH B,YORK J.Amazon.com recommendations:item-to-item collaborative filtering[J].IEEE Internet Computing,2003,7(1):76-80.
[29] KINGMA D P,BA J L.Adam:a method for stochastic optimization[C]//Proceedings of International Conference on Learning Represent.Washington D.C.,USA:IEEE Press,2015:1-15.

选择文件类型/文献管理软件名称

选择包含的内容