融合复杂先验与注意力机制的变分自动编码器

doi:10.19678/j.issn.1000-3428.0063401

摘要/Abstract

摘要： 传统变分自动编码器模型通常使用标准正态分布作为隐向量先验，当应用于推荐系统等复杂任务时容易导致模型过度正则化和隐向量解耦表现不佳。融合复杂隐向量先验与注意力机制，建立变分自动编码器模型。使用多层神经网络生成的隐向量先验分布替代标准正态分布作为假设先验分布，使得模型能根据数据学习先验分布并获得更多的潜在表征。在单层隐向量的基础上添加辅助隐向量，联合辅助隐向量与数据特征向量再生成隐向量，增强了隐向量的低维表现能力和解耦性。借助注意力机制的特征信息选择特点，对隐向量中重要节点赋予更大的权重值，使其能传递更重要的信息。在数据集Movielens-1M、Movielens-Latest-Small、Movielens-20M和Netflix上的实验结果表明，该模型的Recall@20、Recall@50、NDCG@100相较于基线模型平均提升了12.95%、10.80%、10.48%，具有更高的推荐精确度。

Abstract: The traditional Variational Auto-Encoder(VAE) model typically uses the standard normal distribution as the implicit vector priori.When solving complex tasks such as the recommendation system, it easily leads to over-regularization of the model and poor performance of implicit vector decoupling.A VAE model combining a complex implicit vector priori and attention mechanism is built to solve these problems.First, the implicit vector prior distribution generated by a multilayer neural network is used to replace the standard normal distribution as the hypothetical prior distribution so that the model can learn the most appropriate prior distribution based on the data and obtain more potential representations.Next, an auxiliary implicit vector is added to regenerate the auxiliary implicit and data feature vectors into an implicit vector based on the single-layer implicit vector.Compared with the original structure, it can significantly improve the low-dimensional representation ability and decoupling of the implicit vector.Finally, based on the feature information selection of the attention mechanism, the attention mechanism is added to two implicit vectors to increase the weights of the critical nodes so that the implicit vector can transmit more important information.Experiments were performed on the public datasets:Movielens-1M, Movielens-Latest-Small, Movielens-20M, and Netflix.The results show that the proposed model has better evaluation indexes than the experimental comparison model on the Recall@20, Recall@50, and NDCG@100.The average increase values are 12.95%, 10.80%, and 10.48%, which show increased recommendation accuracy.

Key words: recommendation system, collaborative filtering, deep learning, Variational Auto-Encoder(VAE), auxiliary implicit vector, complex prior, attention mechanism

中图分类号:

TP391

沈学利, 马玉营, 梁振兴. 融合复杂先验与注意力机制的变分自动编码器[J]. 计算机工程, 2022, 48(11): 55-61.

SHEN Xueli, MA Yuying, LIANG Zhenxing. Variational Auto-Encoder Combining Complex Priori and Attention Mechanism[J]. Computer Engineering, 2022, 48(11): 55-61.

https://www.ecice06.com/CN/Y2022/V48/I11/55

图/表 11

20230203202435

20230203202438

20230203202441

20230203202445

20230203202448

20230203202452

20230203202455

20230203202458

20230203202502

20230203202506

20230203202509

参考文献

[1] 刘华玲, 马俊, 张国祥.基于深度学习的内容推荐算法研究综述[J].计算机工程, 2021, 47(7):1-12. LIU H L, MA J, ZHANG G X.Review of studies on deep learning-based content recommendation algorithms[J].Computer Engineering, 2021, 47(7):1-12.(in Chinese)
[2] FELFERNIG A, JERAN M, NINAUS G, et al.Toward the next generation of recommender systems:applications and research challenges[M]//TSIHRINTZIS G A, VIRVOU M, JAIN L C.Multimedia services in intelligent environments.Berlin, Germany:Springer, 2013:81-98.
[3] HU Y F, KOREN Y, VOLINSKY C.Collaborative filtering for implicit feedback datasets[C]//Proceedings of the 8th IEEE International Conference on Data Mining.Washington D.C., USA:IEEE Press, 2009:263-272.
[4] HE X N, LIAO L Z, ZHANG H W, et al.Neural collaborative filtering[C]//Proceedings of International World Wide Web Conferences Steering Committee.Washington D.C., USA:IEEE Press, 2017:173-182.
[5] BURGESS C P, HIGGINS I, PAL A, et al.Understanding disentangling in β-VAE[EB/OL].[2021-10-07].https://arxiv.org/abs/1804.03599.
[6] LIANG D W, KRISHNAN R G, HOFFMAN M D, et al.Variational autoencoders for collaborative filtering[C]//Proceedings of 2018 World Wide Web Conference.New York, USA:ACM Press, 2018:689-698.
[7] POLATO M, CARRARO T, AIOLLI F.Conditioned variational autoencoder for top-N item recommendation[EB/OL].[2021-10-07].https://arxiv.org/abs/2004. 11141.
[8] SIDDHARTH N, PAIGE B, DESMAISON A, et al.Learning disentangled representations for recommendation[EB/OL].[2021-10-07].http://stat.columbia.edu/~cunningham/teaching/GR8201/STAT_GR8201_2019_SPRG_slides.pdf.
[9] KINGMA D P, WELLING M.Auto-encoding variational Bayes[EB/OL].[2021-10-07].https://arxiv.org/abs/1312. 6114v4.
[10] MIAO Y S, YU L, BLUNSOM P.Neural variational inference for text processing[EB/OL].[2021-10-07].https://arxiv.org/abs/1511.06038.
[11] LU Y, HUANG B.Structured output learning with conditional generative flows[J].Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(4):5005-5012.
[12] TAHIR R, SARGANO A B, HABIB Z.Voxel-based 3D object reconstruction from single 2D image using variational autoencoders[J].Mathematics, 2021, 9(18):2288.
[13] SOHN K, YAN X C, LEE H, et al.Learning structured output representation using deep conditional generative models[C]//Proceedings of International Conference on Neural Information Processing Systems.Cambridge, USA:MIT Press, 2015:3483-3491
[14] BOUCHACOURT D, TOMIOKA R, NOWOZIN S.Multi-level variational autoencoder:learning disentangled representations from grouped observations[C]//Proceedings of AAAI Conference on Artificial Intelligence.Palo Alto, USA:AAAI Press, 2018:1-10.
[15] OUYANG Y X, LIU W Q, RONG W G, et al.Autoencoder-based collaborative filtering[M].Berlin, Germany:Springer, 2014.
[16] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al.Improving neural networks by preventing co-adaptation of feature detectors[EB/OL].[2021-10-07].https://arxiv.org/abs/1207.0580.
[17] LUONG M T, PHAM H, MANNING C D.Effective approaches to attention-based neural machine translation[EB/OL].[2021-10-07].https://arxiv.org/abs/1508. 04025.
[18] SPERBER M, NIEHUES J, NEUBIG G, et al.Self-attentional acoustic models[EB/OL].[2021-10-07].https://arxiv.org/abs/1803.09519v1.
[19] BRAUN S, NEIL D, ANUMULA J, et al.Multi-channel attention for end-to-end speech recognition[EB/OL].[2021-10-07].https://www.researchgate.net/publication/327388671_Multi-channel_Attention_for_End-to-End_Speech_Recognition.
[20] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[EB/OL].[2021-10-07].https://arxiv.org/abs/1706.03762.
[21] KINGMA D P, BA J.Adam:a method for stochastic optimization[EB/OL].[2021-10-07].https://arxiv.org/abs/1412.6980.
[22] GLOROT X, BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of International Conference on Artificial Intelligence and Statistics.Washington D.C., USA:IEEE Press, 2010:249-256.
[23] HE X N, LIAO L Z, ZHANG H W, et al.Neural collaborative filtering[C]//Proceedings of the 26th International Conference on World Wide Web.Geneva, Switzerland:International World Wide Web Conferences Steering Committee, 2017:1-8.
[24] JI S, PAN S, CAMBRIA E, et al.A survey on knowledge graphs:representation, acquisition, and applications[EB/OL].[2021-10-07].https://blog.csdn.net/byn12345/article/details/106128275.
[25] WANG X, HE X N, WANG M, et al.Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York, USA:ACM Press, 2019:165-174.

选择文件类型/文献管理软件名称

选择包含的内容