Recommendation Algorithm Based on Multi-view Fusion Cross-layer Contrastive Learning

doi:10.19678/j.issn.1000-3428.0066906

Abstract

Abstract:

Existing recommendation models based on graph comparison learning usually use only one view enhancement method in graph data enhancement, ignoring the limitations of a single method. In contrastive learning, only a pair of views from the same node are usually compared, which do not fully utilize the different layer embeddings of each view. To this end, this study proposes a recommendation algorithm framework based on Multi-view Fusion Cross-layer Contrastive Learning (MFCCL). MFCCL constructs two global views using random edge drop and random noise addition enhancement methods, local views using Singular Value Decomposition (SVD), and three global and local views using three different view enhancement methods to achieve effective user representation. Simultaneously, a new MFCCL method is proposed, which embeds different layers of two global views through parallel and cross fusion methods for comparison, to obtain more feature information. Combining MFCCL with global-local view contrastive learning aims to jointly optimize the model and improve recommendation performance. Experiments are conducted on three publicly available datasets: Yelp, Tmall, and Amazon-book, and the results demonstrate that MFCCL is effective and feasible in recommendation tasks. Compared to the baseline model SimGCL, which had the best neutral performance in the comparative model, MFCCL exhibited better performance in the three datasets. The Recall@20 gains reached 15.0%, 13.3%, and 28.7%, and the NDCG@20 values increased by 14.3%, 13.2%, and 29.6%, respectively.

Key words: Graph Neural Network(GNN), contrastive learning, view enhancement, multi-view fusion, recommendation algorithm

摘要：

现有基于图对比学习的推荐模型在图数据增强方面通常只采用一种视图增强方法，忽略了单一方法存在的局限性，在对比学习方面通常只对比同一节点的一对视图，未充分利用各个视图不同的层嵌入。为此，提出一种基于多视图融合跨层对比学习的推荐算法框架(MFCCL)。MFCCL分别使用随机边丢弃和随机添加噪声的增强方法构建2个全局视图，使用奇异值分解的方法构建局部视图，通过3种不同的视图增强方法构造全局和局部共3个视图，以实现有效的用户表示。同时，提出一种新的多视图融合跨层对比学习方法，该方法将2个全局视图不同的层嵌入通过平行和交叉2种方式进行融合后作对比，以获取更多的特征信息。将多视图融合跨层对比学习与全局-局部视图对比学习相结合，联合优化模型，从而提升推荐性能。在Yelp、Tmall和Amazon-book这3个公开数据集上进行实验，结果表明，MFCCL在推荐任务中具有有效性和可行性，相较于对比模型中性能最优的基线模型SimGCL，MFCCL在3个数据集中的Recall@20增益分别达到15.0%、13.3%和28.7%，NDCG@20值分别提升14.3%、13.2%和29.6%。

关键词: 图神经网络, 对比学习, 视图增强, 多视图融合, 推荐算法

Jiajing GU, Dan YANG, Tiezheng NIE, Yue KOU. Recommendation Algorithm Based on Multi-view Fusion Cross-layer Contrastive Learning[J]. Computer Engineering, 2024, 50(1): 120-128.

顾嘉静, 杨丹, 聂铁铮, 寇月. 基于多视图融合跨层对比学习的推荐算法[J]. 计算机工程, 2024, 50(1): 120-128.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066906

http://www.ecice06.com/EN/Y2024/V50/I1/120

Figures/Tables 10

Fig.1 Recommendation algorithm framework based on multi-view fusion cross-layer contrastive learning

Fig.2 Multi-view fusion cross-layer contrastive learning

Fig.3 Experimental results of data sparsity

Fig.4 T-SNE visualization of embedded distribution on Yelp dataset

Fig.5 Effect of

$ {\lambda } $

on model performance

Fig.6 Effect of

$ {\tau } $

on model performance

References 34

1	KAPANIPATHI P, THOST V, SANKALP PATEL S, et al. Infusing knowledge into the textual entailment task using graph convolutional networks. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8074- 8081. doi: 10.1609/aaai.v34i05.6318
2	GALETZKA F, ROSE J, SCHLANGEN D, et al. Space efficient context encoding for non-task-oriented dialogue generation with graph attention transformer[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. [S. l. ]: Association for Computational Linguistics, 2021: 15-26.
3	MOHAMED A, QIAN K, ELHOSEINY M, et al. Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2002.11927.
4	ZHANG Z Q, SHI Y Y, YUAN C F, et al. Object relational graph with teacher-recommended learning for video captioning[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 22-36.
5	潘嘉诚, 董一鸿, 陈华辉. 基于图神经网络的自闭症辅助诊断研究综述. 计算机工程, 2022, 48(9): 1- 11. URL
	PAN J C, DONG Y H, CHEN H H. Review of research on auxiliary diagnosis of autism based on graph neural networks. Computer Engineering, 2022, 48(9): 1- 11. URL
6	DEVLIN J, CHANG M, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. [2023-01-05]. https://arxiv.org/abs/1810.04805.pdf.
7	GIDARIS S, SINGH P, KOMODAKIS N. Unsupervised representation learning by predicting image rotations[EB/OL]. [2023-01-05]. https://arxiv.org/abs/1803.07728.
8	CHI P H, CHUNG P H, WU T H, et al. Audio ALBERT: a lite BERT for self-supervised learning of audio representation[C]//Proceedings of 2021 IEEE Spoken Language Technology Workshop. Washington D. C., USA: IEEE Press, 2021: 123-156.
9	WU J C, WANG X, FENG F L, et al. Self-supervised graph learning for recommendation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2021: 726-735.
10	王曙燕, 郭睿涵, 孙家泽. 基于图对比学习的MOOC推荐方法. 计算机工程, 2023, 49(1): 57-64, 72. URL
	WANG S Y, GUO R H, SUN J Z. Recommendation method for MOOC based on graph contrastive learning. Computer Engineering, 2023, 49(1): 57-64, 72. URL
11	QIU J Z, CHEN Q B, DONG Y X, et al. GCC: graph contrastive coding for graph neural network pre-training[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2020: 1150-1160.
12	ZHU Y Q, XU Y C, YU F, et al. Graph contrastive learning with adaptive augmentation[C]//Proceedings of the 2021 Web Conference. New York, USA: ACM Press, 2021: 2069-2080.
13	SCARSELLI F, GORI M, TSOI A C, et al. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61- 80. doi: 10.1109/TNN.2008.2005605
14	BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs[EB/OL]. [2023-01-05]. https://arxiv.org/abs/1312.6203.
15	HU W, LIU B, GOMES J, et al. Strategies for pre-training graph neural networks[EB/OL]. [2023-01-05]. https://arxiv.org/abs/1905.12265.pdf.
16	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. [2023-01-05]. https://arxiv.org/abs/1609.02907.pdf.
17	KRASANAKIS E, PAPADOPOULOS S, KOMPATSIARIS I. p2pGNN: a decentralized graph neural network for node classification in peer-to-peer networks. IEEE Access, 2022, 10, 34755- 34765. doi: 10.1109/ACCESS.2022.3159688
18	LIBEN-NOWELL D, KLEINBERG J. The link prediction problem for social networks[C]//Proceedings of the 12th International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2003: 556-559.
19	YING R, YOU J X, MORRIS C, et al. Hierarchical graph representation learning with differentiable pooling[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2018: 4805-4815.
20	TAN Q Y, LIU N H, ZHAO X, et al. Learning to hash with graph neural networks for recommender systems[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2003.01917.
21	HE X N, LIAO L Z, ZHANG H W, et al. Neural collaborative filtering[EB/OL]. [2023-01-05]. https://arxiv.org/abs/1708.05031.
22	苗雨欣, 宋春花, 牛保宁, 等. 双通道图协同过滤推荐算法. 计算机工程, 2022, 48(8): 121- 128. URL
	MIAO Y X, SONG C H, NIU B N, et al. Dual-channel graph collaborative filtering recommendation algorithm. Computer Engineering, 2022, 48(8): 121- 128. URL
23	XIA X, YIN H Z, YU J L, et al. Self-supervised hypergraph convolutional networks for session-based recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(5): 4503- 4511. doi: 10.1609/aaai.v35i5.16578
24	YANG Y H, HUANG C, XIA L H, et al. Knowledge graph contrastive learning for recommendation[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2022: 1434-1443.
25	WEI Y W, WANG X, LI Q, et al. Contrastive learning for cold-start recommendation[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2107.05315.
26	HALKO N, MARTINSSON P G, TROPP J A. Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 2011, 53(2): 217- 288. doi: 10.1137/090771806
27	CHEN L, WU L, HONG R, et al. Revisiting graph based collaborative filtering: a linear residual graph convolutional network approach[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2001.10167.pdf.
28	HE X H, DENG K, WANG X, et al. LightGCN: simplifying and powering graph convolution network for recommendation[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2002.02126.
29	WANG X, JIN H Y, ZHANG A, et al. Disentangled graph collaborative filtering[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2007.01764.
30	WANG J L, DING K Z, HONG L J, et al. Next-item recommendation with sequential hypergraphs[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2020: 1101-1110.
31	YU J L, YIN H Z, LI J D, et al. Self-supervised multi-channel hypergraph convolutional network for social recommendation[EB/OL]. [2023-01-05]. https://arxiv.org/abs/2101.06448.
32	XIA L H, HUANG C, XU Y, et al. Hypergraph contrastive collaborative filtering[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2022: 70-79.
33	XIA L H, HUANG C, ZHANG C X. Self-supervised hypergraph transformer for recommender systems[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2022: 2100-2109.
34	YU J L, YIN H Z, XIA X, et al. Are graph augmentations necessary? simple graph contrastive learning for recom-mendation[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2022: 1294-1303.

[1]	Faxin CAO, Yuanyuan SUN, Zhizheng WANG, Dinghao PAN, Hongfei LIN. Similar Case Matching Model for Lending Cases [J]. Computer Engineering, 2024, 50(1): 306-312.
[2]	Zhiqiang WU, Qing XIE, Lin LI, Yongjian LIU. Graph Neural Network Recommendation Algorithm Based on Multimodal Fusion [J]. Computer Engineering, 2024, 50(1): 91-100.
[3]	Xiaoli LIU, Yitong WANG. Multi-density Graph-based Session Recommendation Using Self-supervised Learning [J]. Computer Engineering, 2023, 49(9): 60-68, 78.
[4]	XU Feng, YANG Xingyao, YU Jiong, LI Ziyang, LI Chenyu, ZHANG Jun. Wavelet Convolution Enhanced Contrastive Learning Recommendation Algorithm [J]. Computer Engineering, 2023, 49(5): 105-111,121.
[5]	LI Xiaoteng, ZHANG Panpan, GOU Zhinan, GAO Kai. Multi-Modal Named Entity Recognition Method Based on Multi-Task Learning [J]. Computer Engineering, 2023, 49(4): 114-119.
[6]	LI Pan, XIE Qing, LI Lin, LIU Yongjian. Knowledge-Enhanced Graph Neural Network Model for Sequential Recommendation [J]. Computer Engineering, 2023, 49(2): 70-80.
[7]	Anqi CHEN, Rui CHEN, Zhufang KUANG, Huajun HUANG. Research on Imbalance Fraud Detection Based on Graph Neural Network [J]. Computer Engineering, 2023, 49(11): 150-159.
[8]	Wanhua LI, Yingjuan SUN, Yixuan LIU, Qian LIU. Session Recommendation Based on Global Graph and Multi-granularity Intention Unit [J]. Computer Engineering, 2023, 49(10): 136-144, 153.
[9]	WANG Shuyan, GUO Ruihan, SUN Jiaze. Recommendation Method for MOOC Based on Graph Contrastive Learning [J]. Computer Engineering, 2023, 49(1): 57-64,72.
[10]	XU Shangshang, SUN Fuzhen, WANG Shaoqing, DONG Jiawei, WU Tianhui. Heterogeneous Trust Recommendation Algorithm Based on Graph Neural Networks [J]. Computer Engineering, 2022, 48(9): 89-95,104.
[11]	PAN Jiacheng, DONG Yihong, CHEN Huahui. Review of Research on Auxiliary Diagnosis of Autism Based on Graph Neural Networks [J]. Computer Engineering, 2022, 48(9): 1-11.
[12]	DING Qingfeng, LI Jinguo. A Distributed Abnormal Traffic Detection Scheme in Internet of Things Environment [J]. Computer Engineering, 2022, 48(8): 152-159.
[13]	MIAO Yuxin, SONG Chunhua, NIU Baoning, KANG Ruixue. Dual-Channel Graph Collaborative Filtering Recommendation Algorithm [J]. Computer Engineering, 2022, 48(8): 121-128.
[14]	WANG Yan, FAN Lin, ZHAO Nini. Sequential Recommendation Model Using Gated Network to Construct User's Dynamic Interest [J]. Computer Engineering, 2022, 48(8): 283-291.
[15]	ZHAO Yue, WU Zhihao, ZHAO Yiji. CTR Prediction Method Based on Feature and Domain Perception [J]. Computer Engineering, 2022, 48(3): 60-68.

Please choose a citation manager

Content to export