基于随机自注意力和动量对比学习的自监督序列推荐方法

doi:10.19678/j.issn.1000-3428.0070193

计算机工程 ›› 2026, Vol. 52 ›› Issue (6): 132-140. doi: 10.19678/j.issn.1000-3428.0070193

基于随机自注意力和动量对比学习的自监督序列推荐方法

余正涛¹^,²^,*(), 孙资钦¹^,², 张勇丙¹^,², 高盛祥¹^,², 黄于欣¹^,², 谭凯文¹^,²

1. 昆明理工大学信息工程与自动化学院, 云南昆明 650000
2. 云南省人工智能重点实验室, 云南昆明 650000

收稿日期:2024-08-05 修回日期:2024-10-27 出版日期:2026-06-15 发布日期:2026-06-02
通讯作者: 余正涛
作者简介:
余正涛，男，教授、博士，主研方向为人工智能、自然语言处理
孙资钦，硕士
张勇丙，博士
高盛祥，教授、博士
黄于欣，教授、博士
谭凯文，副教授、博士
基金资助:
国家自然科学基金联合基金重点项目(U23A20388); 国家自然科学基金(U21B2027); 国家自然科学基金(62376111); 国家自然科学基金(62266028); 国家自然科学基金(62266027); 云南省重点研发计划(202303AP140008); 云南省重点研发计划(202401BC070021); 云南省重点研发计划(202103AA080015); 云南省科技人才与平台计划(202105AC160018); 云南省基础研究项目(202301AT070393)

Self-Supervised Sequence Recommendation Method Based on Random Self-Attention and Momentum Contrastive Learning

YU Zhengtao¹^,²^,*(), SUN Ziqin¹^,², ZHANG Yongbing¹^,², GAO Shengxiang¹^,², HUANG Yuxin¹^,², TAN Kaiwen¹^,²

1. School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650000, Yunnan, China
2. Yunnan Key Laboratory of Artificial Intelligence, Kunming 650000, Yunnan, China

Received:2024-08-05 Revised:2024-10-27 Online:2026-06-15 Published:2026-06-02
Contact: YU Zhengtao

摘要/Abstract

摘要：

序列推荐利用用户历史序列行为建模用户兴趣并进行内容推荐, 常被用于新闻、广告和电子商务等领域。基于对比学习的自监督序列推荐是当前研究热点, 然而, 真实的序列数据具有动态不确定, 且对比学习中存在采样偏移问题, 限制了推荐的性能。为了缓解这些问题, 提出基于随机自注意力和动量对比学习的自监督序列推荐方法, 其中随机自注意力用于缓解序列动态不确定问题, 动量对比学习用于缓解对比学习中存在采样偏移问题。为验证模型性能, 在Beauty、Office、Yelp和Toys 4个常用数据集上的实验结果表明, 该方法在HR@K、NDCG@K等多个指标上均优于其他基线模型, 展示了该方法在准确性和鲁棒性方面的显著提升。

Abstract:

Sequence recommendation utilizes user historical sequence behavior to model user interests and provide content recommendations, and is commonly employed in sectors such as news, advertising, and e-commerce. Self-supervised sequence recommendation based on contrastive learning is a current research hotspot. However, real sequence data are dynamically uncertain, and sampling biases exist in contrastive learning, which limit the performance of recommendations. To mitigate these issues, this paper proposes a self-supervised sequence recommendation method based on stochastic self-attention and momentum contrastive learning. Stochastic self-attention is used to alleviate the uncertainty of sequence dynamics, and momentum contrastive learning is used to mitigate the sampling bias problem in contrastive learning. To validate the performance of the model, experiments are conducted on three datasets: Beauty, Office, Yelp, and Toys. The results demonstrate that the proposed method outperforms other baseline models across several metrics, including HR@K and NDCG@K, indicating significant improvements in both accuracy and robustness.

Key words: sequence recommendation, momentum contrastive learning, Wasserstein distance, self-supervised learning, self-attention

余正涛, 孙资钦, 张勇丙, 高盛祥, 黄于欣, 谭凯文. 基于随机自注意力和动量对比学习的自监督序列推荐方法[J]. 计算机工程, 2026, 52(6): 132-140.

YU Zhengtao, SUN Ziqin, ZHANG Yongbing, GAO Shengxiang, HUANG Yuxin, TAN Kaiwen. Self-Supervised Sequence Recommendation Method Based on Random Self-Attention and Momentum Contrastive Learning[J]. Computer Engineering, 2026, 52(6): 132-140.

https://www.ecice06.com/CN/Y2026/V52/I6/132

图/表 11

图1 基于随机自注意力和动量对比学习的自监督序列推荐方法总体框架

Fig.1 Overall framework of self-supervised sequence recommendation method based on random self-attention and momentum contrastive learning

图2 动量对比和实例加权

Fig.2 Momentum contrast and instance weighting

图3 不同训练数据比例的鲁棒性

Fig.3 Robust over different training data ratios

图4 不同噪声比例下的鲁棒性

Fig.4 Robust over different noise ratios

图5 不同批次大小的鲁棒性

Fig.5 Robust over different batch sizes

参考文献 36

1	WANG S, GONG J, WANG J, et al. Attentional graph convolutional networks for knowledge concept recommendation in MOOCs in a heterogeneous view[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2020: 79-88.
2	LIU Z, LI X, FAN Z, et al. Basket recommendation with multi-intent translation graph neural network[C]// Proceedings of the IEEE International Conference on Big Data. Washington D. C., USA: IEEE Press, 2020: 728-737.
3	李忠伟, 周洁, 刘昕, 等. 融合时间和知识信息的生成对抗网络序列推荐算法. 计算机工程, 2024, 50 (11): 70- 79. doi: 10.19678/j.issn.1000-3428.0068300
	LI Z W , ZHOU J , LIU X , et al. Sequence recommendation algorithm based on generative adversarial network integrating time and knowledge information. Computer Engineering, 2024, 50 (11): 70- 79. doi: 10.19678/j.issn.1000-3428.0068300
4	LIN X, ILIA P, SOLANKI S, et al. Phish in sheep's clothing: exploring the authentication pitfalls of browser fingerprinting[C]//Proceedings of the 31st USENIX Security Symposium. Washington D. C., USA: IEEE Press, 2022: 1651-1668.
5	LIN X, ILIA P, PILAKIS J. Fill in the blanks: empirical analysis of the privacy threats of browser form autofill[C]//Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. New York, USA: ACM Press, 2020: 507-519.
6	杨纪元, 马沐阳, 任鹏杰, 等. 基于自监督的预训练在推荐系统中的研究. 山东大学学报(理学版), 2024, 59 (7): 1- 26.
	YANG J Y , MA M Y , REN P J , et al. Research on self-supervised pre-training for recommender systems. Journal of Shandong University (Science Edition), 2024, 59 (7): 1- 26.
7	倪文锴, 杜彦辉, 马兴帮, 等. 面向个性化推荐的node2vec-side融合知识表示. 计算机应用研究, 2024, 41 (2): 361-367, 374.
	NI W K , DU Y H , MA X B , et al. Node2vec-side fusion knowledge representation for personalized recommendation. Application Research of Computers, 2024, 41 (2): 361-367, 374.
8	ZHOU K, WANG H, ZHAO W X, et al. S. 3-Rec: self-supervised learning for sequential recommendation with mutual information maximization[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York, USA: ACM Press, 2020: 1893-1902.
9	WU J, WANG X, FENG F, et al. Self-supervised graph learning for recommendation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2021: 726-735.
10	WANG C , MA W , CHEN C , et al. Sequential recommendation with multiple contrast signals. ACM Transactions on Information Systems, 2023, 41 (1): 11- 27.
11	FAN Z, LIU Z, PENG H, et al. Mutual Wasserstein discrepancy minimization for sequential recommendation[C]//Proceedings of the International World Wide Web Conferences. Washington D. C., USA: IEEE Press, 2023: 1375-1385.
12	ZIMDARS A, CHICKERING D M, MEEK C. Using temporal data for making recommendations[C]//Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence. Seattle, USA: Morgan Kaufmann Publishers Inc., 2001: 580-588.
13	YAP G E, LI X L, YU P S. Effective next-items recommendation via personalized sequential pattern mining[C]//Proceedings of the 17th International Conference on Database Systems for Advanced Applications. Berlin, Germany: Springer, 2012: 48-64.
14	LI J, REN P, CHEN Z, et al. Neural attentive session-based recommendation[C]//Proceedings of the 26th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2017: 1419-1428.
15	TAN Y K, XU X, LIU Y. Improved recurrent neural networks for session-based recommendations[EB/OL]. [2024-07-01]. https://arxiv.org/abs/1606.08117.
16	SUN F, LIU J, WU J, et al. BERT4Rec: sequential recommendation with bidirectional encoder representations from Transformer[EB/OL]. [2024-07-01]. https://arxiv.org/abs/1904.06690.
17	KANG W C, MCAULEY J. Self-attentive sequential recommendation[EB/OL]. [2024-07-01]. http://arxiv.org/abs/1808.09781.
18	XIE Z X, LI J Y. Contrastive learning for sequential recommendation[EB/OL]. [2024-07-01]. https://arxiv.org/abs/2010.14395.
19	王帅, 史艳翠. 基于个性化数据增强的自监督序列推荐算法. 计算机工程, 2025, 51 (8): 190- 202. doi: 10.19678/j.issn.1000-3428.0069636
	WANG S , SHI Y C . Self-supervised sequence recommendation algorithm based on personalized data augmentation. Computer Engineering, 2025, 51 (8): 190- 202. doi: 10.19678/j.issn.1000-3428.0069636
20	MA J, ZHOU C, YANG H, et al. Disentangled self-supervision in sequential recommenders[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2020: 483-491.
21	YU J, YIN H, LI J. Self-supervised multi-channel hypergraph convolutional network for social recommendation[C]//Proceedings of the Web Conference. New York, USA: ACM Press, 2021: 413-424.
22	YAO T, YI X, CHENG D Z, et al. Self-supervised learning for large-scale item recommendations[EB/OL]. [2024-07-01]. http://arxiv.org/abs/2010.11459.
23	BOJCHEVSKI A, GUNNEMANN S. Deep Gaussian embedding of graphs: unsupervised inductive learning via ranking[EB/OL]. [2024-07-01]. https://arxiv.org/abs/1707.03815.
24	HE S, LIU K, JI G, et al. Learning to represent knowledge graphs with Gaussian embedding[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. New York, USA: ACM Press, 2015: 623-632.
25	SUN C, YAN H, QIU X, et al. Gaussian word embedding with a Wasserstein distance loss[EB/OL]. [2024-07-01]. https://arxiv.org/abs/1808.07016.
26	VILNIS L, MCCALLUM A. Word representations via Gaussian embedding[EB/OL]. [2024-07-01]. https://arxiv.org/abs/1412.6623.
27	FAN Z, LIU Z, WANG A, et al. Sequential recommendation via stochastic self-attention[C]//Proceedings of the ACM Web Conference. New York, USA: ACM Press, 2022: 325-336.
28	HE K, FAN H, WU Y, et al. Momentum contrast for unsupervised visual representation learning[EB/OL]. [2024-07-01]. https://arxiv.org/abs/1911.05722.
29	CHEN J , DONG H , WANG X , et al. Bias and debias in recommender system: a survey and future directions. ACM Transactions on Information Systems, 2023, 41 (3): 1- 39.
30	VU T. Correct sampling bias for recommender systems[EB/OL]. [2024-07-01]. https://towardsdatascience.com/correct-sampling-bias-for-recommender-systems-d2f6d9fdddec.
31	ZHOU K, ZHANG B, ZHAO W X, et al. Debiased contrastive learning of unsupervised sentence representations[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. [S. 1. ]: ACL Press, 2022: 6120-6130.
32	安永丽, 方斌, 刘劲芸, 等. 一种基于互信息神经估计改进的未知信道端到端通信系统. 电讯技术, 2024, 64 (9): 1386- 1393.
	AN Y L , FANG B , LIU J Y , et al. An improved end-to-end unknown channel communication system based on mutual information neural estimation. Telecommunication Engineering, 2024, 64 (9): 1386- 1393.
33	FAN Z, LIU Z, ZHENG L, et al. Modeling sequences as distributions with uncertainty for sequential recommendation[EB/OL]. [2024-07-01]. https://arxiv.org/abs/2106.06165.
34	LI J, WANG Y, MCAULEY J. Time interval aware self-attention for sequential recommendation[C]//Proceedings of the 13th ACM International Conference on Web Search and Data Mining. New York, USA: ACM Press, 2020: 3223-3235.
35	RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: bayesian personalized ranking from implicit feedback[C]//Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Montreal, Canada: [s. n. ], 2009: 3321-3335.
36	WEI Z , WU N , LI F , et al. MoCo4SRec: a momentum contrastive learning framework for sequential recommendation. Expert Systems with Applications, 2023, 223, 119911. doi: 10.1016/j.eswa.2023.119911

[1]	胡康源, 郭涛, 穆楠. 基于自注意力机制和动态掩膜机制的文物图像修复方法[J]. 计算机工程, 2026, 52(6): 179-188.
[2]	宋天泽, 曹从军, 何佳琪, 王旭升, 刘晨煜. 基于改进DETR的密集行人检测算法研究[J]. 计算机工程, 2026, 52(5): 250-258.
[3]	刘海军, 付晓东. 结合旋转自监督和CLIP指导的长尾数据联邦学习[J]. 计算机工程, 2026, 52(5): 129-138.
[4]	苏建华, 池云仙, 许云峰, 高凯. 基于注意力模态融合的多模态意图识别[J]. 计算机工程, 2026, 52(3): 234-242.
[5]	刘啸宇, 廖志芳, 谈遂, 余志武. 基于堆叠GRU神经网络的桥梁动应变预测[J]. 计算机工程, 2026, 52(3): 441-450.
[6]	翟志鹏, 曹阳, 沈琴琴, 施佺. 基于多时空图融合与动态注意力的交通流预测[J]. 计算机工程, 2025, 51(9): 139-148.
[7]	冯雅莉, 温雯, 郝志峰, 蔡瑞初. 融合转移关系正则化的序列推荐[J]. 计算机工程, 2025, 51(8): 151-159.
[8]	王帅, 史艳翠. 基于个性化数据增强的自监督序列推荐算法[J]. 计算机工程, 2025, 51(8): 190-202.
[9]	周哲臣, 胡冀苏, 钱旭升, 郑毅, 戴亚康, 周志勇. 基于查询自适应双层自注意力机制的MRI脑组织分割[J]. 计算机工程, 2025, 51(7): 294-304.
[10]	陈思帆, 杨家志, 黄琳, 吕志玮, 沈露. 融合可变形核和自注意力的点云分类分割边卷积网络[J]. 计算机工程, 2025, 51(6): 146-154.
[11]	赵瑶谦, 滕奇志, 何小海, 税爱, 陈洪刚. 基于自注意力特征蒸馏的轻量级图像超分辨率重建[J]. 计算机工程, 2025, 51(5): 257-265.
[12]	陈梓延, 王晓龙, 何迪, 安国成. 基于改进YOLOv8的轻量化车辆检测网络[J]. 计算机工程, 2025, 51(5): 314-325.
[13]	刘云翔, 梁智超. 一种高效的连续时序图注意力网络的交通预测模型[J]. 计算机工程, 2025, 51(4): 350-359.
[14]	孙亭, 杨洁, 李家璇, 王耀宗. 面向弱光交通场景的YOLOv7道路标志检测算法优化[J]. 计算机工程, 2025, 51(3): 342-351.
[15]	王杨, 宋世佳, 王鹤琴, 袁振羽, 赵立军, 吴其林. 基于改进Vision Transformer的局部光照一致性估计[J]. 计算机工程, 2025, 51(2): 312-321.

选择文件类型/文献管理软件名称

选择包含的内容

基于随机自注意力和动量对比学习的自监督序列推荐方法

Self-Supervised Sequence Recommendation Method Based on Random Self-Attention and Momentum Contrastive Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 36

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于随机自注意力和动量对比学习的自监督序列推荐方法

Self-Supervised Sequence Recommendation Method Based on Random Self-Attention and Momentum Contrastive Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 36

相关文章 15

编辑推荐

Metrics

本文评价