Sequential Recommendation Algorithm Based on Data Augmentation and Integrated Self-Attention

doi:10.19678/j.issn.1000-3428.0070584

Abstract

Abstract:

In recommender systems, sequential recommendations aim to predict the future interests of a user based on historical interaction sequences. However, existing deep learning models typically focus on capturing users' long-term behavior patterns while neglecting the fine-grained modeling of temporal information, which limits improvements in recommendation performance. To address this issue, this study proposes a method that combines the global and local self-attention mechanisms. This method processes user interaction sequences in chunks and introduces a weight decay mechanism that assigns differentiated weights to different sequence blocks based on temporal distance, thereby capturing changes in users' short- and long-term interests more accurately. However, this approach introduces additional parameters, thereby increasing the model complexity. While Stochastic Shared Embedding (SSE) technology can reduce the parameter count and mitigate overfitting, its random embedding approach may introduce noisy data, affecting the accuracy of the recommendations. To solve this problem, this study proposes a strategy that combines a Generative Adversarial Network (GAN) with SSE. By using GAN to generate high-quality interaction data that align with user interest distributions and combining this with the random replacement mechanism in SSE for data augmentation, the data are enhanced by randomly selecting generated data, thus reducing the risk of noise introduction while retaining SSE's advantage in reducing overfitting. Experiments are conducted on three public datasets: MovieLens-1M, Amazon Beauty, and Yahoo Music. The results show that the proposed method performs well in terms of Normalized Discounted Cumulative Gain (NDCG), Hit Rate (HR), and Mean Reciprocal Rank (MRR).

Key words: recommendation system, sequential recommendation, Generative Adversarial Network (GAN), data augmentation, attention mechanism

摘要：

在推荐系统中, 序列推荐旨在通过用户的历史交互序列来预测其未来兴趣。然而, 现有深度学习模型通常侧重于捕捉用户的长期行为模式, 而对时间信息的细粒度建模有所忽视, 从而限制了推荐效果的提升。为了解决这一问题, 提出一种结合全局与局部自注意力机制的方法, 通过分块处理用户交互序列并引入权重衰减机制, 根据时序距离为不同序列块分配差异化权重, 从而更精确地捕捉用户长短期兴趣的变化。然而, 这种方法会引入更多的参数, 增加模型的复杂度。虽然随机共享嵌入(SSE)技术能够减少参数量并缓解过拟合, 但其随机嵌入方式可能会引入噪声数据, 影响推荐结果的准确性。针对此问题, 提出一种结合生成对抗网络(GAN)与SSE技术的策略, 通过GAN生成符合用户兴趣分布的高质量交互数据, 并结合SSE中的随机替换机制进行数据增强, 在随机选择生成的数据时, 增加由GAN生成的高质量数据, 从而降低噪声引入的风险, 同时保留SSE技术在降低过拟合方面的优势。在MovieLens-1M、Amazon Beauty和Yahoo Music 3个公开数据集上进行实验, 结果表明, 所提出的方法在归一化折损累计增益(NDCG)、命中率(HR)以及平均倒数排名(MRR)指标上表现优异。

LI Zhongwei, LI Keyi, LIU Xin. Sequential Recommendation Algorithm Based on Data Augmentation and Integrated Self-Attention[J]. Computer Engineering, 2026, 52(7): 94-106.

李忠伟, 李可一, 刘昕. 基于数据增强和融合自注意力的序列推荐算法[J]. 计算机工程, 2026, 52(7): 94-106.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0070584

https://www.ecice06.com/EN/Y2026/V52/I7/94

Figures/Tables 9

Fig.1 Overall architecture of the recommendation model based on data enhancement and integrated self-attention

Fig.2 Generator module architecture

Fig.3 The impact of sample size on performance in MovieLens-1M dataset

Fig.4 The impact of sample size on performance in Yahoo Music dataset

Fig.5 The impact of sample size on performance in Amazon Beauty dataset

References 34

1	WANG S J, CAO L B, WANG Y, et al. A survey on session-based recommender systems. ACM Computing Surveys (CSUR), 2021, 54(7): 1- 38. doi: 10.1145/3465401
2	魏星, 孙浩, 曹健, 等. 基于增强记忆网络的会话推荐算法. 计算机工程, 2024, 50(12): 83- 89. doi: 10.19678/j.issn.1000-3428.0068444
	WEI X, SUN H, CAO J, et al. Session-based recommendation algorithm based on memory augmented network. Computer Engineering, 2024, 50(12): 83- 89. doi: 10.19678/j.issn.1000-3428.0068444
3	夏英, 陈航. 融合Transformer和卷积LSTM的轨迹分类网络. 重庆邮电大学学报(自然科学版), 2024, 36(1): 29- 38. doi: 10.3979/j.issn.1673-825X.202212230376
	XIA Y, CHEN H. Trajectory classification network fusing Transformer and convolutional LSTM. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2024, 36(1): 29- 38. doi: 10.3979/j.issn.1673-825X.202212230376
4	郑雅洲, 刘万平, 黄东. 一种基于注意力机制的BERT-CNN-GRU检测方法. 计算机工程, 2025, 51(1): 258- 268. doi: 10.19678/j.issn.1000-3428.0068479
	ZHENG Y Z, LIU W P, HUANG D. A BERT-CNN-GRU detection method based on attention mechanism. Computer Engineering, 2025, 51(1): 258- 268. doi: 10.19678/j.issn.1000-3428.0068479
5	YAN A, CHENG S, KANG W C, et al. CosRec: 2D convolutional neural networks for sequential recommendation[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 2173-2176.
6	XU C F, ZHAO P P, LIU Y C, et al. Recurrent convolutional neural network for sequential recommendation[C]//Proceedings of the World Wide Web Conference. New York, USA: ACM Press, 2019: 3398-3404.
7	王昂, 何小海, 罗丹, 等. 针对VVC色度预测的注意力卷积神经网络算法. 电讯技术, 2024, 64(11): 1741- 1749. doi: 10.20079/j.issn.1001-893x.240125001
	WANG A, LUO X H, LUO D, et al. Chroma prediction for VVC using CNN with attention mechanism. Telecommunication Engineering, 2024, 64(11): 1741- 1749. doi: 10.20079/j.issn.1001-893x.240125001
8	唐宏, 金哲正, 张静, 等. 融合时间感知和多兴趣提取网络的序列推荐. 重庆邮电大学学报(自然科学版), 2024, 36(4): 807- 818.
	TANG H, JIN Z Z, ZHANG J, et al. Fusing time-aware and multi-interest extraction network for sequential recommendation. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2024, 36(4): 807- 818.
9	WU L W, LI S Q, HSIEH C J, et al. SSE-PT: sequential recommendation via personalized transformer[C]//Proceedings of the 14th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2020: 328-337.
10	HUANG L W, MA Y T, LIU Y B, et al. Position-enhanced and time-aware graph convolutional network for sequential recommendations. ACM Transactions on Information Systems, 2023, 41(1): 1- 32. doi: 10.1145/3511700
11	ZHANG Y H, YANG B, LIU H D, et al. A time-aware self-attention based neural network model for sequential recommendation. Applied Soft Computing, 2023, 133, 109894. doi: 10.1016/j.asoc.2022.109894
12	WANG R Q, LOU J G, JIANG Y L. Session-based recommendation with time-aware neural attention network. Expert Systems with Applications, 2022, 210, 118395. doi: 10.1016/j.eswa.2022.118395
13	CHAE D K, KANG J S, KIM S W, et al. CFGAN: a generic collaborative filtering framework based on generative adversarial networks[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2018: 137-146.
14	WANG J, YU L T, ZHANG W N, et al. IRGAN: a minimax game for unifying generative and discriminative information retrieval models[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2017: 515-524.
15	CHEN X S, LI S, LI H, et al. Generative adversarial user model for reinforcement learning based recommendation system[C]//Proceedings of the International Conference on Machine Learning. [S. l.]: PMLR, 2019: 1052-1061.
16	REN R Y, LIU Z Y, LI Y L, et al. Sequential recommendation with self-attentive multi-adversarial network[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2020: 89-98.
17	HINTON G E, SRIVASTAVA N, KRIZHEVSKY A. Improving neural networks by preventing co-adaptation of feature detectors[EB/OL]. [2024-10-12]. https://arxiv.org/abs/1207.0580.
18	WU L W, LI S Q, HSIEH C J, et al. Stochastic shared embeddings: data-driven regularization of embedding layers[EB/OL]. [2024-10-12]. https://arxiv.org/abs/1905.10630.
19	PAN X R, GE C J, LU R, et al. On the integration of self-attention and convolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE Press, 2022: 805-815.
20	KANG W C, MCAULEY J. Self-attentive sequential recommendation[C]//Proceedings of the IEEE International Conference on Data Mining (ICDM). Singapore: IEEE Press, 2018: 197-206.
21	SUN F, LIU J, WU J, et al. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 1441-1450.
22	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference on North American Chapter of the Association for Computational Linguistics. [S. l.]: ACL, 2019: 3498-4195.
23	HUANG L W, FU M S, LI F, et al. A deep reinforcement learning based long-term recommender system. Knowledge-Based Systems, 2021, 213, 106706. doi: 10.1016/j.knosys.2020.106706
24	ZHAO X Y, XIA L, ZOU L X, et al. Deep reinforcement learning for whole-chain recommendations[EB/OL]. [2024-10-12]. https://arxiv.org/abs/1902.03987.
25	IZMAILOV P, PODOPRIKHIN D, GARIPOV T, et al. Averaging weights leads to wider optima and better generalization[EB/OL]. [2024-10-12]. https://arxiv.org/abs/1803.05407.
26	BOZIC V, DORDEVIC D, COPPOLA D, et al. Rethinking attention: exploring shallow feed-forward neural networks as an alternative to attention layers in transformers[EB/OL]. [2024-10-12]. https://arxiv.org/abs/2311.10642.
27	TANG J X, WANG K. Personalized top-N sequential recommendation via convolutional sequence embedding[C]//Proceedings of the 11th ACM International Conference on Web Search and Data Mining. New York, USA: ACM Press, 2018: 565-573.
28	HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks[EB/OL]. [2024-10-12]. https://arxiv.org/abs/1511.06939.
29	MA C, KANG P, LIU X. Hierarchical gating networks for sequential recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2019: 825-833.
30	FAN Z W, LIU Z W, WANG S, et al. Modeling sequences as distributions with uncertainty for sequential recommendation[C]//Proceedings of the 30th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2021: 3019-3023.
31	HOU Y P, HU B B, ZHANG Z Q, et al. CORE: simple and effective session-based recommendation within consistent representation space[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2022: 1796-1801.
32	DU X Y, YUAN H H, ZHAO P P, et al. Frequency enhanced hybrid attention network for sequential recommendation[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2023: 78-88.
33	CHEN J Y, ZOU G X, ZHOU P, et al. Sparse enhanced network: an adversarial generation method for robust augmentation in sequential recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(8): 8283- 8291. doi: 10.1609/aaai.v38i8.28669
34	WANG J T, RATHI P, SUNDARAM H. A pre-trained zero-shot sequential recommendation framework via popularity dynamics[C]//Proceedings of the 18th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2024: 433-443.

[1]	HE Mingyan, LI Siyuan, LIU Peng, HUANG Jianhua. Opponent Modeling Based on Cross-Attention Diffusion Model [J]. Computer Engineering, 2026, 52(7): 76-85.
[2]	LIU Xiangbin, FANG Cheng, LIU Shuai. Real-Time Semantic Segmentation Algorithm Based on Multi-Scale Feature Fusion and Polarized Self-Attention Mechanism [J]. Computer Engineering, 2026, 52(7): 210-219.
[3]	YANG Xinyi, HE Xiaohai, TENG Qizhi, CHEN Honggang, QING Linbo, WU Xiaohong. Cross-Modal Named Entity Recognition Based on Semantic Fusion [J]. Computer Engineering, 2026, 52(7): 345-353.
[4]	ZHENG Yuhao, LI Jianhua. Initial Phase Training Method of Class Incremental Learning Based on Class Boundary Sample Generation and Fine-Tuning [J]. Computer Engineering, 2026, 52(7): 86-93.
[5]	JIN Jing, HU Chudi, CHEN Gang. Medical Image Segmentation Method Based on Multi-Scale Self-Attention Transformer [J]. Computer Engineering, 2026, 52(7): 199-209.
[6]	JIA Junjie, LI Tianle, LIU Shilong. Multi-View Feature Fusion Method for Shilling Attack Detection [J]. Computer Engineering, 2026, 52(7): 302-312.
[7]	ZHANG Hong, YANG Junyi. Spatio-Temporal Generative Adversarial Clustered Graph Convolutional Network for Anomalous Traffic Flow Prediction [J]. Computer Engineering, 2026, 52(7): 168-178.
[8]	LI Jinru, PAN Qingxian, GAO Zhaolong, WEI Kai, FAN Zequn. Crowdsourcing Task Recommendation Method Based on Collaborative Knowledge Graph and Hybrid Neural Network [J]. Computer Engineering, 2026, 52(7): 155-167.
[9]	LIANG Ziyi, WANG Zihao, LIU Tianquan, LI Liping, ZHU Yuanfei, LU Cunyue. PASRGAN: A Super-Resolution Pathological Image Generation Algorithm Based on Parallel Attention Mechanism [J]. Computer Engineering, 2026, 52(7): 189-198.
[10]	ZHOU Lijun, ZHANG Junran, WANG Kaiyuan, XIANG Junlian. Drug Recommendation Based on Graph-Enhanced Attention Integrating Clinical Features of Patients [J]. Computer Engineering, 2026, 52(6): 314-325.
[11]	DAI Yinqiao, XIAO Wulong, LI Bailin, LI Li. Lettuce Core Detection Algorithm Based on Improved YOLOv5s [J]. Computer Engineering, 2026, 52(6): 352-364.
[12]	ZENG An, ZHENG Jiayu, PAN Dan, ZHAO Jingliang, HUANG Xingqing. Centerline Tracking Algorithm for Aortic Dissection Based on Deep Reinforcement Learning [J]. Computer Engineering, 2026, 52(6): 414-424.
[13]	YU Mengyuan, LIU Xiangyang. Ship Detection Method Based on Multimodal Visible and Infrared Image Fusion [J]. Computer Engineering, 2026, 52(6): 278-287.
[14]	LUO Heng, WAN Liang. Network Traffic Intrusion Detection Method Based on Dynamic Spatio-Temporal Graph Neural Network [J]. Computer Engineering, 2026, 52(6): 202-213.
[15]	XIAO Zeqiu, LI Yong, WANG Xia. Prediction of Blood Glucose Concentration in Diabetic Patients Based on PBI-CLA Model [J]. Computer Engineering, 2026, 52(6): 382-390.

Please choose a citation manager

Content to export