基于用户多类型反馈行为序列的点击率预估模型

doi:10.19678/j.issn.1000-3428.0068484

摘要/Abstract

摘要：

在推荐系统中, 现有的点击率预估模型通常采用用户近期点击过的行为序列作为模型的输入, 这将使模型难以得到全面的用户兴趣表示, 导致模型无法获得最优的精度。为了解决这个问题, 引入一个基于用户多类型反馈行为序列的点击率预估模型(UMFB)。该模型中多种类型的用户行为序列包括隐式反馈序列和显式反馈序列, 并且能够对不同的用户兴趣偏好进行建模。鉴于隐式反馈序列中包含大量的噪声, 结合基于傅里叶变换的兴趣去噪层来减轻干扰。此外, 为了解决显式反馈序列数据的稀疏性问题, 部署基于对比学习的兴趣增强层来提高建模效果。最后采用个性化兴趣融合层对用户的偏好进行建模。为了验证UMFB模型的有效性, 在短视频推荐领域的KuaiRand-Pure和KuaiRand-1K数据集上进行了对比实验, 结果表明, 与DMT基线模型相比, UMFB模型的AUC分别提高了1.07和0.91个百分点。

Abstract:

In recommendation systems, models for predicting click-through rates typically rely on a user's recent clicks as input. However, this approach can fall short in fully capturing user interests, limiting the model's accuracy. To address this issue, a new click-through rate estimation model based on User Multi-Type Feedback Behavior (UMFB) is developed, designed to handle various types of user feedback sequences. The UMFB model incorporates both implicit and explicit feedback sequences, enabling it to capture diverse user interest preferences. Given that implicit feedback sequences often contain significant noise, the study introduced an interest-denoising layer based on Fourier transform to reduce interference. Furthermore, to address the data sparsity issue in explicit feedback sequences, an interest enhancement layer based on contrastive learning is implemented to improve modeling accuracy. Finally, a personalized interest fusion layer is utilized to effectively model user preferences. To evaluate the effectiveness of the UMFB model, extensive experiments are conducted on the KuaiRand-Pure and KuaiRand-1K datasets in the context of short video recommendations. The results demonstrated that the UMFB model significantly outperformed other state-of-the-art baseline models, with an Area Under the receiver operator characteristic Curve (AUC) improvement of 1.07 and 0.91 percentage points on the respective real datasets.

Key words: recommendation system, click-through rate estimation, behavior sequence modeling, multi-type behavioral sequence, contrastive learning

吴永庆, 王钰涵, 朱月. 基于用户多类型反馈行为序列的点击率预估模型[J]. 计算机工程, 2024, 50(10): 405-417.

WU Yongqing, WANG Yuhan, ZHU Yue. Click-Through Rate Estimation Model Based on User Multi-Type Feedback Behavior Sequences[J]. Computer Engineering, 2024, 50(10): 405-417.

https://www.ecice06.com/CN/Y2024/V50/I10/405

图/表 13

图1 基于用户多类型反馈行为序列的点击率预估模型

Fig.1 Click-through rate estimation model based on user multi-type feedback behavior sequences

图2 基于傅里叶变换的兴趣去噪模块

Fig.2 Fourier transform based interest denoising module

图3 显式序列增强

Fig.3 Explicit sequence enhancement

图4 基于傅里叶变换的兴趣去噪层对模型性能的影响

Fig.4 Impact of Fourier transform-based interest denoising layer on model performance

图5 基于对比学习的兴趣增强层对模型性能的影响

Fig.5 Impact of interest enhancement layer based on comparative learning on model performance

图6 个性化兴趣融合层对模型性能的影响

Fig.6 Impact of personalized interest fusion layer on model performance

图7 对比损失权重对模型性能的影响

Fig.7 Effect of comparison loss weights on model performance

图8 去噪网络层数对模型性能的影响

Fig.8 Effect of the number of denoising network layers on model performance

图9 行为序列长度对模型性能的影响

Fig.9 Effect of the behavior sequence length on model performance

图10 Batchsize对模型性能的影响

Fig.10 Effect of the Batchsize on model performance

参考文献 33

1	LIN Z H, WANG H, MAO J S, et al. Feature-aware diversified re-ranking with disentangled representations for relevant recommendation[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2022: 3327-3335.
2	COVINGTON P, ADAMS J, SARGIN E. Deep neural networks for YouTube recommendations[C]//Proceedings of the 10th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2016: 191-198.
3	CAI Q P, LIU S C, WANG X L, et al. Reinforcing user retention in a billion scale short video recommender system[C]//Proceedings of ACM Web Conference. New York, USA: ACM Press, 2023: 368-379.
4	ZHOU G R, MOU N, FAN Y, et al. Deep interest evolution network for click-through rate prediction. Artificial Intelligence, 2019, 33(1): 5941- 5948.
5	ZHOU G R, ZHU X Q, SONG C R, et al. Deep interest network for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2018: 1059-1068.
6	汤佳欣, 陈阳, 周孟莹, 等. 深度学习方法在兴趣点推荐中的应用研究综述. 计算机工程, 2022, 48(1): 12-23, 42. URL
	TANG J X, CHEN Y, ZHOU M Y, et al. A survey of studies on deep learning applications in POI recommendation. Computer Engineering, 2022, 48(1): 12-23, 42. URL
7	CHENG H T, KOC L, HARMSEN J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. New York, USA: ACM Press, 2016: 7-10.
8	HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks[EB/OL]. [2023-08-20]. http://arxvi.org/abs/1511.06939.
9	FENG Y F, LÜ F Y, SHEN W C, et al. Deep session interest network for click-through rate prediction[EB/OL]. [2023-08-20]. http://arxvi.org/abs/1905.06482.
10	CHEN Q, PEI C, LÜ S, et al. End-to-end user behavior retrieval in click-through rateprediction model[EB/OL]. [2023-08-20]. http://arxvi.org/abs/2108.04468.
11	钟川, 陈军. 基于精确欧氏局部敏感哈希的改进协同过滤推荐算法. 计算机工程, 2017, 43(2): 74- 78. URL
	ZHONG C, CHEN J. Improved collaborative filtering recommendation algorithm based on exact Euclidean locality sensitive Hashing. Computer Engineering, 2017, 43(2): 74- 78. URL
12	陈碧毅, 黄玲, 王昌栋, 等. 融合显式反馈与隐式反馈的协同过滤推荐算法. 软件学报, 2020, 31(3): 794- 805.
	CHEN B Y, HUANG L, WANG C D, et al. Explicit and implicit feedback based collaborative filtering algorithm. Journal of Software, 2020, 31(3): 794- 805.
13	王东, 陈志, 岳文静, 等. 基于显式与隐式反馈信息的概率矩阵分解推荐. 计算机应用, 2015, 35(9): 2574-2578, 2601.
	WANG D, CHEN Z, YUE W J, et al. Probabilistic matrix factorization recommendation with explicit and implicit feedback. Journal of Computer Applications, 2015, 35(9): 2574-2578, 2601.
14	LIU N N, XIANG E W, ZHAO M, et al. Unifying explicit and implicit feedback for collaborative filtering[C]//Proceedings of the 19th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2010: 1445-1448.
15	XIE R B, LING C, WANG Y L, et al. Deep feedback network for recommendation[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2020: 2519-2525.
16	WU C H, WU F Z, QI T, et al. FeedRec: news feed recommendation with various user feedbacks[C]//Proceedings of ACM Web Conference. New York, USA: ACM Press, 2022: 2088-2097.
17	邢玉莹, 夏鸿斌, 王涵. 缺失数据建模的改进型ALS在线推荐算法. 计算机工程, 2018, 44(8): 212-217, 223. URL
	XING Y Y, XIA H B, WANG H. Improved ALS online recommendation algorithm with missing data modeling. Computer Engineering, 2018, 44(8): 212-217, 223. URL
18	XU J, WANG C, WU C, et al. Multi-behavior self-supervised learning for recommendation[EB/OL]. [2023-08-20]. http://arxvi.org/abs/2305.18238.
19	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of NIPS'17. Cambridge, USA: MIT Press, 2017: 30-38.
20	PI Q, ZHOU G, ZHANG Y, et al. Search-based user interest modeling with lifelong sequenti-al behavior data for click-through rate prediction[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York, USA: ACM Press, 2020: 2685-2692.
21	GU Y L, DING Z Y, WANG S Q, et al. Deep multifaceted transformers for multi-objective ranking in large-scale E-commerce recommender systems[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York, USA: ACM Press, 2020: 2493-2500.
22	ZHAO Z, HONG L C, WEI L, et al. Recommending what video to watch next: a multitask ranking system[C]//Proceedings of the 13th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2019: 43-51.
23	XIA L H, XU Y, HUANG C, et al. Graph meta network for multi-behavior recommendation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2021: 757-766.
24	SUN F, LIU J, WU J, et al. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 1441-1450.
25	XIE X, SUN F, LIU Z Y, et al. Contrastive learning for sequential recommendation[C]//Proceedings of the 38th IEEE International Conference on Data Engineering. Washington D. C., USA: IEEE Press, 2022: 1259-1273.
26	QIU R H, HUANG Z, YIN H Z, et al. Contrastive learning for representation degeneration problem in sequential recommendation[C]//Proceedings of the 15th ACM International Conference on Web Search and Data Mining. New York, USA: ACM Press, 2022: 813-823.
27	高萍, 杨宇晓. 基于三层加权分数阶傅里叶变换的安全通信系统. 电讯技术, 2022, 62(11): 1622- 1628.
	GAO P, YANG Y X. A safe communication system based on three-layer weighted fractional Fourier transform. Telecommunication Engineering, 2022, 62(11): 1622- 1628.
28	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
29	GUO H F, TANG R M, YE Y M, et al. DeepFM: a factorization-machine based neural network for CTR prediction[EB/OL]. [2023-08-20]. http://arxvi.org/abs/1703.04247.
30	WANG R X, SHIVANNA R, CHENG D, et al. DCN V2: improved deep & cross network and practical lessons for web-scale learning to rank systems[C]//Proceedings of Web Conference. New York, USA: ACM Press, 2021: 1785-797.
31	SONG W, SHI C, XIAO Z, et al. Autoint: automatic feature interaction learning via self-at-tentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 1161-1170.
32	HUANG T W, ZHANG Z Q, ZHANG J L. FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction[C]//Proceedings of the 13th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2019: 169-177.
33	BIAN Z, ZHOU S J, FU H, et al. Denoising user-aware memory network for recommendation[C]//Proceedings of the 15th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2021: 400-410.

[1]	高爽, 史轶伦, 徐巧枝, 于磊. 基于对比学习的非对称编解码结构的心脏MRI分割研究[J]. 计算机工程, 2024, 50(8): 290-300.
[2]	钱来, 赵卫伟. 基于对比学习和注意力机制的文本分类方法[J]. 计算机工程, 2024, 50(7): 104-111.
[3]	刘娟, 段友祥, 陆誉翕, 张鲁. 引入知识增强和对比学习的知识图谱补全[J]. 计算机工程, 2024, 50(7): 112-122.
[4]	武星, 殷浩宇, 姚骏峰, 李卫民, 钱权. 面向视频数据的多模态情感分析[J]. 计算机工程, 2024, 50(6): 218-227.
[5]	张宝鑫, 杨丹, 聂铁铮, 寇月. 基于自监督的多视角图协同过滤推荐方法[J]. 计算机工程, 2024, 50(5): 100-110.
[6]	李田芳, 普园媛, 赵征鹏, 徐丹, 钱文华. 基于CLIP和双空间自适应归一化的图像翻译[J]. 计算机工程, 2024, 50(5): 229-240.
[7]	张斯力, 李梓健, 蔡瑞初, 郝志峰, 闫玉光. 基于因果机制约束的强化推荐系统[J]. 计算机工程, 2024, 50(5): 279-290.
[8]	张洪程, 李林育, 杨莉, 伞晨峻, 尹春林, 颜冰, 于虹, 张璇. 基于对比学习与语言模型增强嵌入的知识图谱补全[J]. 计算机工程, 2024, 50(4): 168-176.
[9]	李政学, 李枝名, 彭德中, 陈杰. 基于特征对比学习和图卷积的社交网络用户分类[J]. 计算机工程, 2024, 50(4): 258-266.
[10]	吴冠荣, 李元祥, 王艺霖, 陆雨寒, 陈秀华. 基于对比学习的小样本金属表面损伤分类[J]. 计算机工程, 2024, 50(3): 36-43.
[11]	杨兴耀, 马帅, 张祖莲, 于炯, 陈嘉颖, 王东晓. 基于偏好感知的去噪图卷积网络社交推荐[J]. 计算机工程, 2024, 50(10): 154-163.
[12]	吴志强, 解庆, 李琳, 刘永坚. 基于多模态融合的图神经网络推荐算法[J]. 计算机工程, 2024, 50(1): 91-100.
[13]	顾嘉静, 杨丹, 聂铁铮, 寇月. 基于多视图融合跨层对比学习的推荐算法[J]. 计算机工程, 2024, 50(1): 120-128.
[14]	曹发鑫, 孙媛媛, 王治政, 潘丁豪, 林鸿飞. 面向借贷案件的相似案例匹配模型[J]. 计算机工程, 2024, 50(1): 306-312.
[15]	唐彦, 卢镘旭. 基于知识图谱与深度涟漪网络的推荐系统[J]. 计算机工程, 2023, 49(5): 63-72,80.

选择文件类型/文献管理软件名称

选择包含的内容