Click-Through Rate Estimation Model Based on User Multi-Type Feedback Behavior Sequences

doi:10.19678/j.issn.1000-3428.0068484

Abstract

Abstract:

In recommendation systems, models for predicting click-through rates typically rely on a user's recent clicks as input. However, this approach can fall short in fully capturing user interests, limiting the model's accuracy. To address this issue, a new click-through rate estimation model based on User Multi-Type Feedback Behavior (UMFB) is developed, designed to handle various types of user feedback sequences. The UMFB model incorporates both implicit and explicit feedback sequences, enabling it to capture diverse user interest preferences. Given that implicit feedback sequences often contain significant noise, the study introduced an interest-denoising layer based on Fourier transform to reduce interference. Furthermore, to address the data sparsity issue in explicit feedback sequences, an interest enhancement layer based on contrastive learning is implemented to improve modeling accuracy. Finally, a personalized interest fusion layer is utilized to effectively model user preferences. To evaluate the effectiveness of the UMFB model, extensive experiments are conducted on the KuaiRand-Pure and KuaiRand-1K datasets in the context of short video recommendations. The results demonstrated that the UMFB model significantly outperformed other state-of-the-art baseline models, with an Area Under the receiver operator characteristic Curve (AUC) improvement of 1.07 and 0.91 percentage points on the respective real datasets.

Key words: recommendation system, click-through rate estimation, behavior sequence modeling, multi-type behavioral sequence, contrastive learning

摘要：

在推荐系统中, 现有的点击率预估模型通常采用用户近期点击过的行为序列作为模型的输入, 这将使模型难以得到全面的用户兴趣表示, 导致模型无法获得最优的精度。为了解决这个问题, 引入一个基于用户多类型反馈行为序列的点击率预估模型(UMFB)。该模型中多种类型的用户行为序列包括隐式反馈序列和显式反馈序列, 并且能够对不同的用户兴趣偏好进行建模。鉴于隐式反馈序列中包含大量的噪声, 结合基于傅里叶变换的兴趣去噪层来减轻干扰。此外, 为了解决显式反馈序列数据的稀疏性问题, 部署基于对比学习的兴趣增强层来提高建模效果。最后采用个性化兴趣融合层对用户的偏好进行建模。为了验证UMFB模型的有效性, 在短视频推荐领域的KuaiRand-Pure和KuaiRand-1K数据集上进行了对比实验, 结果表明, 与DMT基线模型相比, UMFB模型的AUC分别提高了1.07和0.91个百分点。

WU Yongqing, WANG Yuhan, ZHU Yue. Click-Through Rate Estimation Model Based on User Multi-Type Feedback Behavior Sequences[J]. Computer Engineering, 2024, 50(10): 405-417.

吴永庆, 王钰涵, 朱月. 基于用户多类型反馈行为序列的点击率预估模型[J]. 计算机工程, 2024, 50(10): 405-417.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068484

https://www.ecice06.com/EN/Y2024/V50/I10/405

Figures/Tables 13

Fig.1 Click-through rate estimation model based on user multi-type feedback behavior sequences

Fig.2 Fourier transform based interest denoising module

Fig.3 Explicit sequence enhancement

Fig.4 Impact of Fourier transform-based interest denoising layer on model performance

Fig.5 Impact of interest enhancement layer based on comparative learning on model performance

Fig.6 Impact of personalized interest fusion layer on model performance

Fig.7 Effect of comparison loss weights on model performance

Fig.8 Effect of the number of denoising network layers on model performance

Fig.9 Effect of the behavior sequence length on model performance

Fig.10 Effect of the Batchsize on model performance

References 33

1	LIN Z H, WANG H, MAO J S, et al. Feature-aware diversified re-ranking with disentangled representations for relevant recommendation[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2022: 3327-3335.
2	COVINGTON P, ADAMS J, SARGIN E. Deep neural networks for YouTube recommendations[C]//Proceedings of the 10th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2016: 191-198.
3	CAI Q P, LIU S C, WANG X L, et al. Reinforcing user retention in a billion scale short video recommender system[C]//Proceedings of ACM Web Conference. New York, USA: ACM Press, 2023: 368-379.
4	ZHOU G R, MOU N, FAN Y, et al. Deep interest evolution network for click-through rate prediction. Artificial Intelligence, 2019, 33(1): 5941- 5948.
5	ZHOU G R, ZHU X Q, SONG C R, et al. Deep interest network for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2018: 1059-1068.
6	汤佳欣, 陈阳, 周孟莹, 等. 深度学习方法在兴趣点推荐中的应用研究综述. 计算机工程, 2022, 48(1): 12-23, 42. URL
	TANG J X, CHEN Y, ZHOU M Y, et al. A survey of studies on deep learning applications in POI recommendation. Computer Engineering, 2022, 48(1): 12-23, 42. URL
7	CHENG H T, KOC L, HARMSEN J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. New York, USA: ACM Press, 2016: 7-10.
8	HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks[EB/OL]. [2023-08-20]. http://arxvi.org/abs/1511.06939.
9	FENG Y F, LÜ F Y, SHEN W C, et al. Deep session interest network for click-through rate prediction[EB/OL]. [2023-08-20]. http://arxvi.org/abs/1905.06482.
10	CHEN Q, PEI C, LÜ S, et al. End-to-end user behavior retrieval in click-through rateprediction model[EB/OL]. [2023-08-20]. http://arxvi.org/abs/2108.04468.
11	钟川, 陈军. 基于精确欧氏局部敏感哈希的改进协同过滤推荐算法. 计算机工程, 2017, 43(2): 74- 78. URL
	ZHONG C, CHEN J. Improved collaborative filtering recommendation algorithm based on exact Euclidean locality sensitive Hashing. Computer Engineering, 2017, 43(2): 74- 78. URL
12	陈碧毅, 黄玲, 王昌栋, 等. 融合显式反馈与隐式反馈的协同过滤推荐算法. 软件学报, 2020, 31(3): 794- 805.
	CHEN B Y, HUANG L, WANG C D, et al. Explicit and implicit feedback based collaborative filtering algorithm. Journal of Software, 2020, 31(3): 794- 805.
13	王东, 陈志, 岳文静, 等. 基于显式与隐式反馈信息的概率矩阵分解推荐. 计算机应用, 2015, 35(9): 2574-2578, 2601.
	WANG D, CHEN Z, YUE W J, et al. Probabilistic matrix factorization recommendation with explicit and implicit feedback. Journal of Computer Applications, 2015, 35(9): 2574-2578, 2601.
14	LIU N N, XIANG E W, ZHAO M, et al. Unifying explicit and implicit feedback for collaborative filtering[C]//Proceedings of the 19th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2010: 1445-1448.
15	XIE R B, LING C, WANG Y L, et al. Deep feedback network for recommendation[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2020: 2519-2525.
16	WU C H, WU F Z, QI T, et al. FeedRec: news feed recommendation with various user feedbacks[C]//Proceedings of ACM Web Conference. New York, USA: ACM Press, 2022: 2088-2097.
17	邢玉莹, 夏鸿斌, 王涵. 缺失数据建模的改进型ALS在线推荐算法. 计算机工程, 2018, 44(8): 212-217, 223. URL
	XING Y Y, XIA H B, WANG H. Improved ALS online recommendation algorithm with missing data modeling. Computer Engineering, 2018, 44(8): 212-217, 223. URL
18	XU J, WANG C, WU C, et al. Multi-behavior self-supervised learning for recommendation[EB/OL]. [2023-08-20]. http://arxvi.org/abs/2305.18238.
19	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of NIPS'17. Cambridge, USA: MIT Press, 2017: 30-38.
20	PI Q, ZHOU G, ZHANG Y, et al. Search-based user interest modeling with lifelong sequenti-al behavior data for click-through rate prediction[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York, USA: ACM Press, 2020: 2685-2692.
21	GU Y L, DING Z Y, WANG S Q, et al. Deep multifaceted transformers for multi-objective ranking in large-scale E-commerce recommender systems[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York, USA: ACM Press, 2020: 2493-2500.
22	ZHAO Z, HONG L C, WEI L, et al. Recommending what video to watch next: a multitask ranking system[C]//Proceedings of the 13th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2019: 43-51.
23	XIA L H, XU Y, HUANG C, et al. Graph meta network for multi-behavior recommendation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2021: 757-766.
24	SUN F, LIU J, WU J, et al. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 1441-1450.
25	XIE X, SUN F, LIU Z Y, et al. Contrastive learning for sequential recommendation[C]//Proceedings of the 38th IEEE International Conference on Data Engineering. Washington D. C., USA: IEEE Press, 2022: 1259-1273.
26	QIU R H, HUANG Z, YIN H Z, et al. Contrastive learning for representation degeneration problem in sequential recommendation[C]//Proceedings of the 15th ACM International Conference on Web Search and Data Mining. New York, USA: ACM Press, 2022: 813-823.
27	高萍, 杨宇晓. 基于三层加权分数阶傅里叶变换的安全通信系统. 电讯技术, 2022, 62(11): 1622- 1628.
	GAO P, YANG Y X. A safe communication system based on three-layer weighted fractional Fourier transform. Telecommunication Engineering, 2022, 62(11): 1622- 1628.
28	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
29	GUO H F, TANG R M, YE Y M, et al. DeepFM: a factorization-machine based neural network for CTR prediction[EB/OL]. [2023-08-20]. http://arxvi.org/abs/1703.04247.
30	WANG R X, SHIVANNA R, CHENG D, et al. DCN V2: improved deep & cross network and practical lessons for web-scale learning to rank systems[C]//Proceedings of Web Conference. New York, USA: ACM Press, 2021: 1785-797.
31	SONG W, SHI C, XIAO Z, et al. Autoint: automatic feature interaction learning via self-at-tentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 1161-1170.
32	HUANG T W, ZHANG Z Q, ZHANG J L. FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction[C]//Proceedings of the 13th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2019: 169-177.
33	BIAN Z, ZHOU S J, FU H, et al. Denoising user-aware memory network for recommendation[C]//Proceedings of the 15th ACM Conference on Recommender Systems. New York, USA: ACM Press, 2021: 400-410.

[1]	Shuang GAO, Yilun SHI, Qiaozhi XU, Lei YU. Research on Cardiac MRI Segmentation Based on Asymmetric Encoding and Decoding Structure of Contrastive Learning [J]. Computer Engineering, 2024, 50(8): 290-300.
[2]	Juan LIU, Youxiang DUAN, Yuxi LU, Lu ZHANG. Knowledge Graph Completion with Knowledge Enhancement and Contrastive Learning [J]. Computer Engineering, 2024, 50(7): 112-122.
[3]	Lai QIAN, Weiwei ZHAO. Text Classification Method Based on Contrastive Learning and Attention Mechanism [J]. Computer Engineering, 2024, 50(7): 104-111.
[4]	WU Xing, YIN Haoyu, YAO Junfeng, LI Weimin, QIAN Quan. Multimodal Sentiment Analysis for Video Data [J]. Computer Engineering, 2024, 50(6): 218-227.
[5]	LI Tianfang, PU Yuanyuan, ZHAO Zhengpeng, XU Dan, QIAN Wenhua. Image-to-Image Translation Based on CLIP and Dual-Spatially Adaptive Normalization [J]. Computer Engineering, 2024, 50(5): 229-240.
[6]	ZHANG Sili, LI Zijian, CAI Ruichu, HAO Zhifeng, YAN Yuguang. Reinforcement Recommendation System Based on Causal Mechanism Constraint [J]. Computer Engineering, 2024, 50(5): 279-290.
[7]	ZHANG Baoxin, YANG Dan, NIE Tiezheng, KOU Yue. Recommendation Method Based on Self-supervised Multi-view Graph Collaborative Filtering [J]. Computer Engineering, 2024, 50(5): 100-110.
[8]	LI Zhengxue, LI Zhiming, PENG Dezhong, CHEN Jie. User Classification of Social Networks Based on Feature Contrastive Learning and Graph Convolution [J]. Computer Engineering, 2024, 50(4): 258-266.
[9]	ZHANG Hongchen, LI Linyu, YANG Li, SAN Chenjun, YIN Chunlin, YAN Bing, YU Hong, ZHANG Xuan. Knowledge Graph Completion Based on Contrastive Learning and Language Model-Enhanced Embedding [J]. Computer Engineering, 2024, 50(4): 168-176.
[10]	Guanrong WU, Yuanxiang LI, Yilin WANG, Yuhan LU, Xiuhua CHEN. Few-Shot Metal Surface Defect Classification Based on Contrastive Learning [J]. Computer Engineering, 2024, 50(3): 36-43.
[11]	YANG Xingyao, MA Shuai, ZHANG Zulian, YU Jiong, CHEN Jiaying, WANG Dongxiao. Social Recommendation Based on Preference-Aware Denoising Graph Convolutional Networks [J]. Computer Engineering, 2024, 50(10): 154-163.
[12]	Faxin CAO, Yuanyuan SUN, Zhizheng WANG, Dinghao PAN, Hongfei LIN. Similar Case Matching Model for Lending Cases [J]. Computer Engineering, 2024, 50(1): 306-312.
[13]	Jiajing GU, Dan YANG, Tiezheng NIE, Yue KOU. Recommendation Algorithm Based on Multi-view Fusion Cross-layer Contrastive Learning [J]. Computer Engineering, 2024, 50(1): 120-128.
[14]	Zhiqiang WU, Qing XIE, Lin LI, Yongjian LIU. Graph Neural Network Recommendation Algorithm Based on Multimodal Fusion [J]. Computer Engineering, 2024, 50(1): 91-100.
[15]	TANG Yan, LU Manxu. Recommendation System Based on Knowledge Graph and Deep Ripple Network [J]. Computer Engineering, 2023, 49(5): 63-72,80.

Please choose a citation manager

Content to export