基于域内特征间相似性的点击率预估优化

doi:10.19678/j.issn.1000-3428.0064164

计算机工程 ›› 2023, Vol. 49 ›› Issue (2): 238-245. doi: 10.19678/j.issn.1000-3428.0064164

基于域内特征间相似性的点击率预估优化

雷李想^1,2, 武志昊^1,2,4, 刘钰^1,2,4, 周子站^3,4

1. 北京交通大学计算机与信息技术学院, 北京 100044;
2. 北京交通大学网络科学与智能系统研究所, 北京 100044;
3. 中国民航信息网络股份有限公司, 北京 101318;
4. 民航旅客服务智能化应用技术重点实验室, 北京 101318

收稿日期:2022-03-14 修回日期:2022-04-23 发布日期:2023-02-13
作者简介:雷李想(1997-),男,硕士研究生,主研方向为数据挖掘;武志昊,副教授、博士;刘钰,讲师、博士;周子站(通信作者),工程师、硕士。
基金资助:
国家自然科学基金（61603028）；中国民航信息网络股份有限公司和民航旅客服务智能化应用技术重点实验室基金项目（K20L00070）。

Click-Through Rate Prediction and Optimization Based on Intra-Field Features Similarity

LEI Lixiang^1,2, WU Zhihao^1,2,4, LIU Yu^1,2,4, ZHOU Zizhan^3,4

1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China;
2. Institute of Network Science and Intelligent System, Beijing Jiaotong University, Beijing 100044, China;
3. TravelSky Technology Limited, Beijing 101318, China;
4. Key Laboratory of Intelligent Application Technology for Civil Aviation Passenger Services, Beijing 101318, China

Received:2022-03-14 Revised:2022-04-23 Published:2023-02-13

摘要/Abstract

摘要： 基于深度学习的点击率预估模型多数通过建模各个域的特征之间的交互关系提升预估准确率。特征嵌入向量对模型效果具有重要影响，而现有的CTR模型中不同特征的嵌入向量学习过程相互独立，且由于特征长尾分布导致大部分低频特征不能学习到较好的向量表示，严重影响模型的预测效果。基于域内特征间存在隐含的相似性，提出两种分别基于特征间共现概率和游走概率的相似度定义和对应的相似性图构建方法，并给出结合剪枝策略的广度优先遍历算法实现相似特征的高效计算。在此基础上，基于域内特征相似性图，设计一种嵌入生成器，对于低频特征，在域内特征相似性图上通过图神经网络聚合与其相似的特征信息，生成新的特征嵌入，作为预处理过程对特征嵌入向量进行数据增强，提升嵌入向量的表示学习质量。在公开数据集Criteo、Avazu上的实验结果表明，该方法明显提升点击率预估模型的预测准确率，其中对代表性点击率预估模型xDeepFM和AutoInt，AUC指标分别提升了0.007和0.008，LogLoss则下降了0.009和0.006，证明了嵌入生成模型的有效性。

关键词: 点击率预估, 稀疏特征, 特征嵌入, 特征相似性, 图神经网络

Abstract: There exist numerous deep learning-based Click-Through Rate (CTR) models;however, most of them improve prediction accuracy by modeling feature interaction between different fields.The feature embedding vectors have a significant impact on model performance; existing CTR models independently learn the embedding vectors of different features in a field.Consequently, most low-frequency features cannot attain sufficiently good embeddings because of the long-tail feature distribution, which seriously affects model accuracy.Noticing that implicit similarity exists between features of the same field, this study proposes a two intra-field feature similarity based on co-occurrence probability and walk probability, respectively.Moreover, the proposed method develops the corresponding similarity graph construction method and designs a breadth first traversal algorithm combined with pruning strategy to efficiently calculate similar features.Based on the intra-field feature similarity graph, an embedding generator is also proposed.For low-frequency features, information of similar features is aggregated on the similarity graph through a graph neural network.This data augmentation method is used as a preprocessing step to improve the learning quality of feature embedding vectors.Extensive experiments conducted on the public data sets Criteo and Avazu demonstrate that the proposed method improves the prediction accuracy of several classical CTR models.Regarding the representative CTR models xDeepFM and AutoInt, the AUC is increased by 0.007 and 0.008, respectively, while the LogLoss is decreased by 0.009 and 0.006, respectively, which proves the effectiveness of the embedding generator.

Key words: Click-Through Rate(CTR) prediction, sparse feature, feature embedding, feature similarity, graph neural network

中图分类号:

TP391

雷李想, 武志昊, 刘钰, 周子站. 基于域内特征间相似性的点击率预估优化[J]. 计算机工程, 2023, 49(2): 238-245.

LEI Lixiang, WU Zhihao, LIU Yu, ZHOU Zizhan. Click-Through Rate Prediction and Optimization Based on Intra-Field Features Similarity[J]. Computer Engineering, 2023, 49(2): 238-245.

https://www.ecice06.com/CN/Y2023/V49/I2/238

图/表 9

20230216181842

20230216181938

20230216181942

20230216181946

20230216181949

20230216181952

20230216181956

20230216182000

20230216182004

参考文献

[1] GRUNER R L, VOMBERG A, HOMBURG C, et al.Supporting new product launches with social media communication and online advertising:sales volume and profit implications[J].Journal of Product Innovation Management, 2019, 36(2):172-195.
[2] 许王昊, 肖秦琨.基于注意力机制的兴趣网络点击率预估模型[J].计算机工程, 2021, 47(1):101-108. XU W H, XIAO Q K.Click-through rate prediction model of interest network based on attention mechanism[J].Computer Engineering, 2021, 47(1):101-108.(in Chinese)
[3] RENDLE S.Factorization machines[C]//Proceedings of 2010 IEEE International Conference on Data Mining.Washington D.C., USA:IEEE Press, 2010:995-1000.
[4] MIKOLOV T, CHEN K, CORRADO G, et al.Efficient estimation of word representations in vector space[EB/OL].[2021-10-21].https://arxiv.org/abs/1301.3781.
[5] YANG Y W.Click-through rate prediction in online advertising:a literature review[J].Information Processing & Management, 2022, 59(2):102853.
[6] YANG H X.Targeted search and the long tail effect[J].The RAND Journal of Economics, 2013, 44(4):733-756.
[7] PAN F Y, LI S K, AO X, et al.Warm up cold-start advertisements:improving CTR predictions via learning to learn ID embeddings[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York, USA:ACM Press, 2019:695-704.
[8] FINN C, ABBEEL P, LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning.Washington D.C., USA:IEEE Press, 2017:1126-1135.
[9] SAMEK W, MONTAVON G, LAPUSCHKIN S, et al.Explaining deep neural networks and beyond:a review of methods and applications[J].Proceedings of the IEEE, 2021, 109(3):247-278.
[10] CHENG H T, KOC L, HARMSEN J, et al.Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.Washington D.C., USA:IEEE Press, 2016:7-10.
[11] HEATON J.An empirical analysis of feature engineering for predictive modeling[C]//Proceedings of SoutheastCon'16.Washington D.C., USA:IEEE Press, 2016:1-6.
[12] GUO H F, TANG R M, YE Y M, et al.DeepFM:a factorization-machine based neural network for CTR prediction[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.Washington D.C., USA:IEEE Press, 2017:536-548.
[13] HE X N, CHUA T S.Neural factorization machines for sparse predictive analytics[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval.New York, USA:ACM Press, 2017:355-364.
[14] LIAN J X, ZHOU X H, ZHANG F Z, et al.xDeepFM:combining explicit and implicit feature interactions for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York, USA:ACM Press, 2018:1754-1763.
[15] SONG W P, SHI C C, XIAO Z P, et al.AutoInt:automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.New York, USA:ACM Press, 2019:1161-1170.
[16] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[EB/OL].[2021-10-01].https://arxiv.org/abs/1706.03762.
[17] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[18] LI Z K, CUI Z Y, WU S, et al.Fi-GNN:modeling feature interactions via graph neural networks for CTR prediction[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.New York, USA:ACM Press, 2019:539-548.
[19] WU Z H, PAN S R, CHEN F W, et al.A comprehensive survey on graph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1):4-24.
[20] KIPF T N, WELLING M.Semi-supervised classification with graph convolutional networks[EB/OL].[2021-10-01].https://arxiv.org/abs/1609.02907.
[21] 黄伟, 冯晶晶, 黄遥.基于多通道极深卷积神经网络的图像超分辨率算法[J].计算机工程, 2020, 46(9):242-247, 253. HUANG W, FENG J J, HUANG Y.Super-resolution algorithm for images based on multi-channel extremely deep convolutional neural network[J].Computer Engineering, 2020, 46(9):242-247, 253.(in Chinese)
[22] VELICKOVIC P, CUCURULL G, CASANOVA A, et al.Graph attention networks[EB/OL].[2022-02-10].https://arxiv.org/abs/1710.10903.
[23] RONG Y, HUANG W, XU T, et al.Dropedge:towards deep graph convolutional networks on node classification[EB/OL].[2021-02-10].https://arxiv.org/abs/1907.10903.
[24] ZHANG M H, CHEN Y X.Link prediction based on graph neural networks[EB/OL].[2022-02-10].https://arxiv.org/abs/1802.09691.
[25] MAAS A L.Rectifier nonlinearities improve neural network acoustic models[C]//Proceedings of ICML'13.Washington D.C., USA:IEEE Press, 2013:296-308.
[26] RUMELHART D E, HINTON G E, WILLIAMS R J.Learning representations by back-propagating errors[J].Nature, 1986, 323(6088):533-536.

选择文件类型/文献管理软件名称

选择包含的内容

基于域内特征间相似性的点击率预估优化

Click-Through Rate Prediction and Optimization Based on Intra-Field Features Similarity

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	何杏宇, 周易歆, 罗东旭, 杨桂松. 基于图神经网络和多主体评价的教学资源推荐[J]. 计算机工程, 2024, 50(7): 13-22.
[2]	张正康, 杨丹, 聂铁铮, 寇月. 基于图结构聚类的自监督学习疾病诊断方法[J]. 计算机工程, 2024, 50(7): 360-371.
[3]	孙文洁, 李宗民, 孙浩淼. 基于图神经网络的多智能体强化学习值函数分解方法[J]. 计算机工程, 2024, 50(5): 62-70.
[4]	游奔, 李晓红, 姚锦, 冯绍杰. 基于多粒度图与注意力机制的半监督短文本分类[J]. 计算机工程, 2024, 50(5): 83-90.
[5]	张宝鑫, 杨丹, 聂铁铮, 寇月. 基于自监督的多视角图协同过滤推荐方法[J]. 计算机工程, 2024, 50(5): 100-110.
[6]	吴正洋, 张广涛, 黄立, 汤庸. 基于异质图嵌入和会话交互的课程推荐模型[J]. 计算机工程, 2024, 50(4): 95-103.
[7]	王华珍, 许泽, 孙悦, 丘斌, 陈坚, 邱强斌. 基于事件演化图的多标记事件预测模型[J]. 计算机工程, 2024, 50(4): 132-140.
[8]	王俊, 赖会霞, 万玥, 张仕. 基于角度的图神经网络高维数据异常检测方法[J]. 计算机工程, 2024, 50(3): 156-165.
[9]	王柏涵, 姜晓燕, 范柳伊. 基于深度监督隐空间构建的语义分割改进方法[J]. 计算机工程, 2024, 50(3): 191-199.
[10]	李伟健, 胡慧君. 基于潜在特征增强网络的视频描述生成方法[J]. 计算机工程, 2024, 50(2): 266-272.
[11]	吴志强, 解庆, 李琳, 刘永坚. 基于多模态融合的图神经网络推荐算法[J]. 计算机工程, 2024, 50(1): 91-100.
[12]	刘昀抒, 申彦明, 齐恒, 尹宝才. 基于层次结构图的多跳知识图谱问答模型[J]. 计算机工程, 2024, 50(1): 101-109.
[13]	顾嘉静, 杨丹, 聂铁铮, 寇月. 基于多视图融合跨层对比学习的推荐算法[J]. 计算机工程, 2024, 50(1): 120-128.
[14]	刘晓黎, 王轶彤. 基于自监督学习的多密度图会话推荐[J]. 计算机工程, 2023, 49(9): 60-68, 78.
[15]	隋国华, 李陶然, 刘昊, 陈林, 汪卫. 基于图表示学习的领域知识图谱推理技术研究[J]. 计算机工程, 2023, 49(9): 89-98.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于域内特征间相似性的点击率预估优化

Click-Through Rate Prediction and Optimization Based on Intra-Field Features Similarity

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 15

编辑推荐

Metrics

本文评价