Hybrid Advertising Click-through Rate Prediction Model Based on Feature Enhancement Combination

doi:10.19678/j.issn.1000-3428.0060051

Abstract

Abstract: In the existing models for Click-Through Rate (CTR) prediction, the features interact with each other on a single feature level, and the models usually ignore the effective information of different feature levels.This paper proposes a hybrid model, APNN, based on feature enhancement aggregation for ad CTR prediction.First-order information importance is introduced into the embedding layer of the CTR prediction model for feature enhancement.Then the Attentional Factorization Machine (AFM) model and the Product-based Neural Network (PNN) are used to fuse the interaction features of different feature levels and the enhanced embedding features.On this basis, multiple fully connected layers are used to obtain more potential high-order information from fused features.The experimental results show that compared with AFM, PNN, FNN and other models, the proposed APNN model displays a higher accuracy.Its AUC is 1.74 percentage points higher than that of the PNN model, and its LogLoss is 1.42 percentage points higher on the Criteo dataset.

Key words: Click-Through Rate(CTR) prediction, first-order information importance, feature enhancement, Factorization Machine(FM), Deep Neural Network(DNN)

摘要： 传统点击率（CTR）预测模型多在单一特征级上进行特征交互，未能充分利用不同特征级上的有效信息。基于特征增强聚合方法提出一种融合广告CTR预测（APNN）模型。在CTR预测模型的嵌入层中引入一阶信息重要性进行特征增强，通过注意力因子分解机（AFM）模型与基于乘积产生层的神经网络（PNN）模型融合不同特征级交互特征和增强的嵌入特征，并利用多个全连接层从融合特征中获得更多潜在的高阶信息。实验结果表明，相比AFM、PNN、FNN等模型，APNN模型的预测精度较高，其在Criteo数据集上的AUC和LogLoss指标较PNN模型分别提高1.74和1.42个百分点。

关键词: 点击率预测, 一阶信息重要性, 特征增强, 因子分解机, 深度神经网络

CLC Number:

TP391

JIANG Xingyu, HUANG Xianying, CHEN Yujing, XU Fu. Hybrid Advertising Click-through Rate Prediction Model Based on Feature Enhancement Combination[J]. Computer Engineering, 2022, 48(1): 312-320.

蒋兴渝, 黄贤英, 陈雨晶, 徐福. 基于特征增强聚合的融合广告点击率预测模型[J]. 计算机工程, 2022, 48(1): 312-320.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0060051

http://www.ecice06.com/EN/Y2022/V48/I1/312

Figures/Tables 11

References

[1] WANG X.A survey of online advertising click-through rate prediction models[C]//Proceeding of IEEE International Conference on Information Technology, Big Data and Artificial Intelligence.Washington D.C., USA:IEEE Press, 2020:516-521.
[2] JIANG Z, GAO S, DAI W.A CTR prediction approach for text advertising based on the SAE-LR deep neural network[J].Journal of Information Processing Systems, 2017, 13(5):1052-1070.
[3] CHANG Y W, HSIEH C J, CHANG K W, et al.Training and testing low-degree polynomial data mappings via linear SVM[J].Journal of Machine Learning Research, 2010, 11(11):1471-1490.
[4] HOFFMANN F, HOSSEINI B, REN Z, et al.Consistency of semi-supervised learning algorithms on graphs:probit and one-hot methods[J].Journal of Machine Learning Research, 2020, 21(186):1-55.
[5] RENDLE S.Factorization machines with libfm[J].ACM Transactions on Intelligent Systems and Technology, 2012, 3(3):1-22.
[6] JUAN Y, ZHUANG Y, CHIN W S, et al.Field-aware factorization machines for CTR prediction[C]//Proceedings of the 10th ACM Conference on Recommender Systems.New York, USA:ACM Press, 2016:43-50.
[7] XIAO J, YE H, HE X, et al.Attentional factorization machines:learning the weight of feature interactions via attention networks[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.New York, USA:ACM Press, 2017:3119-3125.
[8] HE X, PAN J, JIN O, et al.Practical lessons from predicting clicks on ads at facebook[C]//Proceedings of the 8th International Workshop on Data Mining for Online Advertising.New York, USA:ACM Press, 2014:1-9.
[9] TIAN Y, YANG G, WANG Z, et al.Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J].Computers and Electronics in Agriculture, 2019, 157:417-426.
[10] YANG J, WANG M, ZHOU H, et al.Towards making the most of bert in neural machine translation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2020, 34(5):9378-9385.
[11] SU J, VARGAS D V, SAKURAI K.One pixel attack for fooling deep neural networks[J].IEEE Transactions on Evolutionary Computation, 2019, 23(5):828-841.
[12] ZHANG W, DU T, WANG J.Deep learning over multi-field categorical data[C]//Proceedings of European Conference on Information Retrieval.Berlin, Germany:Springer, 2016:45-57.
[13] QU Y, CAI H, REN K, et al.Product-based neural networks for user response prediction[C]//Proceedigns of the 16th International Conference on Data Mining.Washington D.C., USA:IEEE Press, 2016:1149-1154.
[14] CHENG H T, KOC L, HARMSEN J, et al.Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.New York, USA:ACM Press, 2016:7-10.
[15] GUO H F, TANG R M, YE Y M, et al.DeepFM:a factorization-machine based neural network for CTR prediction[C]//Proceedings of the International Joint Conference on Artificial Intelligence.New York, USA:ACM Press, 2017:1-7.
[16] HUANG T, SHE Q, WANG Z, et al.GateNet:gating-enhanced deep network for click-through rate prediction[EB/OL].[2020-10-15].https://arxiv.org/abs/2007.03519v1.
[17] LI F, WANG B, LU S, et al.Comparison of the sigmoid take-off with other definitions of the rectosigmoid junction:a retrospective comparative cohort analysis[J].International Journal of Surgery, 2020, 80:168-174.
[18] DIAKONIKOLAS I, KANE D M, KONTONIS V, et al.Algorithms and sq lower bounds for pac learning one-hidden-layer relu networks[EB/OL].[2020-10-12].https://arxiv.org/abs/2006.12476v1.
[19] HUANG T, ZHANG Z, ZHANG J.FiBiNET:combining feature importance and bilinear feature interaction for click-through rate prediction[C]//Proceedings of the 13th ACM Conference on Recommender Systems.New York, USA:ACM Press, 2019:169-177.
[20] DENG W, PAN J, ZHOU T, et al.DeepLight:deep lightweight feature interactions for accelerating CTR predictions in ad serving[EB/OL].[2020-10-14].https://arxiv.org/abs/2002.06987.
[21] RADZEVIČIENĖ A, MARQUET P, MASLAUSKIENĖ R, et al.Analyses of AUC (0-12) and C0 compliances within therapeutic ranges in kidney recipients receiving cyclosporine or tacrolimus[J].Journal of Clinical Medicine, 2020, 9(12):3903-3925.
[22] VOVK V.The fundamental nature of the LogLoss function[M].Berlin, Germany:Springer, 2015:307-318.
[23] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2017:5998-6008.
[24] 许王昊, 肖秦琨.基于注意力机制的兴趣网络的点击率预估模型[J].计算机工程, 2021, 47(1):101-108. XU W H, XIAO Q K.A click-through rate prediction model based on attentional interest network[J].Computer Engineering, 2021, 47(1):101-108.(in Chinese)
[25] 邓路佳, 刘平山.基于GMM-FMs的广告点击率预测研究[J].计算机工程, 2019, 45(5):122-126. DENG L J, LIU P S.Research on click-through rate prediction of advertisement based on GMM-FMs[J].Computer Engineering, 2019, 45(5):122-126.(in Chinese)
[26] LI Z, CUI Z, WU S, et al.Fi-GNN:modeling feature interactions via graph neural networks for ctr prediction[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.New York, USA:ACM Press, 2019:539-548.
[27] SONG W, SHI C, XIAO Z, et al.Autoint:automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.New York, USA:ACM Press, 2019:1161-1170.
[28] RAFFEL C, SHAZEER N, ROBERTS A, et al.Exploring the limits of transfer learning with a unified text-to-text transformer[J].Journal of Machine Learning Research, 2020, 21(140):1-67.
[29] LU Y, FANG Y, SHI C.Meta-learning on heterogeneous information networks for cold-start recommendation[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.New York, USA:ACM Press, 2020:1563-1573.
[30] DELDJOO Y, DACREMA M F, CONSTANTIN M G, et al.Movie genome:alleviating new item cold start in movie recommendation[J].User Modeling and User-Adapted Interaction, 2019, 29(2):291-343.

Please choose a citation manager

Content to export