基于注意力机制的兴趣网络点击率预估模型

doi:10.19678/j.issn.1000-3428.0056749

摘要/Abstract

摘要： 广告点击率（CTR）是互联网公司进行流量分配的重要依据，针对目前点击率预估精度较低的问题，结合通用的神经网络解决方案，构建一种基于注意力机制的深度兴趣网络（ADIN）模型。设计一个局部激活单元和自适应激活函数，根据用户历史行为和给定广告自适应地学习用户兴趣。引入注意力机制，区分不同特征对预测结果的影响程度，从而增强模型的可解释性。在3个公开数据集上的实验结果表明，相对LR、PNN等CTR预估模型，ADIN模型具有更高的AUC值和更低的LogLoss值，其预测效果更优。

关键词: 点击率预估, 神经网络, 局部激活, 自适应激活函数, 注意力机制

Abstract: Advertising Click-Through Rate(CTR) is an important basis for Internet companies to allocate traffic.To address the inaccurate CTR prediction,this paper proposes an Attention mechanism-based Deep Interest Network(ADIN) model on the basis of the existing neural network solutions.This model has designed a local activation unit and an adaptive activation function to learn user interests adaptively based on the user's historical behavior and the given advertisements.In addition,an attention mechanism is introduced to distinguish the contribution of different features to the prediction results,so that the interpretability of the model is enhanced.Experimental results on three public datasets show that compared with LR,PNN and other CTR estimation models,the proposed ADIN model has better prediction performance with a higher AUC value and a lower LogLoss value.

Key words: Click-Through Rate(CTR) prediction, neural network, local activation, adaptive activation function, attention mechanism

中图分类号:

TP181

许王昊, 肖秦琨. 基于注意力机制的兴趣网络点击率预估模型[J]. 计算机工程, 2021, 47(1): 101-108.

XU Wanghao, XIAO Qinkun. Click-Through Rate Prediction Model of Interest Network Based on Attention Mechanism[J]. Computer Engineering, 2021, 47(1): 101-108.

https://www.ecice06.com/CN/Y2021/V47/I1/101

图/表 8

20210125163514

20210125163517

20210125163520

20210125163524

20210125163527

20210125163530

20210125163532

20210125163535

参考文献

[1] PAN J W,XU J,RUIZ A L,et al.Field-weighted factorization machines for click-through rate prediction in display advertising[C]//Proceedings of 2018 World Wide Web Conference.New York,USA:ACM Press,2018:16-28.
[2] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[EB/OL].[2019-10-20].https://arxiv.org/abs/1512.03385.
[3] DENG Lujia,LIU Pingshan.Research on click-through rate prediction of advertisement based on GMM-FMs[J]. Computer Engineering,2019,45(5):122-126.(in Chinese)邓路佳,刘平山.基于GMM-FMs的广告点击率预测研究[J].计算机工程,2019,45(5):122-126.
[4] CHEN Junxuan,SUN Baigui,LI Hao,et al.Deep CTR prediction in display advertising[C]//Proceedings of 2016 ACM Multimedia Conference.New York,USA:ACM Press,2016:158-169.
[5] CHEN Jiehao,LI Xueyi,ZHAO Ziqian,et al.A CTR prediction method based on feature engineering and online learning[C]//Proceedings of 2017 International Symposium on Communica-tions and Information Technologies.Washington D.C.,USA:IEEE Press,2017:190-203.
[6] JIN Ziyan,ZHANG Juan,LI Xiangjun,et al.A collaborative filtering advertising recommendation algorithm with tag[J].Computer Engineering,2018,44(4):236-242,247.(in Chinese)金紫嫣,张娟,李向军,等.一种带标签的协同过滤广告推荐算法[J].计算机工程,2018,44(4):236-242,247.
[7] CHENG H T,KOC L,HARMSEN J,et al.Wide & Deep learning for recommender systems[EB/OL].[2019-10-20].https://arxiv.org/abs/1606.07792.
[8] GUO Huifeng,TANG Ruiming,YE Yunming,et al.DeepFM:a factorization-machine based neural network for CTR prediction[EB/OL].[2019-10-20].https://arxiv.org/abs/1703.04247.
[9] LIU Enbo.Boosting-based method for advertising conversion rate prediction[D].Harbin:Harbin Institute of Technology,2018.(in Chinese)刘恩伯.基于提升模型的广告转化率预估[D].哈尔滨:哈尔滨工业大学,2018.
[10] ZHENG L,NOROOZI V,YU P.Joint deep modeling of users and items using reviews for recommendation[EB/OL].[2019-10-20].https://arxiv.org/abs/1701.04783.
[11] ZHENG Y,ZHANG Y J,LAROCHELLE H.A deep and autoregressive approach for topic modeling of multimodal data[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(6):1056-1069.
[12] WU C Y,AHMED A,BEUTEL A,et al.Recurrent recommender networks[C]//Proceedings of the 10th ACM International Conference on Web Search and Data Mining.New York,USA:ACM Press,2017:15-26.
[13] WU Y,DUBOIS C,ZHENG A X,et al.Collaborative denoising auto-encoders for top-N recommender systems[C]//Proceedings of the 9th ACM International Conference on Web Search and Data Mining.New York,USA:ACM Press,2016:125-136.
[14] ZHOU Hao,HUANG Minlie,MAO Yishun,et al.Domain-constrained advertising keyword generation[EB/OL].[2019-10-20].https://arxiv.org/abs/1902.10374.
[15] KAN S C,CEN Y G,HE Z H,et al.Supervised deep feature embedding with handcrafted feature[J].IEEE Transactions on Image Processing,2019,28(12):5809-5823.
[16] CHOROWSKI J,BAHDANAU D,SERDYUK D,et al.Attention-based models for speech recognition[J].Computer Science,2015,10(4):429-439.
[17] SPECHT D F.Probabilistic neural networks[J].Neural Networks,1990,3(1):109-118.
[18] LIU Tong.Research on software defect prediction based on machine learning algorithm[D].Wuhan:Central China Normal University,2018.(in Chinese)刘童.基于机器学习算法的软件缺陷预测技术研究[D].武汉:华中师范大学,2018.
[19] SUN Yi,LIU Di,LI Bin,et al.Application of deep reinforcement learning in demand response[J].Automation of Electric Power Systems,2019,43(5):183-191.(in Chinese)孙毅,刘迪,李彬,等.深度强化学习在需求响应中的应用[J].电力系统自动化,2019,43(5):183-191.
[20] KADAM V J,JADHAV S M,VIJAYAKUMAR K.Breast cancer diagnosis using feature ensemble learning based on stacked sparse autoencoders and softmax regression[J].Journal of Medical Systems,2019,43(8):263-265.
[21] RIAHI-MADVAR H,DEHGHANI M,SEIFI A,et al.Comparative analysis of soft computing techniques RBF,MLP,and ANFIS with MLR and MNLR for predicting grade-control scour hole geometry[J].Engineering Applications of Computational Fluid Mechanics,2019,13(1):529-550.
[22] SHAN Y,HOENS T R,JIAO J,et al.Deep crossing:Web-scale modeling without manually crafted combinatorial features[C]//Proceedings of the 22nd ACM SIGKDD International Conference.New York,USA:ACM Press,2016:12-25.
[23] MO S X,ZHU Y H,ZABARAS N,et al.Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media[J].Water Resources Research,2019,55(1):703-728.
[24] ACHKAR R,ELIAS-SLEIMAN F,EZZIDINE H,et al.Comparison of BPA-MLP and LSTM-RNN for stocks prediction[C]//Proceedings of 2018 International Symposium on Computational and Business Intelligence.Washington D.C.,USA:IEEE Press,2018:125-136.
[25] ZHANG Ce,PAN Xin,LI Huapeng,et al.A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,140:133-144.
[26] WANG Yi,GAN Dahua,ZHANG Ning,et al.Feature selection for probabilistic load forecasting via sparse penalized quantile regression[J].Journal of Modern Power Systems and Clean Energy,2019,7(5):1200-1209.
[27] HAN B,TSANG I W,CHEN L,et al.Progressive stochastic learning for noisy labels[J].IEEE Transactions on Neural Networks and Learning Systems,2018,29(10):5136-5148.
[28] LO H Z,COHEN J P,DING W.Prediction gradients for feature extraction and analysis from convolutional neural networks[C]//Proceedings of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.Washington D.C.,USA:IEEE Press,2015:20-26.
[29] HASSNA G,LOWRY P B.Big data capability,customer agility,and organization performance:a dynamic capability perspective[EB/OL].[2019-10-20].https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2861423.
[30] LU Haofang,ZHOU Ying,ZHANG Zike.Calculate deep convolution neural network on cell unit[M].Berlin,Germany:Springer,2017.
[31] JACKSON F,AMIN R,FU Y H,et al.A user study of Netflix streaming[M].Berlin,Germany:Springer,2015.
[32] LAZLI L,BOUKADOUM M,MOHAMED O A.HMM/MLP speech recognition system using a novel data clustering approach[C]//Proceedings of 2017 IEEE Canadian Conference on Electrical and Computer Engineering.Washington D.C.,USA:IEEE Press,2017:145-163.
[33] SALLAB A,ABDOU M,PEROT E,et al.Deep reinforcement learning framework for autonomous driving[J].Electronic Imaging,2017(19):70-76.
[34] QURESHI A H,NAKAMURA Y,YOSHIKAWA Y,et al.Show,attend and interact:perceivable human-robot social interaction through neural attention Q-network[C]//Proceedings of 2017 IEEE International Conference on Robotics and Automation.Washington D.C.,USA:IEEE Press,2017:456-485.

选择文件类型/文献管理软件名称

选择包含的内容