基于条件生成对抗网络的深度点过程二次预测

doi:10.19678/j.issn.1000-3428.0063648

摘要/Abstract

摘要： 结合深度神经网络和时序点过程的深度点过程模型在进行时间预测时，会因模型本身系统误差和数值计算精度不足而导致预测值序列中存在较大偏差。为提高预测精度并有效避免模型调优同时降低数值误差，建立一种基于条件生成对抗网络（CGAN）的深度点过程二次预测模型，在深度点过程初次预测值序列的基础上进行二次预测。假设初次预测偏差来自时序点过程分布上的差异，利用CGAN对分布的变换能力来修正初次预测值序列分布为原始时序点过程序列分布，从而降低预测值序列误差。在流程上，将初次预测值序列输入生成器生成伪值序列，将伪值序列与对应的真实值序列输入判别器中判别真假，经过对抗训练得到对初次预测值序列具备修正能力的生成器。同时，为增强CGAN对时序点过程数据的匹配度，在其结构上采用CGAN+LSTM的形式，同时改进损失函数为时序点过程Wasserstein距离的对偶形式及其1-Lipschitz约束。实验结果表明，该模型具有较高的时间预测准确度，二次预测值序列的均方误差相较初次预测值序列平均降低77%以上。

关键词: 深度点过程, 二次预测, 条件生成对抗网络, Wasserstein距离, 1-Lipschitz约束

Abstract: The deep point process model that combines a deep neural network and time-series point process is often used for time prediction. However, large deviations in the prediction value series frequently occur because of the systematic error of the model itself and the insufficient accuracy of the numerical calculation. To improve the prediction accuracy, effectively avoid model tuning, and reduce numerical errors, a deep point process secondary prediction model based on a Conditional Generative Adversarial Network(CGAN) is established, and a second prediction is conducted based on the initial prediction value sequence of the deep point process. Based on the assumption that the initial prediction deviation derives from the difference in the process distribution of time-series points, the CGAN, with its ability to transform the distribution, is used to modify the initial prediction value sequence distribution to the original time-series point process sequence distribution, thereby reducing the prediction value sequence error. In this process, the initial predictive value sequence is input into the generator to generate a pseudo-value sequence. The pseudo-value sequence and corresponding real value sequence are then input into the discriminator to determine whether they are true or false. Following confrontation training, a generator that can correct the initial predictive value sequence is obtained. Simultaneously, to enhance the matching degree of the CGAN to the time-series point process data, a CGAN+LSTM structure is adopted, and the loss function is improved such that it becomes a dual form of the Wasserstein distance of the time-series point process and 1-Lipschitz constraint. Experimental results show that the model has a high time-prediction accuracy, and the Mean Square Error(MSE) of the second prediction value series is more than 77% less than that of the first prediction value series.

Key words: deep point process, second prediction, Conditional Generative Adversarial Network(CGAN), Wasserstein distance, 1-Lipschitz constraint

中图分类号:

TP18

卞玮, 李晨龙, 侯红卫. 基于条件生成对抗网络的深度点过程二次预测[J]. 计算机工程, 2022, 48(12): 127-133.

BIAN Wei, LI Chenlong, HOU Hongwei. Second Prediction of the Deep Point Process Based on Conditional Generative Adversarial Network[J]. Computer Engineering, 2022, 48(12): 127-133.

http://www.ecice06.com/CN/Y2022/V48/I12/127

图/表 5

20230112183247

20230112183251

20230112183254

20230112183258

20230112183302

参考文献

[1] BACRY E, MASTROMATTEO I, MUZY J F.Hawkes processes in finance[J].Market Microstructure and Liquidity, 2015, 1(1):1550005.
[2] DALEY D J, VERE-JONES D.An introduction to the theory of point processes[M].2nd ed.Berlin, Germany:Springer, 2003.
[3] DU N, DAI H J, TRIVEDI R, et al.Recurrent marked temporal point processes:embedding event history to vector[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York, USA:ACM Press, 2016:1555-1564.
[4] XIAO S, YAN J C, FARAJTABAR M, et al.Learning time series associated event sequences with recurrent point process networks[J].IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(10):3124-3136.
[5] MEI H, EISNER J.The neural Hawkes process:a neurally self-modulating multivariate point process[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Cambridge, USA:MIT Press, 2017:6754-6764.
[6] LI S, XIAO S, ZHU S, et al.Learning temporal point processes via reinforcement learning[C]//Proceedings of the 32nd Conference on Neural Information Processing Systems.Cambridge, USA:MIT Press, 2018:10781-10791.
[7] UPADHYAY U, DE A, RODRIGUEZ M G.Deep reinforcement learning of marked temporal point processes[C]//Proceedings of the 32nd Conference on Neural Information Processing Systems.Cambridge, USA:MIT Press, 2018:3168-3178.
[8] SHAHAM U, YAMADA Y, NEGAHBAN S.Conditional generative adversarial nets[EB/OL].[2021-11-05].https://arxiv.org/pdf/1411.1784.pdf.
[9] KUPYN O, BUDZAN V, MYKHAILYCH M, et al.DeblurGAN:blind motion deblurring using conditional adversarial networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:8183-8192.
[10] KUPYN O, MARTYNIUK T, WU J R, et al.DeblurGAN-v2:deblurring (orders-of-magnitude) faster and better[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:8877-8886.
[11] AALEN O O, BORGAN O, GJESSING H K.Survival and event history analysis:a process point of view[M].Berlin, Germany:Springer, 2008.
[12] 芦佳明, 李晨龙, 魏毅强.自注意力时序点过程生成模型的Wasserstein学习方法[J].计算机应用研究, 2022, 39(2):456-460. LU J M, LI C L, WEI Y Q.Wasserstein learning method for self-attention temporal point process generation model[J].Application Research of Computers, 2022, 39(2):456-460.(in Chinese)
[13] XIAO S, FARAJTABAR M, YE X J, et al.Wasserstein learning of deep generative point process models[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2017:3250-3259.
[14] OMI T, UEDA N, AIHARA K.Fully neural network based model for general temporal point processes[C]//Proceedings of the 33rd Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2019:15-26.
[15] SHCHUR O, BILOLŠ M, GÜNNEMANN S.Intensity-free learning of temporal point processes[C]//Proceedings of the 8th International Conference on Learning Representations.Washington D.C., USA:IEEE Press, 2020:144-150.
[16] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al.Generative adversarial networks[EB/OL].[2021-11-05].https://arxiv.org/pdf/1406.2661.pdf.
[17] ARJOVSKY M, CHINTALA S, BOTTOU L.Wasserstein GAN[EB/OL].[2021-11-05].https://arxiv.org/pdf/1701.07875v1.pdf.
[18] KOSTELICH E J, SCHREIBER T.Noise reduction in chaotic time-series data:a survey of common methods[J].Physical Review E, 1993, 48(3):1752-1763.
[19] HOCHREITER S, SCHMIDHUBER J.Long short-term memory[J].Neural Computation, 1997, 9(8):1735-1780.
[20] 邱锡鹏.神经网络与深度学习[M].北京:机械工业出版社, 2020. QIU X P.Neural networks and deep learning[M].Beijing:China Machine Press, 2020.(in Chinese)
[21] XU Z, DU J, WANG J J, et al.Satellite image prediction relying on GAN and LSTM neural networks[C]//Proceedings of 2019 IEEE International Conference on Communications.Washington D.C., USA:IEEE Press, 2019:1-6.
[22] TAO X, GAO H Y, SHEN X Y, et al.Scale-recurrent network for deep image deblurring[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:8174-8182.
[23] YOU C, HONG D.Nonlinear blind equalization schemes using complex-valued multilayer feedforward neural networks[J].IEEE Transactions on Neural Networks, 1998, 9(6):1442-1455.
[24] KWOK T Y, YEUNG D Y.Constructive algorithms for structure learning in feedforward neural networks for regression problems[J].IEEE Transactions on Neural Networks, 1997, 8(3):630-645.
[25] IOFFE S, SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning.Washington D.C., USA:IEEE Press, 2015:448-456.
[26] GULRAJANI I, AHMED F, ARJOVSKY M, et al.Improved training of Wasserstein GANs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2017:5769-5779.

选择文件类型/文献管理软件名称

选择包含的内容