基于混合网络模型的多维时间序列预测

doi:10.19678/j.issn.1000-3428.0063718

摘要/Abstract

摘要： 为捕捉时间序列中潜在的特征依赖关系并实现高维时序数据的快速模糊预测，构建基于时间卷积网络（TCN）与自注意力机制的两种混合网络模型：TSANet和TSANet-MF。TSANet模型通过全局和局部两个并行卷积分量结构提取特征后，利用自注意力机制增强特征点关联程度，并结合并行的TCN增大卷积的感受野范围，最大程度地捕捉多维时序数据的周期性特征。TSANet-MF模型将TSANet作为矩阵分解算法的正则化项，使高维数据转化为具有更多时序特征的低维数据，减少计算复杂度，实现高维数据的快速模糊预测。在4种不同领域的时间序列数据集上的实验结果表明，TSANet模型在3种数据集上的预测性能均优于基准模型，尤其在高维Traffic数据集上相对平方根误差降低了19.52%~56.37%，TSANet-MF模型在Electricity和Traffic高维数据集上的训练时间相比于基准模型明显减少。上述实验结果验证了两种混合网络模型均具有较好的多维时间序列预测性能。

关键词: 多维时间序列, 时间卷积网络, 自注意力机制, 卷积神经网络, 矩阵分解, 正则化

Abstract: To capture the potential feature dependencies in times-series and demonstrate the fast fuzzy prediction of high-dimensional time-series data, this study proposes two hybrid network models, TSANet and TSANet-MF, based on the Temporal Convolutional Network(TCN) and self-attention mechanism.TSANet uses global and local parallel convolution component structures to extract features, and enhance the correlation degree of feature points with self-attention.The parallel TCN is also used to increase the receptive field range of convolution, and effectively capture periodic features of multi-dimensional time-series data.The TSANet-MF model uses TSANet as the regularization term of the Matrix Factorization(MF) algorithm, which converts high-dimensional data into low-dimensional data with more temporal characteristics, reduces computational complexity, and realizes the fast fuzzy prediction of high-dimensional data.Experiments are carried out on real-world multivariate time-series datasets in four different fields.Results show that TSANet outperforms the benchmark models in terms of prediction performance, especially on the high-dimensional Traffic data set, where TSANet decreases Root Relative Squared Error(RRSE) by 19.52%-56.37%.Compared with the benchmark models, TSANet-MF also provides a significant reduction in training time on the Electricity and Traffic high-dimensional data sets.These experimental results help verify that both hybrid network models have good performance in multi-dimensional time-series forecasting.

Key words: multi-dimensional time-series, Temporal Convolutional Network(TCN), self-attention mechanism, Convolutional Neural Network(CNN), Matrix Factorization(MF), regularization

中图分类号:

TP183

刘杭, 殷歆, 陈杰, 罗恒. 基于混合网络模型的多维时间序列预测[J]. 计算机工程, 2023, 49(1): 121-129.

LIU Hang, YIN Xin, CHEN Jie, LUO Heng. Multi-Dimensional Time-Series Prediction Based on Hybrid Network Models[J]. Computer Engineering, 2023, 49(1): 121-129.

http://www.ecice06.com/CN/Y2023/V49/I1/121

图/表 11

20230701175557

20230701175600

20230701175603

20230701175606

20230701175609

20230701175613

20230701175616

20230701175619

20230701175623

20230701175626

20230701175629

参考文献

[1] POTDAR K, KINNERKAR R.A non-linear autoregressive neural network model for forecasting Indian index of industrial production[C]//Proceedings of IEEE Region 10 Symposium.Washington D.C., USA:IEEE Press, 2017:1-5.
[2] HANSSENS D M.Market response, competitive behavior, and time-series analysis[M].[S.l.]:World Scientific, 2018.
[3] BAYRAMOGLU T, ARI Y O.The relationship between tourism and economic growth in Greece economy:a time series analysis[J].Computational Methods in Social Sciences, 2015, 3(1):89-94.
[4] 陈艳, 王子健, 赵泽, 等.传感器网络环境监测时间序列数据的高斯过程建模与多步预测[J].通信学报, 2015, 36(10):252-262. CHEN Y, WANG Z J, ZHAO Z, et al.Gaussian process modeling and multi-step prediction for time series data in wireless sensor network environmental monitoring[J].Journal on Communications, 2015, 36(10):252-262.(in Chinese)
[5] WAN H Y, GUO S N, YIN K, et al.CTS-LSTM:LSTM-based neural networks for correlated time series prediction[J].Knowledge-Based Systems, 2020, 191:105239.
[6] LI P X, TAN Z X, YAN L L, et al.Time series prediction of mining subsidence based on a SVM[J].Mining Science and Technology(China), 2011, 21(4):557-562.
[7] 潘臻.基于隐马尔可夫模型的代码仓库审查时间预测方法[D].南京:南京邮电大学, 2020. PAN Z.Code review time prediction method based on hidden Markov model[D].Nanjing:Nanjing University of Posts and Telecommunications, 2020.(in Chinese)
[8] TOKGÖZ A, ÜNAL G.A RNN based time series approach for forecasting Turkish electricity load[C]//Proceedings of the 26th Signal Processing and Communications Applications Conference(SIU).Washington D.C., USA:IEEE Press, 2018:1-4.
[9] SIAMI-NAMINI S, TAVAKOLI N, SIAMI NAMIN A.A comparison of ARIMA and LSTM in forecasting time series[C]//Proceedings of the 17th IEEE International Conference on Machine Learning and Applications.Washington D.C., USA:IEEE Press, 2018:1394-1401.
[10] XIE H L, ZHANG L, LIM C P.Evolving CNN-LSTM models for time series prediction using enhanced grey wolf optimizer[J].IEEE Access, 2020, 8:161519-161541.
[11] SAJJAD M, KHAN Z A, ULLAH A, et al.A novel CNN-GRU-based hybrid approach for short-term residential load forecasting[J].IEEE Access, 2020, 8:143759-143768.
[12] JIN X B, YU X H, WANG X Y, et al.Prediction for Time Series with CNN and LSTM[C]//Proceedings of the 11th International Conference on Modelling, Identification and Control.Berlin, Germany:Springer, 2020:631-641.
[13] HUANG C J, KUO P H.A deep CNN-LSTM model for particulate matter(PM2.5) forecasting in smart cities[J].Sensors(Basel, Switzerland), 2018, 18(7):2220-2232.
[14] CHEN Y R, WANG Y, DONG Z K, et al.2-D regional short-term wind speed forecast based on CNN-LSTM deep learning model[J].Energy Conversion and Management, 2021, 244:114451.
[15] WU Q Y, GUAN F, LÜ C, et al.Ultra-short-term multi-step wind power forecasting based on CNN-LSTM[J].IET Renewable Power Generation, 2021, 15(5):1019-1029.
[16] LAI G K, CHANG W C, YANG Y M, et al.Modeling long- and short-term temporal patterns with deep neural networks[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.New York, USA:ACM Press, 2018:95-104.
[17] SHIH S Y, SUN F K, LEE H Y.Temporal pattern attention for multivariate time series forecasting[J].Machine Learning, 2019, 108(8/9):1421-1441.
[18] HUANG S T, WANG D L, WU X H, et al.DSANet:dual self-attention network for multivariate time series forecasting[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.New York, USA:ACM Press, 2019:2129-2132.
[19] BAI S J, KOLTER J Z, KOLTUN V.An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL].[2021-12-04].https://arxiv.org/abs/1803.01271.
[20] FAN J, ZHANG K, HUANG Y P, et al.Parallel spatio-temporal attention-based TCN for multivariate time series prediction[J].Neural Computing and Applications, 2022, 34(24):21419-21639.
[21] ZHAO B D, LU H Z, CHEN S F, et al.Convolutional neural networks for time series classification[J].Journal of Systems Engineering and Electronics, 2017, 28(1):162-169.
[22] WANG L X.Fast training algorithms for deep convolutional fuzzy systems with application to stock index prediction[J].IEEE Transactions on Fuzzy Systems, 2020, 28(7):1301-1314.
[23] WILSON K W, RAJ B, SMARAGDIS P.Regularized non-negative matrix factorization with temporal dependencies for speech denoising[C]//Proceedings of InterSpeech 2008.Brisbane, Australia:[s.n.], 2008:411-414.
[24] YU B, YIN H, ZHU Z.Spatio-temporal graph convolutional networks:a deep learning framework for traffic forecasting[EB/OL].[2021-12-04].https://arxiv.org/abs/1709.04875.
[25] SEN R, YU H F, DHILLON I.Think globally, act locally:a deep neural network approach to high-dimensional time series forecasting[EB/OL].[2021-12-04].https://www.cnblogs.com/dulun/p/12271730.html.
[26] BAI Y T, JIN X B, WANG X Y, et al.Compound autoregressive network for prediction of multivariate time series[EB/OL].[2021-12-04].https://doi.org/10.1155/2019/9107167.
[27] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[EB/OL].[2021-12-04].https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.
[28] BACHE K, LICHMAN M.UCI machine learning repository[EB/OL].[2021-12-04].https://archive.ics.uci.edu/ml/index.php.

选择文件类型/文献管理软件名称

选择包含的内容