基于翻转网络的低相关性序列数据预测研究

doi:10.19678/j.issn.1000-3428.0067027

摘要/Abstract

摘要：

在某些实际应用中，通常不存在与被预测时间变量具有高相关性的其他维度变量，或者这些维度变量难以采集。而具有较低相关性的时间序列数据普遍存在，其对于数据预测具有更重要的意义。提出一种基于注意力翻转网络的低相关性多维时间序列数据预测模型。针对低相关性时序数据具有相关性随时间而变化的特点，引入批处理滑动窗口以摆脱时间变化带来的干扰，更好地捕获维度相关性。针对传统门控循环单元(GRU)网络大量丢弃低相关性样本的问题，建立翻转GRU网络对低相关性多维数据进行初次过滤，控制多维数据在网络中的传递数量，避免维度变量因相关性较低而被丢弃，提升相关性较低的多维数据在模型中的存活时间。同时，利用基于维度的注意力机制自适应调整不同维度序列在相关性提取过程中的重要性。建立平方长短期记忆（LSTM）网络对分配权重后的数据进行拟合，更细致地确定相关性对被预测参数的影响。实验结果表明，该模型的决定系数可达0.95，预测性能优于GRU、LSTM等传统神经网络模型。

关键词: 时间序列数据, 深度学习, 相关性, 注意力机制, 长短期记忆网络, 门控循环单元

Abstract:

In some practical applications, there are often no other dimensional variables that highly correlate with the predicted time variable, or these dimensional variables are difficult to collect. At the same time, time series data with lower correlations are prevalent, which have more important significance for the improvement of forecasting results. Therefore, this study proposes a low-correlation multi-dimensional time series data prediction model based on attention flipping network. First, for low-correlation time series data, the correlation changes with time and a batch sliding window is introduced to remove the interference caused by time changes to better capture the dimensional correlation. Second, in view of the problem that the traditional Gated Recurrent Unit (GRU) network discards a large number of low-correlation samples, a flipped GRU network to screen low-correlation multi-dimensional data for the first time is established, the number of multi-dimensional data transmitted in the network is controlled, dimensional variables being discarded due to low-correlation is avoided, and survival time of the multi-dimensional data with low-correlation in the model is improved. Simultaneously, a dimension-based attention mechanism is used to adaptively adjust the importance of different dimension sequences in the correlation extraction process. Finally, a square Long Short-Term Memory (LSTM) network is established to fit the weighted data and determine the influence of the correlation on the predicted parameters in more detail. The experimental results show that the determination coefficient of the proposed model can reach 0.95 and its predictive performance is superior to traditional neural network models such as GRU and LSTM.

Key words: time series data, deep learning, correlation, attention mechanism, Long Short-Term Memory(LSTM) network, Gated Recurrent Unit(GRU)

丁国辉, 刘宇琪, 王言开, 耿施展, 姜天昊. 基于翻转网络的低相关性序列数据预测研究[J]. 计算机工程, 2024, 50(2): 78-90.

Guohui DING, Yuqi LIU, Yankai WANG, Shizhan GENG, Tianhao JIANG. Research on Low-Correlation Sequence Data Prediction Based on Flip Network[J]. Computer Engineering, 2024, 50(2): 78-90.

http://www.ecice06.com/CN/Y2024/V50/I2/78

图/表 20

图1 共享单车参数之间的相关性

Fig.1 Correlation between shared bicycle parameters

图2 气温与共享单车使用量的关系

Fig.2 Relationship between temperature and shared bicycle usage

图3 体感温度与共享单车使用量的关系

Fig.3 Relationship between body temperature and shared bicycle usage

图4 气温与体感温度最高的10个点与共享单车使用量的相关性

Fig.4 Correlation between the ten points with the highest temperature and body temperature and shared bicycle usage

图5 基于滑动窗口的数据集划分机制

Fig.5 Dataset partitioning mechanism based on sliding window

图6 批处理滑动窗口

Fig.6 Batch processing sliding window

图7 基于维度的注意力机制模型

Fig.7 Dimension based attention mechanism model

图8 权重共享的注意力机制流程

Fig.8 The procedure of attention mechanism with weight sharing

图9 权重非共享的注意力机制流程

Fig.9 The procedure of attention mechanism without weight sharing

图10 TURN-GRU的神经元结构

Fig.10 The neuronal structure of TURN-GRU

图11 SQ-LSTM的神经元结构

Fig.11 The neuronal structure of SQ-LSTM

图12 本文网络基本流程

Fig.12 Basic procedure of the network in this paper

图13 隐含层ATT-TURN-NN模型结构

Fig.13 Hidden layer ATT-TURN-NN model structure

图14 不使用滑动窗口时模型预测值与真实值的对比结果

Fig.14 Comparison results between model predicted values and true values without using sliding windows

图15 滑动窗口划分后模型预测值与真实值的对比结果

Fig.15 Comparison results between model predicted values and true values after sliding window partitioning

参考文献 27

1	CODY R A, NARASIMHAN S. A field implementation of linear prediction for leak-monitoring in water distribution networks. Advanced Engineering Informatics, 2020, 45, 101103. doi: 10.1016/j.aei.2020.101103
2	CHEN C, LIU H. Dynamic ensemble wind speed prediction model based on hybrid deep reinforcement learning. Advanced Engineering Informatics, 2021, 48, 101290. doi: 10.1016/j.aei.2021.101290
3	OLOYEDE A, OZUOMBA S, ASUQUO P, et al. Data-driven techniques for temperature data prediction: big data analytics approach. Environmental Monitoring and Assessment, 2023, 195 (2): 1- 21. doi: 10.1007/s10661-023-10961-z
4	TIAN Y, ZHOU K, PELLEG D. What and how long: prediction of mobile App engagement. ACM Transactions on Information Systems, 2021, 40 (1): 1- 38. doi: 10.1145/3464301
5	ALAHI A, GOEL K, RAMANATHAN V, et al. Social LSTM: human trajectory prediction in crowded spaces[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 11-23.
6	XIE Y F, ZHUANG X B, XI Z P, et al. Dual-channel and bidirectional neural network for hypersonic glide vehicle trajectory prediction. IEEE Access, 2021, 9, 92913- 92924. doi: 10.1109/ACCESS.2021.3092515
7	WANG J Y, LIN X, ZUO Y, et al. DGeye: probabilistic risk perception and prediction for urban dangerous goods management. ACM Transactions on Information Systems, 2021, 39 (3): 1- 30. URL
8	高世乐, 王滢, 李海林, 等. 基于矩阵画像的金融时序数据预测方法. 计算机应用, 2021, 41 (1): 199- 207. doi: 10.11772/j.issn.1001-9081.2020060877
	GAO S L, WANG Y, LI H L, et al. Prediction method on financial time series data based on matrix profile. Journal of Computer Applications, 2021, 41 (1): 199- 207. doi: 10.11772/j.issn.1001-9081.2020060877
9	刘宇. 基于海洋时序数据的温度预测与补全方法研究[D]. 长春: 吉林大学, 2020.
	LIU Y. Research on temperature prediction and completion method based on marine time series data[D]. Changchun: Jilin University, 2020. (in Chinese)
10	YANG L X, GAO T, LU Y B, et al. Neural network stochastic differential equation models with applications to financial data forecasting. Applied Mathematical Modelling, 2023, 115, 279- 299. doi: 10.1016/j.apm.2022.11.001
11	ENGLE R F. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica, 1982, 50 (4): 987. doi: 10.2307/1912773
12	BOLLERSLEV T. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 1986, 31 (3): 307- 327. doi: 10.1016/0304-4076(86)90063-1
13	BARTHOLOMEW D J. Time series analysis forecasting and control. Journal of the Operational Research Society, 1971, 22 (2): 199- 201. doi: 10.1057/jors.1971.52
14	PETER Ď, SILVIA P. ARIMA vs. ARIMAX—which approach is better to analyze and forecast macroeconomic time series[EB/OL]. [2023-01-05]. http://mme2012.opf.slu.cz/proceedings/pdf/024_Durka.pdf.
15	ZHANG P, LI H, HA Q P, et al. Reinforcement learning based optimizer for improvement of predicting tunneling-induced ground responses. Advanced Engineering Informatics, 2020, 45, 101097. doi: 10.1016/j.aei.2020.101097
16	WANG R. Prediction research and application of financial time series based on big data. Journal of Physics: Conference Series, 2021, 1881 (2): 022093. doi: 10.1088/1742-6596/1881/2/022093
17	SAFARI A, DAVALLOU M. Oil price forecasting using a hybrid model. Energy, 2018, 148, 49- 58. doi: 10.1016/j.energy.2018.01.007
18	ZHENG D X, YUAN Y. Time series data prediction and feature analysis of sports dance movements based on machine learning[EB/OL]. [2023-01-05]. https://www.hindawi.com/journals/cin/2022/5611829/.
19	CHENG Y, HU K, WU J, et al. A convolutional neural network based degradation indicator construction and health prognosis using bidirectional long short-term memory network for rolling bearings. Advanced Engineering Informatics, 2021, 48, 101247. doi: 10.1016/j.aei.2021.101247
20	PAN Z Y, LIANG Y X, WANG W F, et al. Urban traffic prediction from spatio-temporal data using deep meta learning[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2019: 1720-1730.
21	MA X L, ZHANG J Y, DU B W, et al. Parallel architecture of convolutional bi-directional LSTM neural networks for network-wide metro ridership prediction. IEEE Transactions on Intelligent Transportation Systems, 2019, 20 (6): 2278- 2288. doi: 10.1109/TITS.2018.2867042
22	WAN H, GUO S, YIN K, et al. CTS-LSTM: LSTM-based neural networks for correlatedtime series prediction. Knowledge-Based Systems, 2020, 191, 105239. doi: 10.1016/j.knosys.2019.105239
23	XU J J, ZHAO J, ZHOU R, et al. Predicting destinations by a deep learning based approach. IEEE Transactions on Knowledge and Data Engineering, 2021, 33 (2): 651- 666. doi: 10.1109/TKDE.2019.2932984
24	ZHANG X, YAN M, XIE B, et al. An automatic real-time bus schedule redesign method based on bus arrival time prediction. Advanced Engineering Informatics, 2021, 48, 101295. doi: 10.1016/j.aei.2021.101295
25	林靖皓. 基于Self-attention C-BiGRU芒果产量预测研究[D]. 南宁: 广西大学, 2019.
	LIN J H. Study on mango yield prediction based on Self-attention C-BiGRU[D]. Nanning: Guangxi University, 2019. (in Chinese)
26	DUAN Q, WEI X, GAO Y, et al. Base station traffic prediction based on STL-LSTM networks[EB/OL]. [2023-01-05]. https://ieeexplore.ieee.org/document/8633565.
27	MEJIA J, SOSA L A, MEDEROS B, et al. Prediction of time series using an analysis filter bank of LSTM units. Computers & Industrial Engineering, 2021, 157, 107371. URL

[1]	郭祥振, 李思潼, 卢锐, 郭森, 崔学荣, 杨钢. 基于多任务联合注意力的结肠息肉分割网络[J]. 计算机工程, 2024, 50(2): 327-336.
[2]	郑晨俊, 曾艳, 袁俊峰, 张纪林, 王鑫, 韩猛. 基于联邦学习的船舶AIS轨迹预测算法[J]. 计算机工程, 2024, 50(2): 298-307.
[3]	王正家, 胡飞飞, 张成娟, 雷卓, 何涛. 引入轻量级Transformer的自适应窗口立体匹配算法[J]. 计算机工程, 2024, 50(2): 256-265.
[4]	陈虹, 王瀚文, 金海波. 融合改进自编码器和残差网络的入侵检测模型[J]. 计算机工程, 2024, 50(2): 188-195.
[5]	徐浩宸, 刘满华. 基于多层次自注意力网络的人脸特征点检测[J]. 计算机工程, 2024, 50(2): 239-246.
[6]	刘帅威, 李智, 王国美, 张丽. 基于Transformer和GAN的对抗样本生成算法[J]. 计算机工程, 2024, 50(2): 180-187.
[7]	曾嘉忻, 张卫明, 张荣. 基于后门的鲁棒后向模型水印方法[J]. 计算机工程, 2024, 50(2): 132-139.
[8]	安峰民, 张冰冰, 董微, 张建新. 面向视频行为识别深度模型的数据预处理方法[J]. 计算机工程, 2024, 50(2): 281-287.
[9]	吴志强, 解庆, 李琳, 刘永坚. 基于多模态融合的图神经网络推荐算法[J]. 计算机工程, 2024, 50(1): 91-100.
[10]	申秀雨, 姬伟峰, 李映岐, 吴玄. 面向边缘计算的TCA1C DDoS检测模型[J]. 计算机工程, 2024, 50(1): 198-205.
[11]	圣文顺, 余熊峰, 林佳燕, 陈欣. 融合注意力与特征金字塔的小尺度目标检测算法[J]. 计算机工程, 2024, 50(1): 242-250.
[12]	祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测[J]. 计算机工程, 2024, 50(1): 216-223.
[13]	蒋心璐, 陈天恩, 王聪, 赵春江. 大田环境下的农业害虫图像小目标检测算法[J]. 计算机工程, 2024, 50(1): 232-241.
[14]	白尚旺, 王梦瑶, 胡静, 陈志泊. 多区域注意力的细粒度图像分类网络[J]. 计算机工程, 2024, 50(1): 271-278.
[15]	杨瑞君, 秦晋京, 程燕. 基于生成对抗网络的自然场景低照度增强模型[J]. 计算机工程, 2024, 50(1): 279-288.

选择文件类型/文献管理软件名称

选择包含的内容