基于翻转网络的低相关性序列数据预测研究

doi:10.19678/j.issn.1000-3428.0067027

摘要/Abstract

摘要：

在某些实际应用中，通常不存在与被预测时间变量具有高相关性的其他维度变量，或者这些维度变量难以采集。而具有较低相关性的时间序列数据普遍存在，其对于数据预测具有更重要的意义。提出一种基于注意力翻转网络的低相关性多维时间序列数据预测模型。针对低相关性时序数据具有相关性随时间而变化的特点，引入批处理滑动窗口以摆脱时间变化带来的干扰，更好地捕获维度相关性。针对传统门控循环单元(GRU)网络大量丢弃低相关性样本的问题，建立翻转GRU网络对低相关性多维数据进行初次过滤，控制多维数据在网络中的传递数量，避免维度变量因相关性较低而被丢弃，提升相关性较低的多维数据在模型中的存活时间。同时，利用基于维度的注意力机制自适应调整不同维度序列在相关性提取过程中的重要性。建立平方长短期记忆（LSTM）网络对分配权重后的数据进行拟合，更细致地确定相关性对被预测参数的影响。实验结果表明，该模型的决定系数可达0.95，预测性能优于GRU、LSTM等传统神经网络模型。

关键词: 时间序列数据, 深度学习, 相关性, 注意力机制, 长短期记忆网络, 门控循环单元

Abstract:

In some practical applications, there are often no other dimensional variables that highly correlate with the predicted time variable, or these dimensional variables are difficult to collect. At the same time, time series data with lower correlations are prevalent, which have more important significance for the improvement of forecasting results. Therefore, this study proposes a low-correlation multi-dimensional time series data prediction model based on attention flipping network. First, for low-correlation time series data, the correlation changes with time and a batch sliding window is introduced to remove the interference caused by time changes to better capture the dimensional correlation. Second, in view of the problem that the traditional Gated Recurrent Unit (GRU) network discards a large number of low-correlation samples, a flipped GRU network to screen low-correlation multi-dimensional data for the first time is established, the number of multi-dimensional data transmitted in the network is controlled, dimensional variables being discarded due to low-correlation is avoided, and survival time of the multi-dimensional data with low-correlation in the model is improved. Simultaneously, a dimension-based attention mechanism is used to adaptively adjust the importance of different dimension sequences in the correlation extraction process. Finally, a square Long Short-Term Memory (LSTM) network is established to fit the weighted data and determine the influence of the correlation on the predicted parameters in more detail. The experimental results show that the determination coefficient of the proposed model can reach 0.95 and its predictive performance is superior to traditional neural network models such as GRU and LSTM.

Key words: time series data, deep learning, correlation, attention mechanism, Long Short-Term Memory(LSTM) network, Gated Recurrent Unit(GRU)

丁国辉, 刘宇琪, 王言开, 耿施展, 姜天昊. 基于翻转网络的低相关性序列数据预测研究[J]. 计算机工程, 2024, 50(2): 78-90.

Guohui DING, Yuqi LIU, Yankai WANG, Shizhan GENG, Tianhao JIANG. Research on Low-Correlation Sequence Data Prediction Based on Flip Network[J]. Computer Engineering, 2024, 50(2): 78-90.

https://www.ecice06.com/CN/Y2024/V50/I2/78

图/表 20

图1 共享单车参数之间的相关性

Fig.1 Correlation between shared bicycle parameters

图2 气温与共享单车使用量的关系

Fig.2 Relationship between temperature and shared bicycle usage

图3 体感温度与共享单车使用量的关系

Fig.3 Relationship between body temperature and shared bicycle usage

图4 气温与体感温度最高的10个点与共享单车使用量的相关性

Fig.4 Correlation between the ten points with the highest temperature and body temperature and shared bicycle usage

图5 基于滑动窗口的数据集划分机制

Fig.5 Dataset partitioning mechanism based on sliding window

图6 批处理滑动窗口

Fig.6 Batch processing sliding window

图7 基于维度的注意力机制模型

Fig.7 Dimension based attention mechanism model

图8 权重共享的注意力机制流程

Fig.8 The procedure of attention mechanism with weight sharing

图9 权重非共享的注意力机制流程

Fig.9 The procedure of attention mechanism without weight sharing

图10 TURN-GRU的神经元结构

Fig.10 The neuronal structure of TURN-GRU

图11 SQ-LSTM的神经元结构

Fig.11 The neuronal structure of SQ-LSTM

图12 本文网络基本流程

Fig.12 Basic procedure of the network in this paper

图13 隐含层ATT-TURN-NN模型结构

Fig.13 Hidden layer ATT-TURN-NN model structure

图14 不使用滑动窗口时模型预测值与真实值的对比结果

Fig.14 Comparison results between model predicted values and true values without using sliding windows

图15 滑动窗口划分后模型预测值与真实值的对比结果

Fig.15 Comparison results between model predicted values and true values after sliding window partitioning

参考文献 27

1	CODY R A, NARASIMHAN S. A field implementation of linear prediction for leak-monitoring in water distribution networks. Advanced Engineering Informatics, 2020, 45, 101103. doi: 10.1016/j.aei.2020.101103
2	CHEN C, LIU H. Dynamic ensemble wind speed prediction model based on hybrid deep reinforcement learning. Advanced Engineering Informatics, 2021, 48, 101290. doi: 10.1016/j.aei.2021.101290
3	OLOYEDE A, OZUOMBA S, ASUQUO P, et al. Data-driven techniques for temperature data prediction: big data analytics approach. Environmental Monitoring and Assessment, 2023, 195 (2): 1- 21. doi: 10.1007/s10661-023-10961-z
4	TIAN Y, ZHOU K, PELLEG D. What and how long: prediction of mobile App engagement. ACM Transactions on Information Systems, 2021, 40 (1): 1- 38. doi: 10.1145/3464301
5	ALAHI A, GOEL K, RAMANATHAN V, et al. Social LSTM: human trajectory prediction in crowded spaces[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 11-23.
6	XIE Y F, ZHUANG X B, XI Z P, et al. Dual-channel and bidirectional neural network for hypersonic glide vehicle trajectory prediction. IEEE Access, 2021, 9, 92913- 92924. doi: 10.1109/ACCESS.2021.3092515
7	WANG J Y, LIN X, ZUO Y, et al. DGeye: probabilistic risk perception and prediction for urban dangerous goods management. ACM Transactions on Information Systems, 2021, 39 (3): 1- 30. URL
8	高世乐, 王滢, 李海林, 等. 基于矩阵画像的金融时序数据预测方法. 计算机应用, 2021, 41 (1): 199- 207. doi: 10.11772/j.issn.1001-9081.2020060877
	GAO S L, WANG Y, LI H L, et al. Prediction method on financial time series data based on matrix profile. Journal of Computer Applications, 2021, 41 (1): 199- 207. doi: 10.11772/j.issn.1001-9081.2020060877
9	刘宇. 基于海洋时序数据的温度预测与补全方法研究[D]. 长春: 吉林大学, 2020.
	LIU Y. Research on temperature prediction and completion method based on marine time series data[D]. Changchun: Jilin University, 2020. (in Chinese)
10	YANG L X, GAO T, LU Y B, et al. Neural network stochastic differential equation models with applications to financial data forecasting. Applied Mathematical Modelling, 2023, 115, 279- 299. doi: 10.1016/j.apm.2022.11.001
11	ENGLE R F. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica, 1982, 50 (4): 987. doi: 10.2307/1912773
12	BOLLERSLEV T. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 1986, 31 (3): 307- 327. doi: 10.1016/0304-4076(86)90063-1
13	BARTHOLOMEW D J. Time series analysis forecasting and control. Journal of the Operational Research Society, 1971, 22 (2): 199- 201. doi: 10.1057/jors.1971.52
14	PETER Ď, SILVIA P. ARIMA vs. ARIMAX—which approach is better to analyze and forecast macroeconomic time series[EB/OL]. [2023-01-05]. http://mme2012.opf.slu.cz/proceedings/pdf/024_Durka.pdf.
15	ZHANG P, LI H, HA Q P, et al. Reinforcement learning based optimizer for improvement of predicting tunneling-induced ground responses. Advanced Engineering Informatics, 2020, 45, 101097. doi: 10.1016/j.aei.2020.101097
16	WANG R. Prediction research and application of financial time series based on big data. Journal of Physics: Conference Series, 2021, 1881 (2): 022093. doi: 10.1088/1742-6596/1881/2/022093
17	SAFARI A, DAVALLOU M. Oil price forecasting using a hybrid model. Energy, 2018, 148, 49- 58. doi: 10.1016/j.energy.2018.01.007
18	ZHENG D X, YUAN Y. Time series data prediction and feature analysis of sports dance movements based on machine learning[EB/OL]. [2023-01-05]. https://www.hindawi.com/journals/cin/2022/5611829/.
19	CHENG Y, HU K, WU J, et al. A convolutional neural network based degradation indicator construction and health prognosis using bidirectional long short-term memory network for rolling bearings. Advanced Engineering Informatics, 2021, 48, 101247. doi: 10.1016/j.aei.2021.101247
20	PAN Z Y, LIANG Y X, WANG W F, et al. Urban traffic prediction from spatio-temporal data using deep meta learning[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2019: 1720-1730.
21	MA X L, ZHANG J Y, DU B W, et al. Parallel architecture of convolutional bi-directional LSTM neural networks for network-wide metro ridership prediction. IEEE Transactions on Intelligent Transportation Systems, 2019, 20 (6): 2278- 2288. doi: 10.1109/TITS.2018.2867042
22	WAN H, GUO S, YIN K, et al. CTS-LSTM: LSTM-based neural networks for correlatedtime series prediction. Knowledge-Based Systems, 2020, 191, 105239. doi: 10.1016/j.knosys.2019.105239
23	XU J J, ZHAO J, ZHOU R, et al. Predicting destinations by a deep learning based approach. IEEE Transactions on Knowledge and Data Engineering, 2021, 33 (2): 651- 666. doi: 10.1109/TKDE.2019.2932984
24	ZHANG X, YAN M, XIE B, et al. An automatic real-time bus schedule redesign method based on bus arrival time prediction. Advanced Engineering Informatics, 2021, 48, 101295. doi: 10.1016/j.aei.2021.101295
25	林靖皓. 基于Self-attention C-BiGRU芒果产量预测研究[D]. 南宁: 广西大学, 2019.
	LIN J H. Study on mango yield prediction based on Self-attention C-BiGRU[D]. Nanning: Guangxi University, 2019. (in Chinese)
26	DUAN Q, WEI X, GAO Y, et al. Base station traffic prediction based on STL-LSTM networks[EB/OL]. [2023-01-05]. https://ieeexplore.ieee.org/document/8633565.
27	MEJIA J, SOSA L A, MEDEROS B, et al. Prediction of time series using an analysis filter bank of LSTM units. Computers & Industrial Engineering, 2021, 157, 107371. URL

[1]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[2]	屠乃威, 焦猛, 阎馨. 复杂环境下输电线路鸟巢目标图像检测模型[J]. 计算机工程, 2024, 50(7): 216-226.
[3]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[4]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.
[5]	张锡英, 孙守东, 于海浩, 边继龙. 基于空间传播的多视图三维重建[J]. 计算机工程, 2024, 50(7): 293-302.
[6]	贵向泉, 刘世清, 李立, 秦庆松, 李唐艳. 基于改进YOLOv8的景区行人检测算法[J]. 计算机工程, 2024, 50(7): 342-351.
[7]	王炼红, 林飞鹏, 李潇瑶, 谌桂枝, 周莉. 融入课程知识图谱的KMAKT预测[J]. 计算机工程, 2024, 50(7): 23-31.
[8]	钱来, 赵卫伟. 基于对比学习和注意力机制的文本分类方法[J]. 计算机工程, 2024, 50(7): 104-111.
[9]	刘建敏, 林晖, 汪晓丁. 基于图注意力机制的无地图场景轨迹预测方法[J]. 计算机工程, 2024, 50(7): 144-153.
[10]	徐明亮, 李芳媛, 马浩然, 何飞. 大规模神经记录的峰电位聚类算法(特邀)[J]. 计算机工程, 2024, 50(6): 1-34.
[11]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[12]	李子杰, 周菊香, 韩晓瑜, 甘健侯, 鹿泽光, 王俊. 序列特征与学习过程融合的知识追踪模型[J]. 计算机工程, 2024, 50(6): 77-85.
[13]	程腾腾, 姚春龙, 于晓强, 李旭, 王庆丰. 基于多头注意力机制融合常识知识的共情对话生成[J]. 计算机工程, 2024, 50(6): 94-101.
[14]	更藏措毛, 黄鹤鸣, 杨毅杰. 融合多尺度特征与上下文信息的语音增强方法[J]. 计算机工程, 2024, 50(6): 138-147.
[15]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.

选择文件类型/文献管理软件名称

选择包含的内容