Short-term Power Load Forecasting Based on Dynamic Multi-Scale and Dual Attention Mechanisms

doi:10.19678/j.issn.1000-3428.0070067

Abstract

Abstract:

Short-term power load forecasting plays a crucial role in the optimal scheduling and safe operation of power systems. Power load data exhibit multiperiod characteristics, showing different patterns and trends at various time scales. Accurately extracting the scale size helps identify and separate these features. Current methods use a fixed patch length or a set of fixed patch lengths as steps and encode time series into segments called patches. However, these methods cannot adapt to the complex dynamic changes in real-world load series data. Therefore, this paper proposes a prediction model based on a dynamic Multi-scale and Dual Attention Transformer (MDAT). First, Successive Variational Mode Decomposition (SVMD) is used to separate different time patterns in the load series, and Fast Fourier Transform (FFT) is performed to extract the significant period of each pattern. Subsequently, based on the detected significant periods, the load series is divided into different time resolutions using patches of varying sizes, and multiple branches of a transformer are used to simultaneously model the dependencies of the sequences segmented at different scales. Next, dual attention is applied to these patches to capture the global correlations and local details. Finally, nonlinear feature fusion is performed on the outputs of each branch, and the final load prediction results are obtained by stacking multiple transformer modules. Experimental results on two public datasets demonstrate that the proposed model performs well in terms of prediction accuracy. Compared to the latest models based on Transformer and Multilayer Perceptron (MLP), the Mean Absolute Error (MAE) on the Australia and Morocco datasets is reduced by 10.26%-17.06% and 9.08%-70.25%, respectively.

Key words: short-term load forecasting, Successive Variational Mode Decomposition (SVMD), multi-scale features, dual attention, Transformer module

摘要：

短期电力负荷预测在电力系统的优化调度和安全运行中具有至关重要的作用。电力负荷数据具有多周期特性，在不同时间尺度上表现出不同的模式和趋势，准确提取尺度大小有助于识别和分离这些特征。目前方法通过使用一个或一组固定的patch长度作为步长，将称之为patches的片段来编码时间序列，但其无法适应现实世界负荷序列数据的复杂的动态变化。为此，提出一种基于动态多尺度与双重注意力的预测模型(MDAT)。首先，利用逐次变分模态分解(SVMD)分离负荷序列不同的时间模式，通过快速傅里叶变换(FFT)提取出每个模式的显著周期。其次，根据检测到的显著周期，将负荷序列以不同大小的patch划分为不同的时间分辨率，使用Transformer的多个分支同时建模不同尺度分割序列的依赖关系。然后，对这些patches进行双重注意力，以捕获全局相关性和局部细节。最后，对每个分支的输出进行非线性特征融合，通过堆叠多层Transformer模块得到最终的负荷预测结果。在两个公开数据集上的实验结果表明，该模型在预测精度指标上表现良好，相比最新的基于Transformer及多层感知器(MLP)的模型，在Australia数据集和Morocco数据集上平均绝对误差(MAE)分别降低了10.26%~17.06%和9.08%~70.25%。

关键词: 短期负荷预测, 逐次变分模态分解, 多尺度特征, 双重注意力, Transformer模块

ZHU Li, GAO Jingkai, ZHU Chunqiang, DENG Fan. Short-term Power Load Forecasting Based on Dynamic Multi-Scale and Dual Attention Mechanisms[J]. Computer Engineering, 2025, 51(10): 369-380.

朱莉, 高靖凯, 朱春强, 邓凡. 基于动态多尺度与双重注意力的短期电力负荷预测[J]. 计算机工程, 2025, 51(10): 369-380.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0070067

https://www.ecice06.com/EN/Y2025/V51/I10/369

Figures/Tables 16

Fig.1 Overall architecture of MDAT model

Fig.2 Significant period detection procedure

Fig.3 Dual attention mechanism

Fig.4 Adaptive loss function

Fig.5 SVMD algorithm decomposition results and their spectrograms

Fig.6 Heat maps of attention scores encoded by absolute and relative positions

Fig.7 Scatter plot of MDAT prediction results

Fig.8 Prediction performance comparison bubble chart

Fig.9 Comparison of prediction curves of different models on the Australia dataset

References 34

1	EREN Y, KÜÇÜKDEMIRAL. A comprehensive review on deep learning approaches for short-term load forecasting. Renewable and Sustainable Energy Reviews, 2024, 189, 114031. doi: 10.1016/j.rser.2023.114031
2	WAZIRALI R, YAGHOUBI E, ABUJAZAR M S S, et al. State-of-the-art review on energy and load forecasting in microgrids using artificial neural networks, machine learning, and deep learning techniques. Electric Power Systems Research, 2023, 225, 109792. doi: 10.1016/j.epsr.2023.109792
3	TARMANINI C, SARMA N, GEZEGIN C, et al. Short term load forecasting based on ARIMA and ANN approaches. Energy Reports, 2023, 9, 550- 557.
4	SMYL S, DUDEK G, PEȽKA P. Contextually enhanced ES-dRNN with dynamic attention for short-term load forecasting. Neural Networks, 2024, 169, 660- 672. doi: 10.1016/j.neunet.2023.11.017
5	SAYED H A, WILLIAM A, SAID A M. Smart electricity meter load prediction in Dubai using MLR, ANN, RF, and ARIMA. Electronics, 2023, 12(2): 389. doi: 10.3390/electronics12020389
6	AN W H, GAO B, LIU J H, et al. Predicting hourly heating load in residential buildings using a hybrid SSA-CNN-SVM approach. Case Studies in Thermal Engineering, 2024, 59, 104516. doi: 10.1016/j.csite.2024.104516
7	WANG F, CHEN C, ZHANG H, et al. Short-term load forecasting based on variational mode decomposition and chaotic grey wolf optimization improved random forest algorithm. Journal of Applied Science and Engineering, 2022, 26(1): 69- 78.
8	INDIRA G, BHAVANI M, BRINDA R, et al. Electricity load demand prediction for microgrid energy management system using hybrid adaptive barnacle-mating optimizer with artificial neural network algorithm. Energy Technology, 2024, 12(5): 2301091. doi: 10.1002/ente.202301091
9	LU Y T, WANG G C, HUANG X F, et al. Probabilistic load forecasting based on quantile regression parallel CNN and BiGRU networks. Applied Intelligence, 2024, 54(15): 7439- 7460.
10	陆心怡, 关艳, 高曦莹, 等. 面向工业用户的混合DWT-DE-RNN电力负荷预测. 机械设计与制造, 2024, 404(10): 73- 78.
	LU X Y, GUAN Y, GAO X Y, et al. Hybrid DWT-DE-RNN power load forecasting for industrial users. Machinery Design & Manufacture, 2024, 404(10): 73- 78.
11	SMYL S, DUDEK G, PELKA P. ES-dRNN: a hybrid exponential smoothing and dilated recurrent neural network model for short-term load forecasting. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(8): 11346- 11358. doi: 10.1109/TNNLS.2023.3259149
12	BARETH R, YADAV A, GUPTA S, et al. Daily average load demand forecasting using LSTM model based on historical load trends. IET Generation, Transmission & Distribution, 2024, 18(5): 952- 962.
13	李丽芬, 张近月, 曹旺斌, 等. 基于双重注意力时间卷积长短期记忆网络的短期负荷预测[J]. 系统仿真学报, 2024: 1-12[2024-06-29]. https://doi.org/10.16182/j.issn1004731x.joss.24-0282.
	LI L F, ZHANG J Y, CAO W B, et al. Short-term load prediction based on dual-attention temporal convolutional long short-term memory network[J/OL]. Journal of System Simulation, 2024: 1-12[2024-06-29]. https://doi.org/10.16182/j.issn1004731x.joss.24-0282. (in Chinese)
14	李新, 张旭, 余乐安, 等. 基于改进Transformer模型的景区短时客流预测研究. 中国管理科学, 2005(2): 13- 20.
	LI X, ZHANG X, YU A L, et al. Enhancing short-term tourist flow forecasting and evaluation using an improved Transformer framework. Chinese Journal of Management Science, 2005(2): 13- 20.
15	李浩阳, 贺小伟, 王宾, 等. 基于改进Informer的云计算资源负载预测. 计算机工程, 2024, 50(2): 43- 50. doi: 10.19678/j.issn.1000-3428.0066399
	LI H Y, HE X W, WANG B, et al. Cloud computing resource load prediction based on improved informer. Computer Engineering, 2024, 50(2): 43- 50. doi: 10.19678/j.issn.1000-3428.0066399
16	DU D Z, SU B, WEI Z W. Preformer: predictive transformer with multi-scale segment-wise correlations for long-term time series forecasting[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Washington D. C., USA: IEEE Press, 2023: 1-5.
17	NIE Y, NGUYEN N H, SINTHONG P, et al. A time series is worth 64 words: long-term forecasting with Transformers[C]//Proceedings of the 12th International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2021: 157-168.
18	YU G, ZOU J, HU X, et al. Revitalizing multivariate time series forecasting: learnable decomposition with inter-series dependencies and intra-series variations modeling[C]//Proceedings of the 41st International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2020: 457-468.
19	CHEN P, ZHANG Y Y, CHENG Y Y, et al. Pathformer: multi-scale transformers with adaptive pathways for time series forecasting[EB/OL]. [2024-06-01]. https://arxiv.org/abs/2402.05956v5.
20	付文龙, 章轩瑞, 张海荣, 等. 多尺度特征提取与非线性融合的综合能源系统多元负荷短期预测. 电力系统及其自动化学报, 2023, 35(12): 89- 99.
	FU W L, ZHANG X R, ZHANG H R, et al. Short-term multivariate load forecasting of integrated energy system based on multiscale feature extraction and nonlinear fusion. Proceedings of the CSU-EPSA, 2023, 35(12): 89- 99.
21	孟衡, 张涛, 王金, 等. 基于多尺度时空图卷积网络与Transformer融合的多节点短期电力负荷预测方法. 电网技术, 2024, 48(10): 4297- 4305.
	MENG H, ZHANG T, WANG J, et al. Multi-node short-term power load forecasting method based on multi-scale spatiotemporal graph convolution network and Transformer. Power System Technology, 2024, 48(10): 4297- 4305.
22	张鹏飞, 胡博, 胡展硕, 等. 基于STD-ST-Former的现货电价长步时空预测. 中国电机工程学报, 2024, 44(7): 2732- 2743.
	ZHANG P F, HU B, LUO Z S, et al. Long step spatial-temporal prediction of spot electricity price using dual channel ST-Former framework based on seasonal trend decomposition. Proceedings of the CSEE, 2024, 44(7): 2732- 2743.
23	DU L F, XIN J, LABACH A, et al. MultiResFormer: transformer with adaptive multi-resolution modeling for general time series forecasting[EB/OL]. [2024-06-01]. https://arxiv.org/abs/2311.18780v2.
24	MO S T, WANG H X, LI B X, et al. TimeSQL: improving multivariate time series forecasting with multi-scale patching and smooth quadratic loss. Information Sciences, 2024, 671, 120652. doi: 10.1016/j.ins.2024.120652
25	ZHANG Y T, MA L H, PAL S, et al. Multi-resolution time-series transformer for long-term forecasting[EB/OL]. [2024-06-01]. https://arxiv.org/abs/2311.04147v2.
26	ZENG A L, CHEN M X, ZHANG L, et al. Are transformers effective for time series forecasting?. Artificial Intelligence, 2023, 37(9): 11121- 11128.
27	NAZARI M, SAKHAEI S M. Successive variational mode decomposition. Signal Processing, 2020, 174, 107610. doi: 10.1016/j.sigpro.2020.107610
28	LUO X Y, WANG H, HAN T, et al. FFT-trans: enhancing robustness in mechanical fault diagnosis with Fourier transform-based transformer under noisy conditions. IEEE Transactions on Instrumentation Measurement, 2024, 73, 3381688.
29	BARRON J T. A general and adaptive robust loss function[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 4331-4339.
30	ZHANG J L, ZHANG Y J, LI D Z, et al. Forecasting day-ahead electricity prices using a new integrated model. International Journal of Electrical Power & Energy Systems, 2019, 105, 541- 548.
31	SALAM A, EL HIBAOUI A. Comparison of machine learning algorithms for the power consumption prediction[C]//Proceedings of the 6th International Renewable and Sustainable Energy Conference. Washington D. C., USA: IEEE Press, 2018: 1-5.
32	LIU S, YU H, LIAO C, et al. Pyraformer: low-complexity pyramidal attention for long-range time series modeling and forecasting[C]//Proceedings of International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2021: 256-267.
33	ZHOU T, MA Z Q, WEN Q S, et al. FEDformer: frequency enhanced decomposed transformer for long-term series forecasting[EB/OL]. [2024-06-01]. https://arxiv.org/abs/2201.12740v3.
34	ZHANG T P, ZHANG Y Z, CAO W, et al. Less is more: fast multivariate time series forecasting with light sampling-oriented MLP structures[EB/OL]. [2024-06-01]. https://arxiv.org/abs/2207.01186v1.

[1]	MA Gan, GU Yu, PENG Dongliang. Combining Improved YOLOv5s and Dynamic Data Augmentation for Sea Surface Ship Detection [J]. Computer Engineering, 2025, 51(9): 294-305.
[2]	MIAO Ru, LI Yi, ZHOU Ke, ZHANG Yanna, CHANG Ranran, MENG Geng. A Study on Improved Faster R-CNN Model for Multi-Object Detection in Remote Sensing Images [J]. Computer Engineering, 2025, 51(8): 292-304.
[3]	Huanyu LU, Yonghong ZHANG, Guangyi MA, Donglin XIE, Wei TIAN. Semi-Supervised Adversarial Learning-Based Water Body Extraction from Remote Sensing Images [J]. Computer Engineering, 2024, 50(7): 251-263.
[4]	YU Yang, SUN Fangfang, LV Hua, LI Yang, WANG Xiaomin. Micro-Expression Detection Method Based on Multi-Scale Spatiotemporal Attention Network [J]. Computer Engineering, 2024, 50(6): 228-235.
[5]	Zhina SONG, Sha LI, Jianming YANG, Chuan XU. Remote Sensing Ship Target Detection Based on Feature and Region Localization Enhancement [J]. Computer Engineering, 2023, 49(8): 257-264.
[6]	ZHAO Guochuan, WANG Heng, ZHANG Hua, PANG Jie, ZHOU Jian. Hydropower Complex Defect Recognition Method Based on Pure Self-Attention [J]. Computer Engineering, 2022, 48(9): 277-285.
[7]	LI Xiao, LU Xianling. ethod for Forecasting Short-Term Power Load Based on Dual-Stage Attention Mechanism and Gated Recurrent Unit Network [J]. Computer Engineering, 2022, 48(2): 291-296,305.
[8]	CAO Jiawang, TIAN Weiwei, LIU Xueling, LI Yuxin, FENG Rui. Segmentation of Brain Substantia Nigra Pars Compacta Based on Improved U-Net [J]. Computer Engineering, 2022, 48(11): 14-21,29.
[9]	ZHEN Cheng, YANG Yongsheng, LI Yuanxiang, ZHONG Juanjuan. Atmospheric Turbulence Image Restoration Based on Multi-Scale Generative Adversarial Network [J]. Computer Engineering, 2021, 47(11): 227-233.
[10]	JI Xiuyi, LI Jianhua. Chemical Structure Image Recognition Based on Dual Attention Mechanism [J]. Computer Engineering, 2020, 46(9): 213-220.
[11]	ZHANG Zhichang, ZHOU Tong, ZHANG Ruifang, ZHANG Minyu. Medical Entity Relation Recognition Combining Bidirectional GRU and Attention [J]. Computer Engineering, 2020, 46(6): 296-302.

Please choose a citation manager

Content to export