Remote Sensing Cloud Image Prediction Method Based on Multi-scale Motion Memory Model

doi:10.19678/j.issn.1000-3428.0069950

Abstract

Abstract:

Existing deep learning models find it difficult to capture cloud motion patterns, resulting in long-term cloud prediction results that are fuzzy and low in accuracy. To address this problem, this study proposes a remote sensing cloud image prediction method based on a Multi-Scale Motion Memory Network (MSMM_Net). This model adopts a dual-branch memory-flow architecture that combines spatial multi-scale and motion-differential memory flows. It extracts high- and low-frequency spatial features and sequence motion features hidden in the input image sequence, thereby simultaneously obtaining global, detail, and motion information of the image. In the prediction stage, dual-branch memory is fused to alleviate the problem of feature loss and enhance the ability of the model to predict the trajectory of cloud clusters. On this basis, a fusion loss function combining pixel and edge losses is used to guide model training, enhance the model's attention to image edge details, and promote the generation of clear predicted images. Experimental results show that, compared with the benchmark model PredRNN, MSMM_Net reduces the Mean Square Error (MSE) by 31.71% on the Moving MNIST dataset and the Learned Perceptual Image Patch Similarity (LPIPS) by 64.7%. On the remote sensing satellite cloud image dataset, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) indicators improve by 5.51% and 5.38%, respectively, indicating that the predicted image sequence generated by the model is more similar to the real image sequence and can effectively improve long-term prediction accuracy.

Key words: cloud image prediction, spatio-temporal sequence prediction, deep learning, recurrent convolutional network, remote sensing satellite

摘要：

针对现有深度学习模型难以捕获云团运动模式导致云图长期预测结果模糊、准确度低的问题, 提出一种基于多尺度运动记忆模型(MSMM_Net)的遥感云图预测方法。该模型采用空间多尺度记忆流和运动差分记忆流相融合的双分支记忆流架构, 分别提取输入图片序列隐含的高低频空间特征和序列运动特征, 从而同时获得图片的全局信息、细节信息和运动信息, 在预测阶段融合双分支记忆, 缓解特征丢失问题并增强模型对云团运动轨迹的预测能力。在此基础上, 使用像素损失和边缘损失相结合的融合损失函数指导模型的训练, 强化模型对图片边缘细节的关注度, 促使模型生成清晰的预测图片。实验结果表明, 与基准模型PredRNN相比, MSMM_Net在Moving MNIST数据集上的均方误差(MSE)降低了31.71%, 在可学习感知图像块相似性指标(LPIPS)上降低了64.7%, 在遥感卫星云图数据集上, 峰值信噪比(PSNR)和结构相似性指数(SSIM)指标分别提升了5.51%和5.38%, 表明该模型生成的预测图片序列与真实图片序列更加相似, 能够有效提升长期预测准确率。

关键词: 云图预测, 时空序列预测, 深度学习, 循环卷积网络, 遥感卫星

ZHANG Yonghong, SUN Shulin, GONG Meng, WANG Junfei, MA Guangyi. Remote Sensing Cloud Image Prediction Method Based on Multi-scale Motion Memory Model[J]. Computer Engineering, 2026, 52(3): 128-140.

张永宏, 孙书林, 龚蒙, 王俊飞, 马光义. 基于多尺度运动记忆模型的遥感云图预测方法[J]. 计算机工程, 2026, 52(3): 128-140.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069950

https://www.ecice06.com/EN/Y2026/V52/I3/128

Figures/Tables 15

Fig.1 Overall structure of multi-scale motion memory network

Fig.2 PredRNN network structure

Fig.3 MDSTM structure

Fig.4 Motion difference attention module

Fig.5 Comparison example of high and low threshold values for Canny edge detection algorithm

Fig.6 Example of Moving MNIST dataset

Fig.7 Example of the 13th band of GOES-16 satellite

Fig.8 Trends in evaluation metrics on the Moving MNIST dataset

Fig.9 Comparison of prediction effect on the Moving MNIST dataset

Fig.10 Trend of evaluation indicators on the GOES-16 dataset

Fig.11 Comparison 1 of prediction effect of GOES-16 dataset

Fig.12 Comparison 2 of prediction effect of GOES-16 dataset

References 36

1	张弛, 刘钧, 李旭光, 等. 基于可见光-红外图像信息融合的云状识别方法. 气象与环境学报, 2018, 34(1): 82- 90.
	ZHANG C, LIU J, LI X G, et al. Cloud recognition method based on visible-infrared image information fusion. Journal of Meteorology and Environment, 2018, 34(1): 82- 90.
2	XIAN D, ZHANG P, GAO L, et al. Fengyun meteorological satellite products for earth system science applications. Advances in Atmospheric Sciences, 2021, 38(8): 1267- 1284. doi: 10.1007/s00376-021-0425-3
3	方旭源, 金炜, 符冉迪, 等. 注意力机制下多尺度特征融合生成对抗网络的白天海雾监测. 遥感学报, 2023, 27(12): 2736- 2747.
	FANG X Y, JIN W, FU R D, et al. Daytime sea fog detection based on multi-scale feature fusion of generated adversarial network under attention mechanism. National Remote Sensing Bulletin, 2023, 27(12): 2736- 2747.
4	ROJAS Y, MINDER J R, CAMPBELL L S, et al. Assessment of GPM IMERG satellite precipitation estimation and its dependence on microphysical rain regimes over the mountains of south-central Chile. Atmospheric Research, 2021, 253, 105454. doi: 10.1016/j.atmosres.2021.105454
5	黄杰, 张永宏, 马光义, 等. 改进U-Net的风云四号卫星降水估计算法研究. 计算机工程与应用, 2023, 59(11): 285- 293.
	HUANG J, ZHANG Y H, MA G Y, et al. Research on precipitation estimation algorithm from Fengyun-4 satellite based on improved U-Net. Computer Engineering and Applications, 2023, 59(11): 285- 293.
6	郑倩, 毛程燕, 丁丽华, 等. 台风利奇马(1909)与台风摩羯(1814)云特征对比. 应用气象学报, 2022, 33(1): 43- 55.
	ZHENG Q, MAO C Y, DING L H, et al. Comparison of cloud characteristics between typhoon Lekima(1909)and typhoon Yagi(1814). Journal of Applied Meteorological Science, 2022, 33(1): 43- 55.
7	宁子杰, 包文雯, 陈峥蓉, 等. 台风"浪卡"(2016)暴雨特点及卫星云图特征分析. 气象研究与应用, 2021, 42(3): 88- 93.
	NING Z J, BAO W W, CHEN Z R, et al. Typhoon Nangka(2016) rainstorm characteristics and satellite cloud image characteristics analysis. Journal of Meteorological Research and Application, 2021, 42(3): 88- 93.
8	郑宗生, 刘敏, 胡晨雨, 等. 基于Seq2Seq和Attention的时序卫星云图台风等级预测. 遥感信息, 2020, 35(4): 16- 22.
	ZHENG Z S, LIU M, HU C Y, et al. Prediction of typhoon grade with time series typhoon satellite images based on Seq2Seq and Attention. Remote Sensing Information, 2020, 35(4): 16- 22.
9	BOSCH J L, KLEISSL J. Cloud motion vectors from a network of ground sensors in a solar power plant. Solar Energy, 2013, 95, 13- 20. doi: 10.1016/j.solener.2013.05.027
10	HUANG H, XU J, PENG Z Z, et al. Cloud motion estimation for short term solar irradiation prediction[C]//Proceedings of the IEEE International Conference on Smart Grid Communications (SmartGridComm). Washington D.C., USA: IEEE Press, 2013: 696-701.
11	王磊, 周毓荃, 蔡淼, 等. 华北云特征参数与降水相关性的研究. 气象与环境科学, 2019, 42(3): 9- 16.
	WANG L, ZHOU Y Q, CAI M, et al. Study on correlation between cloud characteristic parameters and precipitation in North China. Meteorological and Environmental Sciences, 2019, 42(3): 9- 16.
12	吴现, 吐松江·卡日, 王海龙, 等. 基于时空长短时记忆神经网络的地基云图预测算法. 计算机工程, 2024, 50(3): 298- 305. doi: 10.19678/j.issn.1000-3428.0067504
	WU X, TUSONGJIANG K, WANG H L, et al. Ground-based cloud map prediction algorithm based on spatio-temporal long short-term memory neural network. Computer Engineering, 2024, 50(3): 298- 305. doi: 10.19678/j.issn.1000-3428.0067504
13	MA B, YANG N, CUI C G, et al. Cloud position forecasting based on ConvLSTM network[C]//Proceedings of the 5th International Conference on Power and Renewable Energy (ICPRE). Washington D.C., USA: IEEE Press, 2020: 562-565.
14	XU Z, DU J, WANG J J, et al. Satellite image prediction relying on GAN and LSTM neural networks[C]//Proceedings of the 2019 IEEE International Conference on Communications (ICC). Washington D.C., USA: IEEE Press, 2019: 1-6.
15	严义杰. 基于深度学习的云移动预测方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2020.
	YAN Y J. Research on cloud movement forecasting method based on deep learning[D]. Harbin: Harbin Institute of Technology, 2020. (in Chinese)
16	SHIELDS J, KARR M, TOOMAN T, et al. The whole sky imager-a year of progress[EB/OL]. [2024-05-05]. https://www.semanticscholar.org/paper/The-Whole-Sky-Imager-A-Year-of-Progress-Shields-Karr/239291dffb867b02e5e7ab97c6d215ff60e98ddf.
17	BRAD R, LETIA I A. Cloud motion detection from infrared satellite images[EB/OL]. [2024-05-05]. https://www.researchgate.net/publication/228557524_Cloud_motion_detection_from_infrared_satellite_images.
18	JAMALY M, KLEISSL J. Robust cloud motion estimation by spatio-temporal correlation analysis of irradiance data. Solar Energy, 2018, 159, 306- 317. doi: 10.1016/j.solener.2017.10.075
19	GIBSON J J. The perception of the visual world. Boston, USA: Houghton Mifflin, 1950.
20	WOOD-BRADLEY P, ZAPATA J, PYE J. Cloud tracking with optical flow for short-term solar forecasting[EB/OL]. [2024-05-05]. https://www.semanticscholar.org/paper/Cloud-tracking-with-optical-flow-for-short-term-Wood-Bradley-Zapata/fe698691c892cae10b8a81a5e2b90f24809b2f74/figure/0.
21	DISSAWA D M L H, EKANAYAKE M P B, GODALIYADDA G M R I, et al. Cloud motion tracking for short-term on-site cloud coverage prediction[C]//Proceedings of the 17th International Conference on Advances in ICT for Emerging Regions (ICTer). Washington D.C., USA: IEEE Press, 2018: 1-6.
22	CHOW C W, BELONGIE S, KLEISSL J. Cloud motion and stability estimation for intra-hour solar forecasting. Solar Energy, 2015, 115, 645- 655. doi: 10.1016/j.solener.2015.03.030
23	石玉立, 施声伟. 光流算法在FY-4A红外图像外推中的精度评价研究. 兵器装备工程学报, 2021, 42(9): 150-158, 224.
	SHI Y L, SHI S W. Research on accuracy evaluation of optical flow algorithm in FY-4A infrared image extrapolation. Journal of Ordnance Equipment Engineering, 2021, 42(9): 150-158, 224.
24	SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1506.04214.
25	SHI X J, GAO Z H, LAUSEN L, et al. Deep learning for precipitation nowcasting: a benchmark and a new model[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1706.03458.
26	WANG Y B, LONG M S, WANG J M, et al. PredRNN: recurrent neural networks for predictive learning using spatiotemporal LSTMs[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2103.09504.
27	WANG Y B, GAO Z F, LONG M S, et al. PredRNN++: towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1804.06300.
28	TREBING K, STAǸCZYK T, MEHRKANOON S. SmaAt-UNet: precipitation nowcasting using a small attention-UNet architecture. Pattern Recognition Letters, 2021, 145, 178- 186. doi: 10.1016/j.patrec.2021.01.036
29	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1807.06521.
30	SEIF G, ANDROUTSOS D. Edge-based loss function for single image super-resolution[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D.C., USA: IEEE Press, 2018: 1468-1472.
31	SRIVASTAVA N, MANSIMOV E, SALAKHUTDINOV R. Unsupervised learning of video representations using LSTMs[C]//Proceedings of the 32nd International Conference on Machine Learning. Washington D.C., USA: IEEE Press, 2015: 843-852.
32	WANG Y, JIANG L, YANG M H, et al. Eidetic 3D LSTM: a model for video prediction and beyond[EB/OL]. [2024-05-05]. https://openreview.net/pdf?id=B1lKS2AqtX.
33	YU W, LU Y, EASTERBROOK S, et al. Efficient and information-preserving future frame prediction and beyond[EB/OL]. [2024-05-05]. https://openreview.net/pdf?id=B1eY_pVYvB.
34	PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1912.01703.
35	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 586-595.
36	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600- 612. doi: 10.1109/TIP.2003.819861

[1]	ZHANG Zhi, YIN Yukai, SUN Yiling, MENG Wenjing, PENG Chang. Research on Android Malware Detection Model Based on Multi-modal Feature Fusion [J]. Computer Engineering, 2026, 52(3): 243-254.
[2]	LIU Xiaoyu, LIAO Zhifang, TAN Sui, YU Zhiwu. Bridge Dynamic Strain Prediction Based on Stacked GRU Neural Network [J]. Computer Engineering, 2026, 52(3): 441-450.
[3]	CAO Jiwei, LUO Fei, DING Weichao. BS-YOLO: A Small Object Detection Algorithm Based on BSAM Attention Mechanism and SCConv [J]. Computer Engineering, 2026, 52(3): 119-127.
[4]	QIN Yingxin, ZHANG Kejia, PAN Haiwei, JU Yahao. Adversarial Attacks in Computer Vision: A Survey [J]. Computer Engineering, 2026, 52(2): 46-68.
[5]	ZHAO Xudong, WU Hongyue, MENG Ke, XU Xiaolong, DOU Wanchun. Research Progress and Prospects of Service Recommendation Methods (Invited) [J]. Computer Engineering, 2026, 52(1): 61-75.
[6]	CHEN Liang, ZHAO Ying, SHI Shenghui, YIN Ling. Link Prediction Method Based on Hypergraph Neural Network [J]. Computer Engineering, 2026, 52(1): 136-143.
[7]	XU Ying, FU Ziwei, ZHANG Wei, CHEN Yunfang. Smart Contract Vulnerability Detection Technology Based on Abstract Syntax Tree Embedding [J]. Computer Engineering, 2025, 51(9): 149-157.
[8]	HUANG Jingui, LIU Peng, TANG Wensheng. MMD-YOLOv7: Vehicle Detection Method Under Dark Conditions [J]. Computer Engineering, 2025, 51(9): 340-349.
[9]	ZHOU Chenyang, LIU Xueyu, LIANG Shaohua, WU Yongfei. Segmentation and Quantitative Analysis of Renal Artery Vessel Detection Based on Swin Transformer [J]. Computer Engineering, 2025, 51(9): 252-267.
[10]	HAO Hongda, LUO Jianxu. Multi-Organ Semantic Segmentation Model Based on Multi-Scale Region Feature Fusion [J]. Computer Engineering, 2025, 51(8): 270-280.
[11]	WU Donghui, WANG Jinfeng, QIU Sen, LIU Guozhi. Sign Language Recognition Using Data Gloves Based on EWBiLSTM-ATT [J]. Computer Engineering, 2025, 51(8): 107-119.
[12]	LIN Fan, LI Jianhua. Optical Chemical Structure Recognition Based on Multi-order Gated Aggregation Network [J]. Computer Engineering, 2025, 51(8): 364-372.
[13]	WU Donghui, WANG Jinfeng, QIU Sen, LIU Guozhi. Sign Language Recognition Using Data Gloves Based on EWBiLSTM-ATT [J]. Computer Engineering, 2025, 51(8): 107-119.
[14]	YU Peng, YANG Jiaqi, CHEN Xinran, HE Chaobo. Feature-enhanced Recommendation Algorithm Based on Bipartite Graph Contrastive Learning [J]. Computer Engineering, 2025, 51(7): 100-110.
[15]	LI Jiangxin, WANG Peng, WANG Wei. Multi-mechanism-guided Deep Learning Framework for Industrial Time-series Forecasting [J]. Computer Engineering, 2025, 51(7): 47-58.

Please choose a citation manager

Content to export