Atmospheric Turbulence Image Restoration Based on Multi-Scale Generative Adversarial Network

doi:10.19678/j.issn.1000-3428.0059695

Abstract

Abstract: Atmospheric turbulence can produce distortions and blurs in images.To restore a single image affected by atmospheric turbulence,an image restoration method is proposed based on a multi-scale Generative Adversarial Network(GAN).Based on the GAN architecture,the method introduces units for the extraction of multi-scale attention features and dynamic fusion of multi-level features to enlarge the receptive field.On this basis,a feature fusion mechanism is added to implement restoration of images affected by atmospheric turbulence.The experimental results show that compared with the standard GAN and the SIU-Net model,the proposed multi-scale GAN can significantly improve the visual quality of images,and reduce the blurs and geometric distortions.

Key words: image restoration, image reconstruction, Generative Adversarial Network(GAN), atmospheric turbulence effect, multi-scale features

摘要： 大气湍流会导致图像发生畸变和模糊，为对单幅大气湍流退化图像进行复原，提出一种基于多尺度生成对抗网络（GAN）的图像复原方法。采用GAN框架，在生成器网络中添加多尺度注意力特征提取单元与多层次特征动态融合单元，从而提升模型的感受野范围。在此基础上，引入特征融合机制以实现湍流退化图像复原。实验结果表明，相比标准GAN、SIU-Net模型，多尺度GAN能显著提高图像的视觉质量，有效降低图像的模糊和几何畸变程度。

关键词: 图像复原, 图像重建, 生成对抗网络, 大气湍流效应, 多尺度特征

CLC Number:

TP391.4

ZHEN Cheng, YANG Yongsheng, LI Yuanxiang, ZHONG Juanjuan. Atmospheric Turbulence Image Restoration Based on Multi-Scale Generative Adversarial Network[J]. Computer Engineering, 2021, 47(11): 227-233.

甄诚, 杨永胜, 李元祥, 钟娟娟. 基于多尺度生成对抗网络的大气湍流图像复原[J]. 计算机工程, 2021, 47(11): 227-233.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0059695

http://www.ecice06.com/EN/Y2021/V47/I11/227

Figures/Tables 9

References

[1] YITZHAKY,YITZHAK.Restoration of atmospherically blurred images according to weather-predicted atmospheric modulation transfer functions[J].Optical Engineering,1997,36(11):3064-3072.
[2] SHACHAM O,HAIK O,YITZHAKY Y.Blind restoration of atmospherically degraded images by automatic best step-edge detection[J].Pattern Recognition Letters,2007,28(15):2094-2103.
[3] 陈云龙,王平,王鹏.基于LR非线性迭代的降质图像复原算法[J].计算机工程,2010,36(4):202-204. CHEN Y L,WANG P,WANG P.Degradation image restoration algorithm based on LR nonlinear iteration[J].Computer Engineering,2010,36(4):202-204.(in Chinese)
[4] LI D,MERSEREAU R M,SIMSKE S.Atmospheric turbulence-degraded image restoration using principal components analysis[J].IEEE Geoscience and Remote Sensing Letters,2007,4(3):340-344.
[5] HIRSCH M,SRA S,SCHÖLKOPF B,et al.Efficient filter flow for space-variant multiframe blind deconvolution[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2010:607-614.
[6] JOHN S,VORONTSOV M A.Multiframe selective information fusion from robust error estimation theory[J].IEEE Transactions on Image Processing,2005,14(5):577-584.
[7] AUBAILLY M,VORONTSOV M A,CARHART G W,et al.Automated video enhancement from a stream of atmospherically-distorted images:the lucky-region fusion approach[EB/OL].[2020-09-05].https://ui.adsabs.harvard.edu/abs/2009SPIE.7463E..0CA/abstract.
[8] ZHU X,MILANFAR P.Removing atmospheric turbulence via space-invariant deconvolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,35(1):157-170.
[9] LUCAS A,ILIADIS M,MOLINA R,et al.Using deep neural networks for inverse problems in imaging:beyond analytical methods[J].IEEE Signal Processing Magazine,2018,35(1):20-36.
[10] LAU C P,HOSSEIN S,RAMA C.ATFaceGAN:single face image restoration and recognition from atmospheric turbulence[EB/OL].[2020-09-05].https://arxiv.org/pdf/1910.03119.pdf.
[11] XIE Y,ZHANG W,TAO D,et al.Removing turbulence effect via hybrid total variation and deformation-guided kernel regression[J].IEEE Transactions on Image Processing,2016,25(10):4943-4958.
[12] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[EB/OL].[2020-09-05].https://www.cse.fau.edu/~xqzhu/courses/cap6619/5423-generative-adversarial-nets.pdf.
[13] KUPYN O,BUDZAN V,MYKHAILYCH M,et al.Deblurgan:blind motion deblurring using conditional adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:8183-8192.
[14] MAO X,LI Q,XIE H,et al.Least squares generative adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:2794-2802.
[15] RONNEBERGER O,FISCHER P,BROX T.U-Net:convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin,Germany:Springer,2015:234-241.
[16] ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:1125-1134.
[17] SZEGEDY C,IOFFE S,VANHOUCKE V,et al.Inception-V4,inception-ResNet and the impact of residual connections on learning[EB/OL].[2020-09-05].https://arxiv.org/pdf/1602.07261.pdf.
[18] 熊亚辉,陈东方,王晓峰.基于多尺度反向投影的图像超分辨率重建算法[J].计算机工程,2020,46(7):251-259. XIONG Y H,CHEN D F,WANG X F.Super-resolution image reconstruction algorithm based on multi-scale back projection[J].Computer Engineering,2020,46(7):251-259.(in Chinese)
[19] HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:7132-7141.
[20] LIU S,HUANG D,WANG Y.Learning spatial fusion for single-shot object detection[EB/OL].[2020-09-05].https://arxiv.org/pdf/1911.09516.pdf.
[21] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2020-09-05].https://arxiv.org/pdf/1409.1556.pdf.
[22] LAU C P,LAI Y H,LUI L M,et al.Variational models for joint subsampling and reconstruction of turbulence-degraded images[J].Journal of Scientific Computing,2019,78(3):1488-1525.
[23] YE M,LYU D,CHEN G.Scale-iterative upscaling network for image deblurring[J].IEEE Access,2020,8:18316-18325.

Please choose a citation manager

Content to export