基于生成式对抗网络的图像超分辨率重建算法

doi:10.19678/j.issn.1000-3428.0057168

摘要/Abstract

摘要： 针对传统图像超分辨率重建算法存在网络训练困难与生成图像存在伪影的问题，提出一种利用生成式对抗网络的超分辨率重建算法。去除生成式对抗网络的批量归一化层降低计算复杂度，将其中的残差块替换为密集残差块构成生成网络，使用VGG19网络作为判别网络的基础框架，以全局平均池化代替全连接层防止过拟合，引入纹理损失函数、感知损失函数、对抗损失函数和内容损失函数构成生成器的总目标函数，利用纹理损失增强局部信息匹配度，采用激活层前的特征信息计算感知损失获取更多细节特征，使用WGAN-GP理论优化网络模型的对抗损失加速收敛，运用内容损失提升图像低频信息的准确性。实验结果表明，该算法重建图像的平均峰值信噪比为27.97 dB，平均结构相似性为0.777，与SRGAN和EDSR等算法相比，其在未延长较多运行时间的情况下，重建结果的纹理细节更清晰且亮度信息更准确，更符合视觉感官评价要求。

关键词: 超分辨率重建, 生成式对抗网络, 密集卷积网络, 纹理损失, 梯度惩罚Wasserstein生成式对抗网络

Abstract: The existing image Super-Resolution (SR) reconstruction algorithms have difficulty in network training and cause artifacts in the generated images.To address the problem,this paper proposes a SR reconstruction algorithm based on Generative Adversarial Networks (GAN).The Batch Normalization(BN)layer of the Generative Adversarial Networks(SRGAN)is removed to reduce the computational complexity,and the Residual Block (RB) is replaced by Residual Dense Block (RDB) to form the generative network.VGG19 network is used as the basic framework of the discrimination network.The Global Average Pooling (GAP)is used to replace the full connection layer to prevent over fitting.Texture loss function, perceptual loss function, adversarial loss function and content loss function are introduced to form the overall objective function of the generator. Texture loss is used to enhance the matching degree of local information, and the feature information in front of the activation layer is used to calculate the perceptual loss to obtain more detailed features. Wasserstein Generative Adversarial Nets-Gradient Penalty(WGAN-GP) theory is used to optimize the adversarial loss of the network model to accelerate the convergence, and content loss is used to improve the accuracy of low-frequency information of the image.Experimental results show that the average Peak Signal to Noise Ratio (PSNR) of the image reconstructed by the proposed algorithm is 27.97 dB,and its average Structural Similarity(SSIM)is 0.777.Compared with SRGAN and EDSR algorithms,the proposed algorithm improves the texture details and brightness of the reconstructed image,making it more in line with the requirements of visual sensory evaluation without prolonging much running time.

Key words: Super-Resolution(SR) reconstruction, Generative Adversarial Networks(GAN), Dense Convolutional Networks(DenseNet), texture loss, Wasserstein Generative Adversarial Nets-Gradient Penalty (WGAN-GP)

中图分类号:

TP391.41

姜玉宁, 李劲华, 赵俊莉. 基于生成式对抗网络的图像超分辨率重建算法[J]. 计算机工程, 2021, 47(3): 249-255.

JIANG Yuning, LI Jinhua, ZHAO Junli. Image Super-Resolution Reconstruction Algorithm Based on Generative Adversarial Networks[J]. Computer Engineering, 2021, 47(3): 249-255.

http://www.ecice06.com/CN/Y2021/V47/I3/249

图/表 9

20210322174143

20210322174147

20210322174150

20210322174154

20210322174157

20210322174201

20210322174204

20210322174208

20210322174211

参考文献

[1] KEYS R.Cubic convolution interpolation for digital image processing[J].IEEE Transactions on Acoustics,Speech,and Signal Processing,1981,29(6):1153-1160.
[2] ZHANG Lei,WU Xiaolin.An edge-guided image interpolation algorithm via directional filtering and data fusion[J].IEEE Transactions on Image Processing,2006,15(8):2226-2238.
[3] FATTAL R.Image upsampling via imposed edge statistics[J].ACM Transactions on Graphics,2007,26(3):95-103.
[4] JUNG S W,KIM T H,KO S J.A novel multiple image deblurring technique using fuzzy projection onto convex sets[J].IEEE Signal Processing Letters,2009,16(3):192-195.
[5] NAYAK R,HARSHAVARDHAN S,PATRA D.Morphology based iterative back-projection for super-resolution reconstruction of image[C]//Proceedings of the 2nd International Conference on Emerging Technology Trends in Electronics,Communication and Networking.Washington D.C.,USA:IEEE Press,2014:1-6.
[6] SUN Dong,GAO Qingwei,LU Yixiang,et al.A novel image denoising algorithm using linear Bayesian MAP estimation based on sparse representation[J].Signal Processing,2014,100(7):132-145.
[7] DONG C,LOY C C,HE K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[8] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[9] LEDIG C,THEIS L,HUSZAR F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4681-4690.
[10] LAI W S,HUANG J B,AHUJA N,et al.Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:624-632.
[11] TAI Ying,YANG Jian,LIU Xiaoming.Image super-resolution via deep recursive residual network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:3147-3155.
[12] JOHNSON J,ALAHI A,LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:694-711.
[13] SAJJADI M S M,SCHOLKOPF B,HIRSCH M.EnhanceNet:single image super-resolution through automated texture synthesis[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:4491-4500.
[14] GOODFELLOW I,ABADIE J P,MIRZA M,et al.Generative adversarial nets[EB/OL].[2019-11-25].https://www.researchgate.net/publication/319770355_Generative_Adversarial_Nets.
[15] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:136-144.
[16] WANG Xintao,YU Ke,WU Shixiang,et al.ESRGAN:enhanced super-resolution generative adversarial net-works[C]//Proceedings of 2019 European Conference on Computer Vision.Berlin,Germany:Springer,2019:63-79.
[17] RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]//Proceedings of 2013 Annual Allerton Conference on Communication.Washington D.C.,USA:IEEE Press,2013:917-924.
[18] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Identity mappings in deep residual networks[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:630-645.
[19] SRIVASTAVA R K,GREFF K,SCHMIDHUBER J.Training very deep networks[EB/OL].[2019-11-25].https://www.researchgate.net/publication/280329704_Training_Very_Deep_Networks.
[20] HUANG Gao,SUN Yu,LIU Zhuang,et al.Deep networks with stochastic depth[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:646-661.
[21] HUANG G,LIU Z,LAURENS V D M,et al.Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4700-4708.
[22] JEGOU S,DROZDZAL M,VAZQUEZ D,et al.The one hundred layers Tiramisu:fully convolutional DenseNets for semantic segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:11-19.
[23] PARK S,JEONG Y,KIM H S.Multi-resolution DenseNet based acoustic models for reverberant speech recognition[J].Phonetics and Speech Sciences,2018:10(1):33-38.
[24] ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,38(2):295-307.
[25] GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved training of Wasserstein GANs[EB/OL].[2019-11-25].https://www.researchgate.net/publication/315765101_Imp roved_Training_of_Wasserstein_GANs.
[26] NAH S,KIM T H,LEE K M.Deep multi-scale convolu-tional neural network for dynamic scene deblurring[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:3883-3891.
[27] ZHANG Yulun,TIAN Yapeng,KONG Yu,et al.Residual dense network for image super-resolution[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2472-2481.
[28] ZHANG Yulun,LI Kunpeng,LI Kai,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of ECCV'18.Berlin,Germany:Springer,2018:294-310.
[29] SZEGEDY C,LIU W,JIA Y Q,et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-9.

选择文件类型/文献管理软件名称

选择包含的内容