基于单幅图像学习的生成对抗网络模型

doi:10.19678/j.issn.1000-3428.0058711

摘要/Abstract

摘要： 将卷积神经网络引入生成对抗网络可提高所生成图像的质量，但网络的感受野较小且难以学习各个特征通道之间的重要关系。在SinGAN网络的基础上，提出一种能从单幅图像中学习的生成对抗网络模型。在SinGAN网络的生成器和鉴别器中引入Inception V2模块以增加网络宽度扩大感受野，采用多个卷积核提取图像特征并进行特征融合，利用SENet模块学习各个通道的重要程度以获取更好的图像表征。实验结果表明，与Bicubic和SinGAN模型相比，该模型峰值信噪比和结构相似性值更高，可有效提升图像生成质量。

关键词: 深度学习, 生成对抗网络, 生成模型, 注意力, 对抗学习

Abstract: By introducing the Convolutional Neural Network(CNN) into the Generative Adversarial Network(GAN), the quality of images generated by GAN can be improved. However, limited by the size of the receptive field, the network cannot fully learn the important relationships between the feature channels. To address the problem, a SinGAN-based GAN model which can learn from a single image is proposed. By introducing the Inception V2 module into the generator and discriminator of SinGAN, the network width is increased, and the receptive field is extended. Then multiple convolution cores are used to extract image features and fuse them. At the same time, the SENet module is used to learn the importance of each channel to obtain better image representation. Experimental results show that compared with Bicubic and SinGAN models, the proposed model has higher Peak Signal to Noise Ratio(PSNR) and Structural Similarity(SSIM), and significantly improves the quality of the generated images.

Key words: deep learning, Generative Adversarial Network(GAN), generative model, attention, adversarial learning

中图分类号:

TP391.41

朱海琦, 李宏, 李定文. 基于单幅图像学习的生成对抗网络模型[J]. 计算机工程, 2021, 47(8): 271-276,283.

ZHU Haiqi, LI Hong, LI Dingwen. Generative Adversarial Network Model Based on Single Image Learning[J]. Computer Engineering, 2021, 47(8): 271-276,283.

https://www.ecice06.com/CN/Y2021/V47/I8/271

图/表 13

20210819202307

20210819202311

20210819202316

20210819202320

20210819202326

20210819202330

20210819202334

20210819202339

20210819202343

20210819202347

20210819202352

20210819202357

20210819202401

参考文献

[1] GOODFELLOW I J, POUGET-ABADIE J, M-IRZA M, et al. Generative adversarial nets[EB/OL]. [2020-05-07]. https://www.researchgate.net/publication/319770355_Generative_Adversarial_Nets.
[2] MERTENS J F, ZAMIR S.The value of two-person zero-sum repeated games with lack of information on both sides[J]. International Journal of Game Theory, 1971, 1(1): 39-64.
[3] RATLIFF L J, BURDEN S A, SASTRY S S.Charact-erization and computation of local Nash equilibria in continuous games[C]//Proceedings of 2013 Annual Allerton Conference on Communication, Control, and Computing.Washington D.C., USA:IEEE Press, 2013:917-924.
[4] 卢健, 孙怡.结合稀疏编码模型的多帧图像超分辨率重建[J]. 计算机工程, 2015, 41(5): 264-269, 273. LU J, SUN Y.Multi-frame image super-resolution reconstruction combined with sparse coding model[J]. Computer Engineering, 2015, 41(5): 264-269, 273.(in Chinese).
[5] 熊亚辉, 陈东方, 王晓峰.基于多尺度反向投影的图像超分辨率重建算法[J]. 计算机工程, 2020, 46(7): 251-259. XIONG Y H, CHEN D F, WANG X F.Super resolution image reconstruction algorithm based on multi-scale back projection[J]. Computer Engineering, 2020, 46(7): 251-259.(in Chinese)
[6] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2223-2232.
[7] CHEN Y S, WANG Y C, KAO M H, et al. Deep photo enhancer:unpaired learning for image enhancement from photographs with gans[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:6306-6314.
[8] KIM T, CHA M, KIM H, et al. Learning to discover cross-domain relations with generative adversarial networks[C]//Proceedings of 2017 International Conference on Machine Learning.Washington D.C., USA:IEEE Press, 2017:1857-1865.
[9] CHEN Y, LAI Y K, LIU Y J.Cartoongan:generative adversarial networks for photo cartoonization[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:9465-9474.
[10] MIRZA M, OSINDERO S.Conditional generative adversarial nets[EB/OL]. [2020-05-07]. https://arxiv.org/pdf/1411. 1784.pdf.
[11] RADFORD A, METZ L, CHINTALA S.Unsupervised representation learning with deep convolutional genera-tive adversarial networks[EB/OL]. [2020-05-07]. https://arxiv.org/pdf/1511.06434.pdf.
[12] ARJOVSKY M, CHINTALA S, BOTTOU L.Wasserstein gan[EB/OL]. [2020-05-07]. https://arxiv.org/pdf/1701.07875.pdf.
[13] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of wasserstein gans[EB/OL]. [2020-05-07]. https://www.researchgate.net/publication/315765101_Improved_Training_of_Wasserstein_GANs.
[14] ZHANG H, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks[EB/OL]. [2020-05-07]. https://arxiv.org/pdf/1805.08318.pdf.
[15] BROCK A, DONAHUE J, SIMONYAN K.Large scale gan training for high fidelity natural image synthesis[EB/OL]. [2020-05-07]. https://arxiv.org/pdf/1809.11096.pdf.
[16] 刘其开, 姜代红, 李文吉.基于分段损失的生成对抗网络[J]. 计算机工程, 2019, 45(5): 155-160, 168. LIU Q K, JIANG D H, LI W J.Generative adversarial network based on piecewise loss[J]. Computer Engineering, 2019, 45(5): 155-160, 168.(in Chinese)
[17] SHAHAM T R, DEKEL T, MICHAELI T.Singan:learning a generative model from a single natural image[C]//Proceedings of 2019 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:4570-4580.
[18] IOFFE S, SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covaryate shift[EB/OL]. [2020-05-07]. https://arxiv.org/pdf/1502.03167.pdf.
[19] HU J, SHEN L, SUN G.Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7132-7141.
[20] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversaryal networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:1125-1134.
[21] ZHOU B L, LAPEDRIZA A, XIAO J X, et al. Learning deep features for scene recognition using places database[EB/OL]. [2020-05-07]. https://www.researchgate.net/publication/279839496_Learning_Deep_Features_for_Scene_Recognition_using_Places_Database.

选择文件类型/文献管理软件名称

选择包含的内容