计算机工程 ›› 2021, Vol. 47 ›› Issue (1): 230-238.doi: 10.19678/j.issn.1000-3428.0056287

• 图形图像处理 • 上一篇    下一篇

基于谱归一化条件生成对抗网络的图像修复算法

雷蕾1, 郭东恩1,2, 靳峰1   

  1. 1. 南阳理工学院 软件学院, 河南 南阳 473000;
    2. 重庆邮电大学 计算机科学与技术学院 计算智能重庆市重点实验室, 重庆 400065
  • 收稿日期:2019-10-14 修回日期:2019-12-31 发布日期:2020-01-15
  • 作者简介:雷蕾(1986-),女,硕士研究生,主研方向为图像处理、机器学习、大数据分析;郭东恩,副教授、博士;靳峰,讲师、博士。
  • 基金项目:
    国家自然科学基金(61671091);重庆市自然科学基金(cstc2017jcyjBX0037,cstc2017jcyjA0982);河南省高等学校重点科研项目(19A520030);重庆市教委一般项目(CYB19173);重庆邮电大学博士创新人才项目(BYJS201812)。

Image Inpainting Algorithm Based on Conditional Generative Adversarial Network with Spectral Normalization

LEI Lei1, GUO Dongen1,2, JIN Feng1   

  1. 1. School of Software, Nanyang Institute of Technology, Nanyang, Henan 473000, China;
    2. Chongqing Key Laboratory of Computational Intelligence, College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
  • Received:2019-10-14 Revised:2019-12-31 Published:2020-01-15

摘要: 基于生成对抗网络的图像修复算法在修复大尺寸缺失图像时,存在图像失真较多与判别网络性能不可控等问题,基于谱归一化条件生成对抗网络,提出一种新的图像修复算法。引入谱归一化来约束判别网络的判别性能,间接提高修复网络的修复能力,并根据控制判别网络性能对谱归一化进行理论分析。通过类别信息约束特征生成,保证修复图像的内容不变性,引入扩展卷积算子对待修复图像进行像素级操作,解决修复图像缺乏局部一致性的问题。在此基础上,运用PSNR、SSIM等图像评价方法及分片Wasserstein距离、Inception分数、流形距离度量、GAN-train和GAN-test等流形结构相似度评价指标对修复图像进行综合评价。实验结果表明,与CE、GL等算法相比,该算法获得的修复图像在主观感受和客观评价指标上均有明显提高。

关键词: 谱归一化, 条件生成对抗网络, 图像修复, 判别性能, 图像评价

Abstract: To solve the problem of large image distortion and uncontrollable discriminative network performance in image inpainting based on Generative Adversarial Network(GAN),this paper proposes a new image inpainting algorithm based on conditional generative adversarial network with spectral normalization. Spectral normalization is introduced to constrain the discriminative performance of discriminative network,and thus bring an improvement to the inpainting network performance followed by detailed theoretical analysis of spectral normalization in controlling the discriminative network performance.Category information is used to constrain feature generation to ensure that content of the inpainted image is close to that of the original image.The extended convolution operator is also introduced to perform pixel-level operation on the to-be-inpainted image to address the lack of local consistency in image inpainting. On this basis,PSNR,SSIM and other image evaluation methods,as well as slice Wasserstein distance,Inception score, manifold distance measurement,GAN-train,GAN-test and other manifold structure similarity evaluation indicators are used to comprehensively evaluate the inpainted image.Experimental results show that compared with CE,GL and other algorithms,the proposed algorithm can significantly improve the subjective and objective evaluation indicators of the inpainted images.

Key words: spectral normalization, conditional Generative Adversarial Network(GAN), image inpainting, discrimination performance, image evaluation

中图分类号: