基于改进SRGAN模型的人脸图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0064174

摘要/Abstract

摘要： 传统生成对抗网络模型重建人脸图像时出现过多失真，难以在减少失真的情况下有效提高人脸图像真实感。针对该问题，在生成对抗网络SRGAN模型的基础上，提出一种改进的人脸图像超分辨率重建方法。为提高重建像素点与周围像素点的相关性，将双注意力机制模块嵌入到SRGAN模型的生成器和判别器中，在空间域和通道域中获取更精准的特征依赖关系。同时应用自适应激活函数ACON取代原SRGAN网络中的激活函数，通过动态学习ACON激活函数参数为每个神经元设计不同激活形式，从而提高网络特征表达能力。使用改进SRGAN的人脸图像超分辨率重建算法在CelebA测试集上进行重建实验，结果表明：该算法较原算法PSNR值提高0.675 dB，SSIM值提高0.016，LPIPS值优化0.036，有效减少了重建人脸图像中眼睛等重点部位的失真情况；与其他非生成对抗网络的主流算法相比，LPIPS值最低优化0.107，最高优化0.205，有效提高了重建人脸图像的真实感。

关键词: 超分辨率重建, 生成对抗网络, 注意力机制, 自适应激活函数, 特征提取

Abstract: Because the traditional generation countermeasure network model exhibits excessive distortion when reconstructing the face image, it is difficult to effectively improve the realism of the face image.Under the condition of reducing the distortion, an improved super-resolution reconstruction method of face images based on SRGAN model is proposed.To increase the correlation between the reconstructed and surrounding pixels, the Dual Attention(DA) mechanism module is embedded in the generator and discriminator of the SRGAN model to obtain more accurate feature dependency in the spatial and channel domains.An adaptive activation function, termed ACON, is used to replace the original activation function in an SRGAN network, and different activation forms are designed for each neuron by dynamically learning the parameters of the ACON activation function, to improve the ability of network feature expression.The reconstruction experiment is performed on the CelebA using the improved SRGAN face image reconstruction algorithm.The results show that compared with the original algorithm, the PSNR value increases by 0.675 dB, the SSIM value increases by 0.016, and the LPIPS value is optimized by 0.036, effectively reducing the distortion of key parts, such as the eyes, in the reconstructed face image.Compared with other mainstream algorithms that do not generate countermeasure networks, the LPIPS value is optimized by 0.107 at the lowest and 0.205 at the highest, effectively improving the realism of the reconstructed face image.

Key words: super-resolution reconstruction, generative adversarial network, attention mechanism, adaptive activation function, feature extraction

中图分类号:

TP391

李培育, 张雅丽. 基于改进SRGAN模型的人脸图像超分辨率重建[J]. 计算机工程, 2023, 49(4): 199-205.

LI Peiyu, ZHANG Yali. Face Image Super-Resolution Reconstruction Based on Improved SRGAN Model[J]. Computer Engineering, 2023, 49(4): 199-205.

http://www.ecice06.com/CN/Y2023/V49/I4/199

图/表 14

20230417185747

20230417185750

20230417185753

20230417185756

20230417185800

20230417185803

20230417185806

20230417185810

20230417185813

20230417185817

20230417185820

20230417185823

20230417185826

20230417185829

参考文献

[1] KIM J, LEE J K, LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1637-1645.
[2] SHI W, CABALLERO J, HUSZÁR F, et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1874-1883.
[3] DONG C, LOY C C, TANG X.Accelerating the super-resolution convolutional neural network[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:391-407.
[4] KIM J, LEE J K, LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1646-1654.
[5] TONG T, LI G, LIU X, et al.Image super-resolution using dense skip connections[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:4799-4807.
[6] ZHANG Y, LI K, LI K, et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:286-301.
[7] 刘锡泽, 范红, 海涵, 等.基于密集反馈网络的单幅图像超分辨率重建[J].计算机工程, 2021, 47(11):254-261. LIU X Z, FAN H, HAI H, et al.Super-resolution reconstruction of a single image based on dense feedback network[J].Computer Engineering, 2021, 47(11):254-261.(in Chinese)
[8] 程德强, 郭昕, 陈亮亮, 等.多通道递归残差网络的图像超分辨率重建[J].中国图象图形学报, 2021, 26(3):605-618. CHENG D Q, GUO X, CHEN L L, et al.Image super-resolution reconstruction from multi-channel recursive residual network[J].Journal of Image and Graphics, 2021, 26(3):605-618.(in Chinese)
[9] SHA F, ZANDAVI S M, CHUNG Y Y.Fast deep parallel residual network for accurate super resolution image processing[J].Expert Systems with Applications, 2019, 128:157-168.
[10] SHANG T, DAI Q, ZHU S, et al.Perceptual extreme super-resolution network with receptive field block[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2020:440-441.
[11] XIAO M, ZHENG S, LIU C, et al.Invertible image rescaling[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:126-144.
[12] LUO X, XIE Y, ZHANG Y, et al.Latticenet:towards lightweight image super-resolution with lattice block[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:272-289.
[13] 胡德敏, 王揆豪, 林静.渐进式生成对抗网络的人脸超分辨率重建[J].小型微型计算机系统, 2021, 42(9):1955-1961. HU D M, WANG K H, LIN J.Progressive GAN for face image super-resolution[J].Journal of Chinese Computer Systems, 2021, 42(9):1955-1961.(in Chinese)
[14] 孙超文, 陈晓.基于多尺度特征融合反投影网络的图像超分辨率重建[J].自动化学报, 2021, 47(7):1689-1700. SUN C W, CHEN X.Multiscale feature fusion back-projection network for image super-resolution[J].Acta Automatica Sinica, 2021, 47(7):1689-1700.(in Chinese)
[15] LEDIG C, THEIS L, HUSZÁR F, et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:4681-4690.
[16] LIM B, SON S, KIM H, et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2017:136-144.
[17] RATLIFF L J, BURDEN S A, SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]//Proceedings of the 51st Annual Allerton Conference on Communication, Control, and Computing.Washington D.C., USA:IEEE Press, 2013:917-924.
[18] HU J, SHEN L, SUN G.Squeeze-and-excitation networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7132-7141.
[19] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[20] FU J, LIU J, TIAN H, et al.Dual attention network for scene segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:3146-3154.
[21] HAHNLOSER R H R, SARPESHKAR R, MAHOWALD M A, et al.Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit[J].Nature, 2000, 405(6789):947-951.
[22] CLEVERT D A, UNTERTHINER T, HOCHREITER S.Fast and accurate deep network learning by exponential linear units (ELUs)[EB/OL].[2021-10-20].https://arxiv.org/abs/1511.07289
[23] RAMACHANDRAN P, ZOPH B, LE Q V.Searching for activation functions[EB/OL].[2021-10-20].https://arxiv.org/pdf/1710.05941.pdf.
[24] MA N, ZHANG X, LIU M, et al.ACtivate Or Not:learning customized ctivation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2021:8032-8042.
[25] LIU Z, LUO P, WANG X, et al.Deep learning face attributes in the wild[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:3730-3738.
[26] ZHANG R, ISOLA P, EFROS A A, et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:586-595.

选择文件类型/文献管理软件名称

选择包含的内容