基于特征图注意力机制的图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0056618

摘要/Abstract

摘要： 图像超分辨率重建中的高频分量通常包含较多轮廓、纹理等细节信息，为更好地处理特征图中的高频分量与低频分量，实现自适应调整信道特征，提出一种基于特征图注意力机制的图像超分辨重建网络模型。利用特征提取块提取原始低分辨率图像中的特征信息，基于多个结合特征图注意力机制的信息提取块，通过特征信道之间的相互依赖性自适应调整信道特征，以恢复更多细节信息。在此基础上利用重建模块重建出不同尺度的高分辨率图像。在Set5数据集上的实验结果表明，与基于双三次插值的重建模型相比，该模型能够有效提升图像的视觉效果，且峰值信噪比与结构相似度分别提高了3.92 dB和0.056。

关键词: 超分辨率重建, 特征图注意力机制, 自适应调整, 残差信息, 高分辨率图像

Abstract: High-frequency components in image Super-Resolution(SR) reconstruction usually include more details such as contour and texture.In order to deal with the high-frequency components and low-frequency components in feature map better and adjust the channel features adaptively,this paper proposes an image SR reconstruction network model based on the attention mechanism.The model uses the feature extraction module to extract the feature information from the original Low-Resolution(LR) image.Then multiple information extraction modules using the attention mechanism of the feature map are used to adjust the channel features adaptively through the interdependence between the feature channels,so as to recover more detailed information.On this basis,the reconstruction module is used to reconstruct High-Resolution(HR) images of different scales.The experimental results on the Set5 dataset show that compared with the reconstruction model based on Bicubic interpolation,this model can effectively improve the visual effect of the image,and its Peak Signal-to-Noise Ratio(PSNR) and Structural Similarity(SSIM) are improved by 3.92 dB and 0.056 respectively.

Key words: Super-Resolution(SR) reconstruction, attention mechanism of feature map, adaptive adjustment, residual information, High-Resolution(HR) image

中图分类号:

TP391.4

鲁甜, 刘蓉, 刘明, 冯杨. 基于特征图注意力机制的图像超分辨率重建[J]. 计算机工程, 2021, 47(3): 261-268.

LU Tian, LIU Rong, LIU Ming, FENG Yang. Image Super-Resolution Reconstruction Based on Attention Mechanism of Feature Map[J]. Computer Engineering, 2021, 47(3): 261-268.

http://www.ecice06.com/CN/Y2021/V47/I3/261

图/表 11

20210322174300

20210322174303

20210322174306

20210322174309

20210322174312

20210322174315

20210322174318

20210322174321

20210322174325

20210322174328

20210322174330

参考文献

[1] DONG C,LOY C C,HE K M,et al.Learning a deep convolutional network for image super-resolution[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2014:184-199.
[2] ZHANG Kai,ZUO Wangmeng,ZHANG Lei.Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:3262-3271.
[3] CHEN Xiaofan,SHEN Haijie,BIAN Qian,et al.Face image super-resolution with an attention mechanism[J].Journal of Xidian University(Natural Science),2019,46(3):148-153.(in Chinese)陈晓范,申海杰,边倩,等.结合注意力机制的人脸超分辨率重建[J].西安电子科技大学学报(自然科学版),2019,46(3):148-153.
[4] KIM J,KWON L J,MU L K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[5] KIM J,KWON L J,MU L K.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1637-1645.
[6] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[7] HE Yufei,GAO Hongwei.Super-resolution reconstruction using multilayer connected convolutional neural network for single-frame image[J].Computer Applications and Software,2019,36(5):220-224,326.(in Chinese)贺瑜飞,高宏伟.基于多层连接卷积神经网络的单帧图像超分辨重建[J].计算机应用与软件,2019,36(5):220-224,326.
[8] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:136-144.
[9] TAI Ying,YANG Jian,LIU Xiaoming.Image super-resolution via deep recursive residual network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:3147-3155.
[10] ZHANG H,GOODFELLOW I,METAXAS D,et al.Self-attention generative adversarial networks[EB/OL].[2019-10-02].https://arxiv.org/pdf/1805.08318.pdf.
[11] LIN Hong, REN Shuo, YANG Yi, et al. Unsupervised image-to-image translation with self-attention and relativistic discriminator adversarial networks[J/OL]. Acta Automatica Sinica:1-10[2019-09-04].https://kns.cnki.net/kcms/detail/11.2109.TP.20190530.1347.001.html.(in Chinese)林泓,任硕,杨益,等.融合自注意力机制和相对鉴别的无监督图像翻译[J/OL].自动化学报:1-10[2019-09-04].https://kns.cnki.net/kcms/detail/11.2109.TP.20190530.1347.001.html.
[12] ZHANG Yulun,LI Kunpeng,LI Kai,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2018:286-301.
[13] HU J,SHEN L,ALBANIE S.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(8):7132-7141.
[14] SHI W,CABALLERO J,HUSZAR F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2016:1874-1883.
[15] ZHENG Hui,WANG Xiumei,GAO Xinbo.Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:723-731.
[16] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Delving deep into rectifiers:surpassing human-level performance on imagenet classification[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1026-1034.
[17] KINGMA D P,BA J.Adam:a method for stochastic optimization[EB/OL].[2019-10-02].http://de.arxiv.org/pdf/1412.6980.
[18] TIMOFTE R,AGUSTSSON E,VAN GOOL L,et al.Ntire 2017 challenge on single image super-resolution:methods and results[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:114-125.
[19] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[EB/OL].[2019-10-02].https://www.ixueshu.com/document/7e12bde83a725a9f318947a18e7f9386.html.
[20] MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2001:1286-1296.
[21] HUANG J B,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:5197-5206.
[22] MATSUI Y,ITO K,ARAMAKI Y,et al.Sketch-based manga retrieval using manga109 dataset[J].Multimedia Tools and Applications,2017,76(20):21811-21838.
[23] LEDIG C,THEIS L,HUSZAR F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4681-4690.
[24] YANG Wenming,ZHANG Xuechen,TIAN Yapeng,et al.Deep learning for single image super-resolution:a brief review[EB/OL].[2019-10-02].https://www.researchgate.net/profile/Yapeng_Tian/publication/326988108_Deep_Learning_for_Single_Image_Super-Resolution_A_Brief_Review/links/5b96dad392851c78c412dfb4/Deep-Learning-for-Single-Image-Super-Resolution-A-Brief-Review.pdf.

选择文件类型/文献管理软件名称

选择包含的内容