基于注意力机制与特征融合的图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0057208

摘要/Abstract

摘要： 目前多数利用卷积神经网络进行图像超分辨率重建的方法忽视对自然图像固有属性的捕捉，并且仅在单一尺度下提取特征。针对该问题，提出一种基于注意力机制和多尺度特征融合的网络结构。利用注意力机制融合图像的非局部信息和二阶特征，提高网络的特征表达能力，同时使用不同尺度的卷积核提取图像的不同尺度信息，以保存多尺度完整的信息特征。实验结果表明，该方法重建图像的客观评价指标和视觉效果均优于Bicubic、SRCNN、SCN和LapSRN方法。

关键词: 超分辨率重建, 卷积神经网络, 非局部信息, 二阶特征, 注意力机制, 多尺度特征

Abstract: In super-resolution image reconstruction,most of the existing methods using Convolutional Neural Network(CNN) neglect the inherent attributes of natural images,and extract features only at a single scale.To address the problem,this paper proposes a network structure based on attention mechanism and multi-scale feature fusion.By using the attention mechanism,the non-local information and second-order features of the image are fused to improve the feature expression ability of the network.At the same time,different scales of convolutional kernels are used to extract different scales of information of the image,so as to preserve the complete information characteristics at different scales.Experimental results show that the reconstructed image by the proposed method outperforms Bicubic,SRCNN,SCN and LapSRN methods in terms of objective evaluation metrics and visual quality.

Key words: super-resolution reconstruction, Convolutional Neural Network(CNN), non-local information, second-order feature, attention mechanism, multi-scale feature

中图分类号:

TP391

王诗言, 曾茜, 周田, 吴华东. 基于注意力机制与特征融合的图像超分辨率重建[J]. 计算机工程, 2021, 47(3): 269-275,283.

WANG Shiyan, ZENG Xi, ZHOU Tian, WU Huadong. Image Super-Resolution Reconstruction Based on Attention Mechanism and Feature Fusion[J]. Computer Engineering, 2021, 47(3): 269-275,283.

https://www.ecice06.com/CN/Y2021/V47/I3/269

图/表 13

20210322174340

20210322174343

20210322174345

20210322174348

20210322174351

20210322174403

20210322174409

20210322174413

20210322174416

20210322174421

20210322174424

20210322174428

20210322174431

参考文献

[1] LIAN Yiya,WU Xiaojun.Research on image super-resolution reconstruction of super deep convolutional neural network[J].Computer Engineering,2019,45(1):217-220.(in Chinese)连逸亚,吴小俊.超深卷积神经网络的图像超分辨率重建研究[J].计算机工程,2019,45(1):217-220.
[2] LI Sumei,LEI Guoqing,FAN Ru.Depth map super-resolution reconstruction based on convolutional neural networks[J].Acta Optica Sinica,2017,37(12):124-132.(in Chinese)李素梅,雷国庆,范如.基于卷积神经网络的深度图超分辨率重建[J].光学学报,2017,37(12):124-132.
[3] DONG C,LOY C,HE K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[4] DONG C,LOY C,HE K M,et al.Accelerating the super-resolution convolutional neural network[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2016:391-407.
[5] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[6] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[7] KIM J,LEE J K,LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1637-1645.
[8] SHI W,CABALLERO J,HUSZÁR F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1874-1883.
[9] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Procee-dings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:1132-1140.
[10] BUADES A,COLL B,MOREL J M.A non-local algorithm for image denoising[C]//Proceedings of CVPR'05.Washington D.C.,USA:IEEE Press,2005:1-3.
[11] DONG Weisheng,ZHANG Lei,SHI Guangming,et al.Nonlocally centralized sparse representation for image restoration[J].IEEE Transactions on Image Processing,2013,22(4):1620-1630.
[12] LI Peihua,XIE Jiantao,WANG Qinglong,et al.Is second-order information helpful for large-scale visual recognition?[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:2089-2097.
[13] WANG X L,GIRSHICK R,GUPTA A,et al.Non-local neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:7794-7803.
[14] HU J,SHEN L,SAMUEL A.Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:7132-7141.
[15] WOO S,PARK J C,LEE J Y,et al.CBAM:convolutional block attention module[C]//Proceedings of ECCV'18.Berlin,Germany:Springer,2018:3-19.
[16] ZHANG Yulun,LI Kunpeng,WANG Lichen,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of ECCV'18.Berlin,Germany:Springer,2018:294-310.
[17] DAI Tao,CAI Jianrui,ZHANG Yongbing,et al.Second-order attention network for single image super-resolution[C]//Proceedings of CVPR'19.Washington D.C.,USA:IEEE Press,2019:11065-11074.
[18] HU Xuecai,MU Haoyuan,ZHANG Xiangyu,et al.Meta-SR:a magnification-arbitrary network for super-resolution[C]//Proceedings of CVPR'19.Washington D.C.,USA:IEEE Press,2019:1575-1584.
[19] MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of ICCV'01.Washington D.C.,USA:IEEE Press,2001:1-5.
[20] BEVILACQUA M.Low complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of BMVC'12.Berlin,Germany:2012:1-10.
[21] ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of Interna-tional Conference on Curves and Surface.Berlin,Germany:Springer,2010:711-730.
[22] KINGMA D,BA J.Adam:a method for stochastic optimiza-tion[EB/OL].(2014-12-22)[2019-12-20].https://arxiv.org/pdf/1412.6980.pdf.
[23] WANG Zhaowen,LIU Ding,YANG Jianchao,et al.Deep networks for image super-resolution with sparse prior[C]//Proceedings of ICCV'15.Washington D.C.,USA:IEEE Press,2015:1-7.
[24] LAI W S,HUANG J B,AHUJA N,et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of CVPR'17.Washington D.C.,USA:IEEE Press,2017:5835-5843.

选择文件类型/文献管理软件名称

选择包含的内容