基于深度残差自适应注意力网络的图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0064243

摘要/Abstract

摘要： 针对现有图像超分辨率重建算法中常见的图像边缘模糊、卷积核尺寸选择单一、重建网络结构冗余等问题，提出一种基于深度残差自适应注意力网络的图像超分辨率重建算法。构建嵌套残差网络结构增加残差网络深度，在保证网络拟合性能的前提下提升网络整体性能。建立自适应注意力模块，使用空洞空间金字塔池化模块融合不同尺度的特征图，获得更多的有效特征，恢复图像纹理细节，同时基于选择性卷积核模块和像素注意力模块的并行结构自适应调整卷积核尺寸，并应用注意力机制提取图像高频特征，最终将提取特征通过重建模块实现超分辨率图像重建。在Set5、Set14、BSD100 3个测试数据集上的实验结果表明，与Bicubic、SRCNN、MemNet、DCSR等重建算法相比，该算法的峰值信噪比和结构相似性指标平均提升了0.57 dB和0.006 8，具有更高的超分辨率图像重建质量。

关键词: 卷积神经网络, 超分辨率重建, 注意力机制, 残差网络, 多尺度特征

Abstract: To solve problems typically encountered in existing image Super-Resolution（SR） reconstruction algorithms，such as blurred image edges，single selection of convolution kernel size，and redundant reconstruction network structure，this paper proposes an image SR reconstruction algorithm based on a Deep Residual Adaptive Attention Network（DRAAN）.Applying a Residual in Residual（RIR） network structure increases the depth of the residual network and improves the overall performance by ensuring the fitting performance of the network.In the DRAAN，an Adaptive Attention（AA） module is established，and an Atrous Spatial Pyramid Pooling（ASPP） module is used to fuse feature maps of different scales，obtain effective features，and restore image texture details.Additionally，based on the parallel structure of the Select Kernel（SK） and Pixel Attention（PA） modules，the size of the convolution kernel is adaptively adjusted，and the attention mechanism is applied to efficiently extract the high-frequency features of images.Finally，to achieve SR image reconstruction，the extracted features are reconstructed via module reconstruction.Test and simulation results on three benchmark datasets，Set5，Set14，and BSD100，show that compared with Bicubic，deep Convolutional Network for image Super-Resolution（SRCNN），persistent Memory Network（MemNet） for image restoration，Dilated Convolutions for single-image Super-Resolution（DCSR），and other reconstruction algorithms，the proposed algorithm yields 0.57 dB and 0.006 8 higher values of Peak Signal to Noise Ratio（PSNR） and Structural SIMilarity（SSIM），respectively，on average as well as higher a SR image reconstruction quality.

Key words: Convolutional Neural Network（CNN）, Super-Resolution（SR） reconstruction, attention mechanism, residual network, multiscale feature

中图分类号:

TP391.41

丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建[J]. 计算机工程, 2023, 49(5): 231-238.

DING Zixuan, YU Lei, ZHANG Juan, LI Xiang, WANG Xinyu. Image Super-Resolution Reconstruction Based on Depth Residual Adaptive Attention Network[J]. Computer Engineering, 2023, 49(5): 231-238.

https://www.ecice06.com/CN/Y2023/V49/I5/231

图/表 12

20230515190610

20230515190613

20230515190616

20230515190619

20230515190622

20230515190625

20230515190629

20230515190632

20230515190636

20230515190639

20230515190642

20230515190646

参考文献

[1] YANG J C,WRIGHT J,HUANG T,et al.Image super-resolution as sparse representation of raw image patches[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2008:1-8.
[2] TIMOFTE R,DE SMET V,VAN GOOL L.A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Proceedings of ACCV'14.Berlin,Germany:Springer,2014:111-126.
[3] YANG J C,WRIGHT J,HUANG T S,et al.Image super-resolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[4] SCHULTER S,LEISTNER C,BISCHOF H.Fast and accurate image upscaling with super-resolution forests[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:3791-3799.
[5] 汤佳欣,陈阳,周孟莹,等.深度学习方法在兴趣点推荐中的应用研究综述[J].计算机工程,2022,48(1):12-23,42. TANG J X,CHEN Y,ZHOU M Y,et al.A survey of studies on deep learning applications in POI recommendation[J].Computer Engineering,2022,48(1):12-23,42.(in Chinese)
[6] DONG C,LOY C C,HE K M,et al.Learning a deep convolutional network for image super-resolution[C]//Proceedings of ECCV'14.Berlin,Germany:Springer,2014:184-199.
[7] DONG C,LOY C C,TANG X O.Accelerating the super-resolution convolutional neural network[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:391-407.
[8] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[9] TAI Y,YANG J,LIU X M,et al.MemNet:a persistent memory network for image restoration[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:4549-4557.
[10] LAI W S,HUANG J B,AHUJA N,et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:5835-5843.
[11] ZHANG K,ZUO W M,ZHANG L.Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:3262-3271.
[12] ZHANG Z D,WANG X R,JUNG C.DCSR:dilated convolutions for single image super-resolution[J].IEEE Transactions on Image Processing,2019,28(4):1625-1635.
[13] TIAN C W.Lightweight image super-resolution with enhanced CNN[J].Knowledge-Based Systems,2020,205:106235.
[14] 黄伟,冯晶晶,黄遥.基于多通道极深卷积神经网络的图像超分辨率算法[J].计算机工程,2020,46(9):242-247,253. HUANG W,FENG J J,HUANG Y.Super-resolution algorithm for images based on multi-channel extremely deep convolutional neural network[J].Computer Engineering,2020,46(9):242-247,253.(in Chinese)
[15] ESMAEILZEHI A,AHMAD M O,SWAMY M N S.MuRNet:a deep recursive network for super resolution of bicubically interpolated images[J].Signal Processing:Image Communication,2021,94:116228.
[16] ZHANG Y L,LI K P,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of ECCV'18.Berlin,Germany:Springer,2018:294-310.
[17] 施荣华,金鑫,胡超.基于图注意力网络的方面级别文本情感分析[J].计算机工程,2022,48(2):34-39. SHI R H,JIN X,HU C.Aspect-level text emotion analysis based on graph attention network[J].Computer Engineering,2022,48(2):34-39.(in Chinese)
[18] 殷亚博,杨文忠,杨慧婷,等.基于卷积神经网络和KNN的短文本分类算法研究[J].计算机工程,2018,44(7):193-198. YIN Y B,YANG W Z,YANG H T,et al.Research on short text classification algorithm based on convolutional neural network and KNN[J].Computer Engineering,2018,44(7):193-198.(in Chinese)
[19] 张顺,龚怡宏,王进军.深度卷积神经网络的发展及其在计算机视觉领域的应用[J].计算机学报,2019,42(3):453-482. ZHANG S,GONG Y H,WANG J J.The development of deep convolution neural network and its applications on computer vision[J].Chinese Journal of Computers,2019,42(3):453-482.(in Chinese)
[20] WANG F,JIANG M Q,QIAN C,et al.Residual attention network for image classification[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:6450-6458.
[21] ZHAO H Y,KONG X T,HE J W,et al.Efficient image super-resolution using pixel attention[C]//Proceedings of ECCV'20.Berlin,Germany:Springer,2020:56-72.
[22] LI X,WANG W H,HU X L,et al.Selective kernel networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:510-519.
[23] TIMOFTE R,AGUSTSSON E,GOOL L V,et al.NTIRE 2017 challenge on single image super-resolution:methods and results[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:1110-1121.
[24] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of British Machine Vision Conference.Durham,UK:British Machine Vision Association,2012:1-8.
[25] ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[M].Berlin,Germany:Springer,2012.
[26] MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2001:416-423.

选择文件类型/文献管理软件名称

选择包含的内容