Super-Resolution Image Reconstruction Algorithm Based on Multi-Scale Back Projection

doi:10.19678/j.issn.1000-3428.0055551

Abstract

Abstract: In order to solve the problem that existing mainstream super-resolution image reconstruction algorithms fail to fully utilize the detailed information in Low-Resolution(LR) images,this paper proposes a super-resolution image reconstruction algorithm based on multi-scale back projection.The algorithm uses multiple convolutional kernels of different scales to extract feature information of different dimensions from the shallow feature extraction layer.Then the extracted feature information is input into the back projection module,and the upsampling and downsampling methods are used alternatively to optimize the projection error of High-Resolution(HR) and LR images.Also,the idea of residual learning is used to connect the features extracted in the upsampling and downsampling stages in a cascade manner,so as to improve the image reconstruction effect.Experimental results on the Set5,Set14 and Urban100 datasets show that the proposed algorithm improves the Peak Signal-to-Noise Ratio(PSNR) and Structural Similarity(SSIM) compared with the five mainstream algorithms such as Bicubic,SRCNN,ESPCN,VDSR and LapSRN.

Key words: super-resolution image reconstruction algorithm, multi-scale, back projection, iterative upsampling and downsampling, deep learning

摘要： 为解决当前主流图像超分辨率重建算法对低分辨率图像中细节信息利用不够充分的问题，提出一种基于多尺度反向投影的图像超分辨率重建算法。使用多个不同尺度的卷积核从浅层特征提取层中提取出不同维度的特征信息，输入到反向投影模块后，交替使用升采样和降采样来优化高分辨率和低分辨率图像的投影误差，同时运用残差学习的思想将升采样和降采样阶段提取到的特征使用级联的方式进行连接，从而提升图像的重建效果。实验结果表明，在Set5、Set14和Urban100数据集上，与Bicubic、SRCNN、ESPCN、VDSR和LapSRN 5种主流算法相比，该算法的峰值信噪比和结构相似性均有所提高。

关键词: 图像超分辨率重建算法, 多尺度, 反向投影, 迭代式升采样和降采样, 深度学习

CLC Number:

TP391

XIONG Yahui, CHEN Dongfang, WANG Xiaofeng. Super-Resolution Image Reconstruction Algorithm Based on Multi-Scale Back Projection[J]. Computer Engineering, 2020, 46(7): 251-259.

熊亚辉, 陈东方, 王晓峰. 基于多尺度反向投影的图像超分辨率重建算法[J]. 计算机工程, 2020, 46(7): 251-259.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0055551

http://www.ecice06.com/EN/Y2020/V46/I7/251

References

[1] BATZ M,EICHENSEER A,SEILER J,et al.Hybrid super-resolution combining example-based single-image and interpolation-based multi-image reconstruction approaches[C]//Proceedings of 2015 IEEE International Conference on Image Processing.Washington D.C.,USA:IEEE Press,2015:58-62.
[2] YAO Xunxiang,ZHANG Yunfeng,NING Yang,et al.Multi-scale feature image interpolation based on a rational fractal function[J].Journal of Image and Graphics,2016,21(4):482-489.(in Chinese)姚勋祥,张云峰,宁阳,等.多尺度有理分形的图像插值算法[J].中国图象图形学报,2016,21(4):482-489.
[3] ZHANG Kaibing,GAO Xinbo,TAO Dacheng,et al.Single image super-resolution with non-local means and steering kernel regression[J].IEEE Transactions on Image Processing,2012,21(11):4544-4556.
[4] TAO Zhiqiang,LI Hailin,ZHANG Hongbing.Iterative back projection super resolution reconstruction algorithm based on new edge directed interpolation[J].Computer Engineering,2016,42(6):255-260.(in Chinese)陶志强,李海林,张红兵.基于新边缘指导插值的迭代反投影超分辨率重建算法[J].计算机工程,2016,42(6):255-260.
[5] TIMOFTE R,DE V,GOOL L V.Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of 2013 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2013:1920-1927.
[6] RADU T,VINCENT D S,LUC V G.A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Proceedings of the 12th Asian Conference on Computer Vision.Berlin,Germany:Springer,2014:111-126.
[7] PELEG T,ELAD M.A statistical prediction model based on sparse representations for single image super-resolution[J].IEEE Transactions on Image Processing,2014,23(6):2569-2582.
[8] YANG J C,WRIGHT J,HUANG T S,et al.Image super-resolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[9] ZHANG Wanxu,SHI Jianxiong,CHEN Xiaoxuan,et al.Image super-resolution reconstruction based on sparse representation and guided filtering[J].Computer Engineering,2018,44(9):212-217.(in Chinese)张万绪,史剑雄,陈晓璇,等.基于稀疏表示与引导滤波的图像超分辨率重建[J].计算机工程,2018,44(9):212-217.
[10] YUAN Kunpeng,XI Zhihong.Image super resolution based on depth jumping cascade[J].Acta Optica Sinica,2019,39(7):243-252.(in Chinese)袁昆鹏,席志红.基于深度跳跃级联的图像超分辨率重建[J].光学学报,2019,39(7):243-252.
[11] HARIS M,SHAKHNAROVICH G,UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:1664-1673.
[12] WU Lei,LÜ Guoqiang,XUE Zhitian,et al.Super-resolution reconstruction of images based on multi-scale recursive network[J].Acta Optica Sinica,2019,39(6):90-97.(in Chinese)吴磊,吕国强,薛治天,等.基于多尺度递归网络的图像超分辨率重建[J].光学学报,2019,39(6):90-97.
[13] DONG C,LOY C C,HE K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[14] SHI W Z,CABALLERO J,HUSZAR F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1874-1883.
[15] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[16] LAI W S,HUANG J B,AHUJA N,et al.Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:624-632.
[17] LAI Rui,YANG Yintang,ZHOU Huixin,et al.Total variation regularized iterative back-projection method for single frame image super resolution[C]//Proceedings of 2012 IEEE 11th International Conference on Signal Processing.Washington D.C.,USA:IEEE Press,2012:931-934.
[18] SZEGEDY C,LIU W,JIA Y Q,et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-9.
[19] TIMOFTE R,AGUSTSSON E,GOOL L V,et al.Ntire 2017 challenge on single image super-resolution methods and results[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:114-125.
[20] ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of International Conference on Curves and Surfaces.Berlin,Germany:Springer,2012:711-730.
[21] GLOROT X,BORDES A,BENGIO Y.Deep sparse rectifier neural networks[J].Journal of Machine Learning Research,2011,15:315-323.
[22] HUANG Yihui,FENG Qianjin.Segmentation of brain tumor on magnetic resonance images using 3D full-convolutional densely connected convolutional net-works[J].Journal of Southern Medical University,2018,38(6):661-668.

Please choose a citation manager

Content to export