Super-Resolution Image Reconstruction Method Under Joint Constraints of External and Internal Gradient

doi:10.19678/j.issn.1000-3428.0059639

Abstract

Abstract: Image acquisition is affected by sensors, optical systems, and atmospheric disturbances and so on, which often leads to the degradation of image resolution.This paper presents a novel super-resolution image reconstruction method under the joint constraints of external and internal gradient prior.Based on the framework of Maximum a Posterior Probability(MAP), the proposed method integrates the external gradient priors with the joint constraint prior of brightness and the internal gradient in norm.By combining the complementary advantages of external and internal gradient prior, the quality of reconstructed super-resolution images is improved by using the high-resolution gradient information.Results of simulation and real experiments show that compared with ASAR, DRRN and other methods, the proposed method can greatly enhance the high-frequency details and edge information, and reduce the possible noise and artifact effects in reconstructed images.

Key words: super-resolution reconstruction, gradient prior, joint constraints, deep learning, Maximum A Posterior (MAP) probability

摘要： 图像在获取过程中受传感器、光学系统、大气扰动等多因素影响，导致图像分辨率下降。提出一种结合内外部梯度先验的图像超分辨率重建方法。基于最大后验估计理论框架，将深度学习的外部梯度先验和范数的内部梯度、亮度联合约束先验相结合，通过联合利用外部梯度先验和内部梯度先验的互补优势，借助高分辨率的梯度信息提高重建的超分辨率图像质量。实验结果表明，与ASAR、DRRN等方法相比，该方法能够有效增强图像的高频细节和边缘信息，减少重建图像中可能存在的噪声与伪影效果。

关键词: 超分辨率重建, 梯度先验, 联合约束, 深度学习, 最大后验概率

CLC Number:

TP391

JIANG Hongtao, SUN Jing, XIE Cheng, LAI Shaochuan, SHEN Huanfeng. Super-Resolution Image Reconstruction Method Under Joint Constraints of External and Internal Gradient[J]. Computer Engineering, 2022, 48(1): 220-227,235.

姜红涛, 孙京, 谢成, 赖少川, 沈焕锋. 内外部梯度联合约束的图像超分辨率重建方法[J]. 计算机工程, 2022, 48(1): 220-227,235.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0059639

http://www.ecice06.com/EN/Y2022/V48/I1/220

Figures/Tables 9

References

[1] FAN C, WU C, LI G, et al.Projections onto convex sets super-resolution reconstruction based on point spread function estimation of low-resolution remote sensing images[J].Sensors, 2017, 17(2):362.
[2] ISAAC J S, KULKARNI R.Super resolution techniques for medical image processing[C]//Proceedings of 2015 International Conference on Technologies for Sustainable Development.Washing D.C., USA:IEEE Press, 2015:1-6.
[3] POULIOT D, LATIFOVIC R, PASHER J, et al.Landsat super-resolution enhancement using convolution neural networks and sentinel-2 for training[J].Remote Sensing, 2018, 10(3):394.
[4] ZHANG L, ZHANG H, SHEN H, et al.A super-resolution reconstruction algorithm for surveillance images[J].Signal Processing, 2010, 90(3):848-859.
[5] 苏秉华, 金伟其, 牛丽红, 等.超分辨率图像复原及其进展[J].光学技术, 2001, 27(1):6-9. SU B H, JIN W Q, NIU L H, et al.Super-resolution image restoration and progress[J].Optical Technology, 2001, 27(1):6-9.(in Chinese)
[6] NGUYEN K, FOOKES C, SRIDHARAN S, et al.Super-resolution for biometrics:a comprehensive survey[J].Pattern Recognition, 2018, 78:23-42.
[7] TSAI R Y.Multi-frame image restoration and registration[J].Visual and Image Processing, 1984, 1:317-339.
[8] MAISELI B J, ALLY N, GAO H.A noise-suppressing and edge-preserving multi-frame super-resolution image reconstruction method[J].Signal Processing Image Communication, 2015, 34:1-13.
[9] ZHANG H, ZHANG L, SHEN H.A super-resolution reconstruction algorithm for hyperspectral images[J].Signal Processing, 2012, 92(9):2082-2096.
[10] HE Y, YAP K H, CHEN L, et al.A nonlinear least square technique for simultaneous image registration and super-resolution[J].IEEE Transactions on Image Processing, 2007, 16(11):2830-2841.
[11] SHAH N R, ZAKHOR A.Resolution enhancement of color video sequences[J].IEEE Transactions on Image Processing, 1999, 8(6):879-885.
[12] IRANI M, PELEG S.Improving resolution by image registration[J].Graphical Models and Image Processing, 1991, 53(3):231-239.
[13] LIANG F, XU Y, ZHANG M, et al.A POCS algorithm based on text features for the reconstruction of document images at super-resolution[J].Symmetry, 2016, 8(10):102.
[14] SCHULTZ R R, STEVENSON R L.Extraction of high-resolution frames from video sequences[J].IEEE Transactions on TImage Processing, 1996, 5(6):996-1011.
[15] SHEN H, ZHANG L, HUANG B, et al.A MAP approach for joint motion estimation, segmentation, and super resolution[J].IEEE Transactions on Image processing, 2007, 16(2):479-490.
[16] YUE L, SHEN H, LI J, et al.Image super-resolution:the techniques, applications, and future[J].Signal Processing, 2016, 128:389-408.
[17] ZHANG L, WU X.An edge-guided image interpolation algorithm via directional filtering and data fusion[J].IEEE Transactions on Image Processing, 2006, 15(8):2226-2238.
[18] SUN J, XU Z, SHUM H Y.Image super-resolution using gradient profile prior[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2008:1-8.
[19] DONG C, LOY C C, HE K M, et al.Learning a deep convolutional network for image super-resolution[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2014:184-199.
[20] DONG C, LOY C C, HE K, et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[21] KIM J, LEE J K, LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1646-1654.
[22] 谢珍珠, 吴从中, 詹曙.边缘增强深层网络的图像超分辨率重建[J].中国图象图形学报, 2018, 23(1):114-122. XIE Z, WU C Z, ZHAN S.Image super-resolution reconstruction via deep network based on edge-enhancement[J].Journal of Image and Graphics, 2018, 23(1):114-122.(in Chinese)
[23] 张慧, 常莉红, 马旭, 等.一种基于曲波变换与引导滤波增强的图像融合方法[J].吉林大学学报(理学版), 2020, 58(1):113-119. ZHANG H, CHENG L H, MA X, et al.An Image Fusion Method Based on Curvelet Transform and Guided Filter Enhancement[J].Journal of Jilin University(Science Edition), 2020, 58(1):113-119.(in Chinese)
[24] KAPPELER A, YOO S, DAI Q, et al.Video super-resolution with convolutional neural networks[J].IEEE Transactions on Computational Imaging, 2016, 2(2):109-122.
[25] PROTTER M, ELAD M.Super resolution with probabilistic motion estimation[J].IEEE Transactions on Image Processing, 2009, 18(8):1899-1904.
[26] TAI Y, YANG J, LIU X.Image super-resolution via deep recursive residual network[C]//Proceedings of IEEE Conference on Computer Vision & Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:346-358.
[27] YANG J, WRIGHT J, HUANG T, et al.Image super-resolution as sparse representation of raw image patches[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2008:1-8.
[28] MARTIN D, FOWLKES C, TAL D, et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2001:579-588.
[29] PAN J, HU Z, SU Z, et al.L0-regularized intensity and gradient prior for deblurring text images and beyond[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2):342-355.
[30] XU L, JIA J, MATSUSHITA Y.Motion detail preserving optical flow estimation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9):1744-1757.
[31] BOYD S, PARIKH N, CHU E, et al.Distributed optimization and statistical learning via the alternating direction method of multipliers[J].Foundations and Trends in Machine Learning, 2011, 3(1):1-122.
[32] LI K, ZHU Y, YANG J, et al.Video super-resolution using an adaptive superpixel-guided auto-regressive model[J].Pattern Recognition, 2006, 51:59-71.
[33] 孙京, 袁强强, 李冀玮, 等.亮度-梯度联合约束的车牌图像超分辨率重建[J].中国图象图形学报, 2018, 23(6):802-813. SUN J, YUAN Q Q, LI J W, et al.License plate image super-resolution based on intensity-gradient prior combination[J].Journal of Image and Graphics, 2018, 23(6):802-813.(in Chinese)
[34] MDSP SR datasets[EB/OL].[2020-09-01].http://users.soe.ucsc.edu/~milanfar/software/sr-datasets.html.

Please choose a citation manager

Content to export