基于稀疏神经网络的图像超分辨率重建算法

doi:10.19678/j.issn.1000-3428.0062364

摘要/Abstract

摘要： 部分基于深度学习的图像超分辨率重建算法通过扩展网络层的深度来提高网络模型的整体特征表达能力。然而，一味过度地扩展网络的深度会造成网络模型过参数化和复杂化，并且冗余的网络参数会增加特征表达的不稳定性。在LTH剪枝算法基础上改变权重参数并使用均衡学习策略，提出一种适用于图像超分辨率重建任务的神经网络非结构化剪枝算法RLTH。在不改变网络结构和不增加计算复杂度的前提下，通过搜索原始网络模型的最优稀疏子网络排除冗余参数带来的影响，在有限的参数资源中捕获更细粒度和丰富的图像特征，进而提高网络模型的整体特征表达能力。基于Set5、Set14和BSD100测试集的实验结果表明，与原始网络模型和应用LTH剪枝算法相比，应用RLTH算法获得的重建图像PSNR和SSIM均得到提升，且具有更丰富的细节特征，整体和局部轮廓更清晰。

关键词: 单帧图像超分辨率重建, 神经网络, 非结构化剪枝, 深度学习, 稀疏网络

Abstract: Many deep learning-based image super-resolution reconstruction algorithms improve the overall feature expression ability of a network by extending the depth of the network.However, excessively extending the depth of the network causes the model to be over-parameterized and complicated.Furthermore, redundant parameters increase the instability of feature expression.To address this issue, based on the LTH pruning algorithm, the weight parameters are changed and the balanced learning strategy is used, this paper proposes a neural network unstructured pruning algorithm which is suitable for image super-resolution reconstruction tasks, called the RLTH pruning algorithm.Without changing the network structure and increasing the computational complexity, the overall feature expression ability of the network is improved by searching for an optimal yet sparse sub-network of the original network, which excludes the influence of redundant parameters and maximizes the ability of capturing fine-grained and richer features with limited parameters.The experimental results based on Set5, Set14 and BSD100 test sets show that, compared with the original network model and LTH pruning algorithm, the PSNR and SSIM of the reconstructed images obtained by RLTH algorithm are improved, and they have richer detail features and clearer overall and local contours.

Key words: Single Image Super-Resolution(SISR) reconstruction, neural network, unstructured pruning, deep learning, sparse network

中图分类号:

TP391

黎浩民, 李光平. 基于稀疏神经网络的图像超分辨率重建算法[J]. 计算机工程, 2022, 48(7): 247-253.

LI Haomin, LI Guangping. Image Super-Resolution Reconstruction Algorithm Based on Sparse Neural Network[J]. Computer Engineering, 2022, 48(7): 247-253.

https://www.ecice06.com/CN/Y2022/V48/I7/247

图/表 10

20221029175354

20221029175358

20221029175402

20221029175405

20221029175409

20221029175413

20221029175417

20221029175420

20221029175424

20221029175427

参考文献

[1] KIM J, LEE J K, LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1646-1654.
[2] LEDIG C, THEIS L, HUSZÁR F, et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:105-114.
[3] LIM B, SON S, KIM H.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:1132-1140.
[4] WANG X T, YU K, WU S X.ESRGAN:enhanced super-resolution generative adversarial networks[C]//Proceedings of 2018 European Conference on Computer Vision.Berlin, Germany:Springer, 2018:63-79.
[5] LIU Z, SUN M J, ZHOU T H, et al.Rethinking the value of network pruning[EB/OL].[2021-08-09].https://arxiv.org/abs/1810.05270.
[6] WEI D Y, WANG Z W.Multi-scale channel network based on filer pruning for image super-resolution[J].Optik, 2021, 236:1-10.
[7] CHEN S, HUANG K, LI B W, et al.Adaptive hybrid composition based super-resolution network via fine-grained channel pruning[C]//Proceedings of 2020 European Conference on Computer Vision.Berlin, Germany:Springer, 2020:119-135.
[8] FRANKLE J, CARBIN M.The lottery ticket hypothesis:finding sparse, trainable neural networks[EB/OL].[2021-08-09].https://arxiv.org/abs/1803.03635.
[9] DONG C, LOY C C, HE K M, et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[10] GUO Y, CHEN J, WANG J D, et al.Closed-loop matters:dual regression networks for single image super-resolution[C]//Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:5406-5415.
[11] LIU J, ZHANG W J, TANG Y T, et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:2356-2365.
[12] YANG F Z, YANG H, FU J L, et al.Learning texture transformer network for image super-resolution[C]//Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:5790-5799.
[13] HAN S, LIU X Y, MAO H Z, et al.EIE:efficient inference engine on compressed deep neural network[J].ACM SIGARCH Computer Architecture News, 2016, 44(3):243-254.
[14] HAN S, POOL J, TRAN J, et al.Learning both weights and connections for efficient neural networks[EB/OL].[2021-08-09].https://arxiv.org/pdf/1506.02626.pdf.
[15] LOUIZOS C, WELLING M, KINGMA D P.Learning sparse neural networks through L0 regularization[EB/OL].[2021-08-09].https://arxiv.org/abs/1712.01312.
[16] HAN S, MAO H Z, DALLY W J.Deep compression:compressing deep neural network with pruning, trained quantization and huffman coding[EB/OL].[2021-08-09].https://arxiv.org/pdf/1510.00149.pdf.
[17] TARTAGLIONE E, LEPSØY S, FIANDROTTI A, et al.Learning sparse neural networks via sensitivity-driven regularization[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.Cambridge, USA:MIT Press, 2018:3882-3892.
[18] MAO H Z, HAN S, POOL J, et al.Exploring the regularity of sparse structure in convolutional neural networks[EB/OL].[2021-08-09].https://arxiv.org/pdf/1705.08922.pdf.
[19] GUO Y W, YAO A B, CHEN Y R.Dynamic network surgery for efficient DNNs[EB/OL].[2021-08-09].https://arxiv.org/pdf/1608.04493.pdf.
[20] LUGMAYR A, DANELLJAN M, TIMOFTE R.NTIRE 2020 challenge on real-world image super-resolution:methods and results[C]//Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:2058-2076.
[21] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of British Machine Vision Conference.Berlin, Germany:Springer, 2012:135.
[22] ZEYDE R, ELAD M, PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of International Conference on Curves and Surfaces.Berlin, Germany:Springer, 2012:711-730.
[23] MARTIN D, FOWLKES C, TAL D, et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of 2001 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2001:416-423.
[24] HE K M, ZHANG X Y, REN S Q, et al.Delving deep into rectifiers:surpassing human-level performance on ImageNet classification[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:1026-1034.
[25] KINGMA D, BA J.Adam:a method for stochastic optimization[EB/OL].[2021-08-09].https://arxiv.org/pdf/1412.6980.pdf.

选择文件类型/文献管理软件名称

选择包含的内容