基于动态自适应层叠网络的轻量化图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0064098

摘要/Abstract

摘要： 轻量化超分辨率网络对安防监控、实时人脸识别等领域具有重要意义。然而，现有超分辨率重建网络以牺牲内存和计算成本为代价提高重建效果，从而限制其在实际场景中的应用。提出基于动态自适应层叠网络的轻量化超分辨率重建网络。利用双路残差块中的深度可分离卷积提取低频特征，并引入像素注意力机制获取更丰富的细节特征，以减少参数量并增强网络的重建能力。将双路残差块中的一部分卷积参数作为动态卷积核的子卷积，并与动态自适应模块共享，利用可学习参数调节共享卷积的权重，增强网络的非线性映射关系，充分提取图像的纹理细节信息。实验结果表明，相比VDSR、CARN、PAN等网络，该网络重建得到的图像纹理更接近原始图像，其参数量仅为传统轻量化网络CARN的1/2，在放大因子为4的Set5数据集上峰值信噪比相比CARN提高0.16 dB。

关键词: 超分辨率重建, 轻量化网络, 动态自适应层叠网络, 动态卷积, 注意力机制, 深度学习

Abstract: Lightweight super-resolution network is of great importance in security monitoring, real-time face recognition, and other fields.However, the existing super-resolution reconstruction network improves the reconstruction effect at the expense of memory and computing cost, which limits its application in real-world scenarios.This study proposes a lightweight super-resolution reconstruction network based on the Dynamic Adaptive Cascade Network (DACN).The low frequency features are extracted by the depthwise separable convolution in the Dual Residual Block (DRB), and the pixel attention mechanism is introduced to obtain more detailed features to reduce the parameters quanitity and to enhance the network reconstruction ability.A part of the convolution parameters in the DRB is used as the sub-convolution of the dynamic convolution kernel and is shared with the Dynamic Adaptive Block(DAB).The weight of the shared convolution is adjusted by the learnable parameters to enhance the non-linear mapping relationship of the network and fully extract the texture details of the image.The experimental results show that compared with VDSR, CARN, PAN, and other networks, the reconstructed image texture of the proposed network is closer to the original image, and its parameter quanitity is only 1/2 of the traditional lightweight network CARN.The peak signal to noise ratio of Set5 dataset with an amplification factor of 4 is 0.16 dB higher than that of CARN.

Key words: super-resolution reconstruction, lightweight network, Dynamic Adaptive Cascade Network(DACN), dynamic convolution, attention mechanism, Deep Learning(DL)

中图分类号:

TP391

张法正, 杨娟, 汪荣贵, 薛丽霞. 基于动态自适应层叠网络的轻量化图像超分辨率重建[J]. 计算机工程, 2022, 48(12): 196-202.

ZHANG Fazheng, YANG Juan, WANG Ronggui, XUE Lixia. Lightweight Image Super-Resolution Reconstruction Based on Dynamic Adaptive Cascade Network[J]. Computer Engineering, 2022, 48(12): 196-202.

https://www.ecice06.com/CN/Y2022/V48/I12/196

图/表 7

20230112183839

20230112183842

20230112183846

20230112183849

20230112183853

20230112183856

20230112183900

参考文献

[1] GOHSHI S.Real-time super resolution algorithm for security cameras[C]//Proceedings of the 12th International Joint Conference on e-Business and Telecommunications.Washington D.C., USA:IEEE Press, 2015:92-97.
[2] TAO H J, LU X B.Contour-based smoky vehicle detection from surveillance video for alarm systems[J].Signal, Image and Video Processing, 2019, 13(2):217-225.
[3] ISAAC J S, KULKARNI R.Super resolution techniques for medical image processing[C]//Proceedings of International Conference on Technologies for Sustainable Development.Washington D.C., USA:IEEE Press, 2015:1-6.
[4] ARÀNDIGA F.A nonlinear algorithm for monotone piecewise bicubic interpolation[J].Applied Mathematics and Computation, 2016, 272:100-113.
[5] 戚曹, 朱桂斌, 唐鉴波, 等.基于稀疏表示的红外视频图像超分辨率算法[J].计算机工程, 2016, 42(3):278-282. QI C, ZHU G B, TANG J B, et al.Super-resolution algorithm of infrared video image based on sparse representation[J].Computer Engineering, 2016, 42(3):278-282.(in Chinese)
[6] 王诗言, 曾茜, 周田, 等.基于注意力机制与特征融合的图像超分辨率重建[J].计算机工程, 2021, 47(3):269-275, 283. WANG S Y, ZENG X, ZHOU T, et al.Image super-resolution reconstruction based on attention mechanism and feature fusion[J].Computer Engineering, 2021, 47(3):269-275, 283.(in Chinese)
[7] CHEN Y B, LIU S F, WANG X L.Learning continuous image representation with local implicit image function[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2021:8624-8634.
[8] DONG C, LOY C C, HE K M, et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[9] KIM J W, LEE J K, LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1646-1654.
[10] KIM J W, LEE J K, LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1637-1645.
[11] TAI Y, YANG J, LIU X M.Image super-resolution via deep recursive residual network[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:3147-3155.
[12] SHI W Z, CABALLERO J, HUSZÁR F, et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1874-1883.
[13] LIM B, SON S, KIM H, et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2017:1132-1140.
[14] ZHANG Y L, LI K P, LI K, et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:294-310.
[15] HU J, SHEN L, SUN G.Squeeze-and-excitation networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7132-7141.
[16] AHN N, KANG B, SOHN K A.Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision.New York, USA:ACM Press, 2018:252-268.
[17] MUQEET A, HWANG J, YANG S B, et al.Ultra lightweight image super-resolution with multi-attention layers[EB/OL].[2022-03-01].https://arxiv.org/abs/2008.12912.
[18] ZHAO H Y, KONG X T, HE J W, et al.Efficient image super-resolution using pixel attention[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:56-72.
[19] LUO X, XIE Y, ZHANG Y, et al.LatticeNet:towards lightweight image super-resolution with lattice block[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2020:272-289.
[20] CHEN Y P, DAI X Y, LIU M C, et al.Dynamic convolution:attention over convolution kernels[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:11027-11036.
[21] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of British Machine Vision Conference.Berlin, Germany:Springer, 2012:135.
[22] ZEYDE R, ELAD M, PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of International Conference on Curves and Surfaces.Berlin, Germany:Springer, 2012:711-730.
[23] MARTIN D, FOWLKES C, TAL D, et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2001:416-423.
[24] HUANG J B, SINGH A, AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:5197-5206.
[25] LAI W S, HUANG J B, AHUJA N, et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:5835-5843.
[26] TAI Y, YANG J, LIU X M, et al.MemNet:a persistent memory network for image restoration[C]//Proceedings of International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:4539-4547.
[27] LIU J, ZHANG W J, TANG Y T, et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:2359-2368.

选择文件类型/文献管理软件名称

选择包含的内容