基于深度可分离卷积的轻量级图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0061892

摘要/Abstract

摘要： 图像超分辨率重建旨在依据低分辨率图像重建出接近真实的高分辨率图像，现有基于卷积神经网络的图像超分辨率重建方法存在网络参数量大、重建速度慢等问题，从而限制其在内存资源小的终端设备上的应用。提出一种基于深度可分离卷积的轻量级图像超分辨率重建网络，利用深度可分离卷积提取图像的特征信息，减少网络的参数量，采用对比度感知通道注意力机制获取图像的对比度信息，并将其作为全局信息，同时对提取特征的不同通道权重进行重新分配，增强重建图像的细节纹理信息。在此基础上，采用亚像素卷积对图像特征进行上采样操作，提高整体重建图像质量。实验结果表明，当放大倍数为2、3和4时，该网络的参数量分别为140 000、147 000和152 000，重建时间为0.020 s、0.014 s和0.011 s，相比VDSR、RFDN、IDN等网络，在保证重建效果的前提下能够有效减少网络参数量。

关键词: 卷积神经网络, 图像超分辨率重建, 轻量级网络, 深度可分离卷积, 注意力机制

Abstract: Image super-resolution reconstruction aims to reconstruct a high-resolution image close to the real image according to the low-resolution image.The existing image super-resolution reconstruction methods based on the Convolutional Neural Network(CNN) usually have problems such as large network parameters and slow reconstruction speeds, which have limited their application in terminal devices with small memory resources.This paper proposes a lightweight image super-resolution reconstruction network based on depthwise separable convolution.The depthwise separable convolution is used to extract the feature information of the image and reduce network parameters.The contrast information of the image is obtained using the contrast perception channel attention mechanism, which is used as the global information.At the same time, the weights of different channels of the extracted features are reassigned to enhance the detailed texture information of the reconstructed image.On this basis, sub-pixel convolution is used to upsample the image features and improve the overall reconstructed image quality.The experimental results show that when the magnification is 2, 3, and 4, the network parameters are 140 000, 147 000, and 152 000, respectively, and the reconstruction time is 0.020 s, 0.014 s, and 0.011 s.Compared with VDSR, RFDN, IDN, and other networks, it can effectively reduce the amount of network parameters on the premise of ensuring the reconstruction effect.

Key words: Convolutional Neural Network(CNN), image super-resolution reconstruction, lightweight network, depthwise separable convolution, attention mechanism

中图分类号:

TP391

柳聪, 屈丹, 司念文, 魏紫薇. 基于深度可分离卷积的轻量级图像超分辨率重建[J]. 计算机工程, 2022, 48(6): 228-234.

LIU Cong, QU Dan, SI Nianwen, WEI Ziwei. Lightweight Image Super-Resolution Reconstruction Based on Depthwise Separable Convolution[J]. Computer Engineering, 2022, 48(6): 228-234.

http://www.ecice06.com/CN/Y2022/V48/I6/228

图/表 11

20220625180013

20220625180017

20220625180021

20220625180024

20220625180030

20220625180034

20220625180037

20220625180042

20220625180045

20220625180052

20220625180055

参考文献

[1] DONG C, LOY C C, HE K M, et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[2] ZHANG Y L, TIAN Y P, KONG Y, et al.Residual dense network for image restoration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7):2480-2495.
[3] MEI S H, JIANG R T, LI X, et al.Spatial and spectral joint super-resolution using convolutional neural network[J].IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7):4590-4603.
[4] TANG Y L, GONG W G, CHEN X, et al.Deep inception-residual Laplacian pyramid networks for accurate single-image super-resolution[J].IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(5):1514-1528.
[5] YANG W M, WANG W, ZHANG X C, et al.Lightweight feature fusion network for single image super-resolution[J].IEEE Signal Processing Letters, 2019, 26(4):538-542.
[6] HUI Z, WANG X M, GAO X B.Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:723-731.
[7] LIU J, TANG J, WU G S.Residual feature distillation network for lightweight image super-resolution[EB/OL].[2021-05-07].https://arxiv.org/abs/2009.11551.
[8] KIM J, LEE J K, LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1637-1645.
[9] TAI Y, YANG J, LIU X M.Image super-resolution via deep recursive residual network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:2790-2798.
[10] HOWARD A G, ZHU M L, CHEN B, et al.MobileNets:efficient convolutional neural networks for mobile vision applications[EB/OL].[2021-05-07].https://arxiv.org/pdf/1704.04861.pdf.
[11] 程德强, 郭昕, 陈亮亮, 等.多通道递归残差网络的图像超分辨率重建[J].中国图象图形学报, 2021, 26(3):605-618. CHENG D Q, GUO X, CHEN L L, et al.Image super-resolution reconstruction from multi-channel recursive residual network[J].Journal of Image and Graphics, 2021, 26(3):605-618.(in Chinese)
[12] 刘超, 张晓晖, 胡清平.图像超分辨率卷积神经网络加速算法[J].国防科技大学学报, 2019, 41(2):91-97. LIU C, ZHANG X H, HU Q P.Image super resolution convolution neural network acceleration algorithm[J].Journal of National University of Defense Technology, 2019, 41(2):91-97.(in Chinese)
[13] HH J, SHEN L, SUN G.Squeeze-and-Excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023.
[14] 王诗言, 曾茜, 周田, 等.基于注意力机制与特征融合的图像超分辨率重建[J].计算机工程, 2021, 47(3):269-275, 283. WANG S Y, ZENG X, ZHOU T, et al.Image super-resolution reconstruction based on attention mechanism and feature fusion[J].Computer Engineering, 2021, 47(3):269-275, 283.(in Chinese)
[15] LAI W S, HUANG J B, AHUJA N, et al.Fast and accurate image super-resolution with deep Laplacian pyramid networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11):2599-2613.
[16] YANG X, MEI H Y, ZHANG J Q, et al.DRFN:deep recurrent fusion network for single-image super-resolution with large factors[J].IEEE Transactions on Multimedia, 2019, 21(2):328-337.
[17] YANG W M, ZHANG X C, TIAN Y P, et al.Deep learning for single image super-resolution:a brief review[J].IEEE Transactions on Multimedia, 2019, 21(12):3106-3121.
[18] 鲁甜, 刘蓉, 刘明, 等.基于特征图注意力机制的图像超分辨率重建[J].计算机工程, 2021, 47(3):261-268. LU T, LIU R, LIU M, et al.Image super-resolution reconstruction based on attention mechanism of feature map[J].Computer Engineering, 2021, 47(3):261-268.(in Chinese)
[19] MATSUI Y, ITO K, ARAMAKI Y, et al.Sketch-based manga retrieval using manga109 dataset[J].Multimedia Tools and Applications, 2017, 76(20):21811-21838.
[20] KIM J, LEE J K, LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1646-1654.
[21] TAI Y, YANG J, LIU X M, et al.MemNet:a persistent memory network for image restoration[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:4549-4557.
[22] SUN L, LIU Z B, SUN X Y, et al.Lightweight image super-resolution via weighted multi-scale residual network[J].IEEE/CAA Journal of Automatica Sinica, 2021, 8(7):1271-1280.
[23] ZHAO X L, ZHANG Y L, ZHANG T, et al.Channel splitting network for single MR image super-resolution[J].IEEE Transactions on Image Processing, 2019, 28(11):5649-5662.

选择文件类型/文献管理软件名称

选择包含的内容