基于自注意力特征蒸馏的轻量级图像超分辨率重建

doi:10.19678/j.issn.1000-3428.0069822

摘要/Abstract

摘要： 单幅图像超分辨率(SISR)旨在从给定的低分辨率(LR)图像中重建出高分辨率(HR)图像。近年来,基于深度学习的SISR算法取得了出色的重建效果,引起了广泛关注。然而,大多数基于深度学习的SISR算法存在复杂度高、参数量大等问题,影响实际应用。为了克服以上问题,提出一种基于自注意力特征蒸馏的模块,在降低模型复杂度的同时充分提取图像深层特征,实现轻量化的超分辨率重建。该模块包含2个技术创新:一是在全局注意力的计算中提出基于非对称卷积的前馈网络,利用非对称卷积优越的非线性特征提取能力压缩输入通道,节省计算开销;二是在空间注意力模块中引入部分通道位移操作,在不增加计算量的同时,通过位移部分通道达到提高特征多样性的目的。在6个常用公开数据集上的超分辨率实验结果表明,相比于CARN、SMSR、DLGSANet等具有代表性的轻量级SISR算法,所提算法在峰值信噪比(PSNR)、结构相似度(SSIM)、LPIPS评价指标上取得了更好的表现,同时重建结果的主观视觉效果更优,更好地平衡了模型复杂度与重建性能。

关键词: 图像超分辨率, 特征蒸馏, 深度学习, 非对称卷积, 自注意力

Abstract: Single Image Super-Resolution (SISR) reconstructs High-Resolution (HR) images from Low-Resolution (LR) images. In recent years, deep learning-based SISR methods have achieved outstanding reconstruction results, attracting widespread attention. However, most models suffer from high complexity and large parameter size, which affects their practical application. To overcome these issues, this study proposes a module based on self-attention feature distillation, which reduces complexity while fully extracting deep image features, achieving lightweight super-resolution reconstruction. The proposed module has two technical features. First, a feedback network based on asymmetric convolution is proposed for computing global attention, utilizing the superior nonlinear feature extraction capability of asymmetric convolution to compress input channels and reduce computational costs. Second, a partial channel shifting operation is introduced in the spatial attention module to increase feature diversity by shifting partial channels without increasing the computational complexity. In experiments on six commonly used public datasets, the proposed method outperforms representative lightweight SISR methods such as CARN, SMSR, and DLGSANet in terms of Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS). In addition, the subjective visual quality of the reconstruction results produced by the proposed method is superior. Overall, the proposed method achieves a better balance between model complexity and reconstruction performance.

Key words: image super-resolution, feature distillation, deep learning, asymmetric convolution, self-attention

中图分类号:

TP391

赵瑶谦, 滕奇志, 何小海, 税爱, 陈洪刚. 基于自注意力特征蒸馏的轻量级图像超分辨率重建[J]. 计算机工程, 2025, 51(5): 257-265.

ZHAO Yaoqian, TENG Qizhi, HE Xiaohai, SHUI Ai, CHEN Honggang. Lightweight Image Super-Resolution Reconstruction Based on Self-Attention Feature Distillation[J]. Computer Engineering, 2025, 51(5): 257-265.

https://www.ecice06.com/CN/Y2025/V51/I5/257

参考文献

[1] CHEN H G, HE X H, QING L B, et al. Real-world single image super-resolution:a brief review[J]. Information Fusion, 2022, 79:124-145.
[2] 丁子轩,俞雷,张娟,等.基于深度残差自适应注意力网络的图像超分辨率重建[J].计算机工程, 2023, 49(5):231-238. DING Z X, YU L, ZHANG J, et al. Image super-resolution reconstruction based on depth residual adaptive attention network[J]. Computer Engineering, 2023, 49(5):231-238.(in Chinese)
[3] JEEVAN P, SRINIDHI A, PRATHIBA P, et al. WaveMixSR:resource-efficient neural network for image super-resolution[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington D.C.,USA:IEEE Press,2024:5884-5892.
[4] 王晔,孙志宽,李征.基于生成对抗网络与噪声分布的图像超分辨率重建方法[J].四川大学学报(自然科学版), 2023, 60(3):45-54. WANG Y, SUN Z K, LI Z. An image super-resolution reconstruction method based on generative adversarial network and noise distribution[J]. Journal of Sichuan University (Natural Science Edition), 2023, 60(3):45-54.(in Chinese)
[5] DONG C, LOY C C, HE K M, et al. Learning a deep convolutional network for image super-resolution[EB/OL].[2024-03-05] . https://link.springer.com/chapter/10.1007/978-3-319-10593-2_13.
[6] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2016:1646-1654.
[7] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C.,USA:IEEE Press,2017:136-144.
[8] 柳聪,屈丹,司念文,等.基于深度可分离卷积的轻量级图像超分辨率重建[J].计算机工程, 2022, 48(6):228-234. LIU C, QU D, SI N W, et al. Lightweight image super-resolution reconstruction based on depthwise separable convolution[J]. Computer Engineering, 2022, 48(6):228-234.(in Chinese)
[9] AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[EB/OL].[2024-03-05] . https://arxiv.org/abs/1803.08664.
[10] HUI Z, WANG X M, GAO X B. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2018:723-731.
[11] HUI Z, GAO X B, YANG Y C, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York,USA:ACM Press,2019:2024-2032.
[12] LI W, ZHOU K, QI L, et al. LAPAR:linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond[EB/OL].[2024-03-05] . https://arxiv.org/abs/2105.10422.
[13] WANG L, DONG X, WANG Y, et al. Exploring sparsity in image super-resolution for efficient inference[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2021:4917-4926.
[14] GAO Q Q, ZHAO Y, LI G, et al. Image super-resolution using knowledge distillation[EB/OL].[2024-03-05] . https://link.springer.com/chapter/10.1007/978-3-030-20890-5_34.
[15] ZHANG X M, LI T R, ZHAO X L. Boosting single image super-resolution via partial channel shifting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C.,USA:IEEE Press,2023:13223-13232.
[16] DING X H, GUO Y C, DING G G, et al. ACNet:strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C.,USA:IEEE Press,2019:1911-1920.
[17] KONG F Y, LI M X, LIU S W, et al. Residual local feature network for efficient super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C.,USA:IEEE Press,2022:766-776.
[18] AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on single image super-resolution:dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C.,USA:IEEE Press,2017:126-135.
[19] KINGMA D P, BA J. Adam:a method for stochastic optimization[EB/OL].[2024-03-05] . https://arxiv.org/abs/1412.6980.
[20] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[EB/OL].[2024-03-05] . http://eprints.imtlucca.it/2412/1/Bevilacqua_2012.pdf.
[21] ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[EB/OL].[2024-03-05] . https://link.springer.com/chapter/10.1007/978-3-642-27413-8_47.
[22] MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision. Washington D.C.,USA:IEEE Press,2001:416-423.
[23] HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2015:5197-5206.
[24] MATSUI Y, ITO K, ARAMAKI Y, et al. Sketch-based manga retrieval using Manga109 dataset[J]. Multimedia Tools and Applications, 2017, 76(20):21811-21838.
[25] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2018:586-595.
[26] LIU J, TANG J, WU G S. Residual feature distillation network for lightweight image super-resolution[EB/OL].[2024-03-05] . https://arxiv.org/abs/2009.11551.
[27] DU Z C, LIU D, LIU J, et al. Fast and memory-efficient network towards efficient image super-resolution[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C.,USA:IEEE Press,2022:853-862.
[28] ZHANG X D, ZENG H, GUO S, et al. Efficient long-range attention network for image super-resolution[EB/OL].[2024-03-05] . https://arxiv.org/abs/2203.06697.
[29] LI X, DONG J X, TANG J H, et al. DLGSANet:lightweight dynamic local and global self-attention network for image super-resolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C.,USA:IEEE Press,2023:12792-12801.

选择文件类型/文献管理软件名称

选择包含的内容