Lightweight Image Super-Resolution Reconstruction Based on Self-Attention Feature Distillation

doi:10.19678/j.issn.1000-3428.0069822

Abstract

Abstract:

Single Image Super-Resolution (SISR) reconstructs High-Resolution (HR) images from Low-Resolution (LR) images. In recent years, deep learning-based SISR methods have achieved outstanding reconstruction results, attracting widespread attention. However, most models suffer from high complexity and large parameter size, which affects their practical application. To overcome these issues, this study proposes a module based on self-attention feature distillation, which reduces complexity while fully extracting deep image features, achieving lightweight super-resolution reconstruction. The proposed module has two technical features. First, a feedback network based on asymmetric convolution is proposed for computing global attention, utilizing the superior nonlinear feature extraction capability of asymmetric convolution to compress input channels and reduce computational costs. Second, a partial channel shifting operation is introduced in the spatial attention module to increase feature diversity by shifting partial channels without increasing the computational complexity. In experiments on six commonly used public datasets, the proposed method outperforms representative lightweight SISR methods such as CARN, SMSR, and DLGSANet in terms of Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS). In addition, the subjective visual quality of the reconstruction results produced by the proposed method is superior. Overall, the proposed method achieves a better balance between model complexity and reconstruction performance.

Key words: image super-resolution, feature distillation, deep learning, asymmetric convolution, self-attention

摘要：

单幅图像超分辨率(SISR)旨在从给定的低分辨率(LR)图像中重建出高分辨率(HR)图像。近年来, 基于深度学习的SISR算法取得了出色的重建效果, 引起了广泛关注。然而, 大多数基于深度学习的SISR算法存在复杂度高、参数量大等问题, 影响实际应用。为了克服以上问题, 提出一种基于自注意力特征蒸馏的模块, 在降低模型复杂度的同时充分提取图像深层特征, 实现轻量化的超分辨率重建。该模块包含2个技术创新: 一是在全局注意力的计算中提出基于非对称卷积的前馈网络, 利用非对称卷积优越的非线性特征提取能力压缩输入通道, 节省计算开销; 二是在空间注意力模块中引入部分通道位移操作, 在不增加计算量的同时, 通过位移部分通道达到提高特征多样性的目的。在6个常用公开数据集上的超分辨率实验结果表明, 相比于CARN、SMSR、DLGSANet等具有代表性的轻量级SISR算法, 所提算法在峰值信噪比(PSNR)、结构相似度(SSIM)、LPIPS评价指标上取得了更好的表现, 同时重建结果的主观视觉效果更优, 更好地平衡了模型复杂度与重建性能。

关键词: 图像超分辨率, 特征蒸馏, 深度学习, 非对称卷积, 自注意力

ZHAO Yaoqian, TENG Qizhi, HE Xiaohai, SHUI Ai, CHEN Honggang. Lightweight Image Super-Resolution Reconstruction Based on Self-Attention Feature Distillation[J]. Computer Engineering, 2025, 51(5): 257-265.

赵瑶谦, 滕奇志, 何小海, 税爱, 陈洪刚. 基于自注意力特征蒸馏的轻量级图像超分辨率重建[J]. 计算机工程, 2025, 51(5): 257-265.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069822

https://www.ecice06.com/EN/Y2025/V51/I5/257

Figures/Tables 13

Fig.1 The overall framework of the algorithm in this paper

Fig.2 The structure of feature distillation block

Fig.3 The structure of ASGA

Fig.4 Schematic diagram of partial channel displacement method principle

Fig.5 The structure of CSSA

Fig.6 Visual effects of MiraiSan image under different algorithms

Fig.7 Comparison of computational complexity on Urban100 dataset

References 29

1	CHEN H G , HE X H , QING L B , et al. Real-world single image super-resolution: a brief review. Information Fusion, 2022, 79, 124- 145. doi: 10.1016/j.inffus.2021.09.005
2	丁子轩, 俞雷, 张娟, 等. 基于深度残差自适应注意力网络的图像超分辨率重建. 计算机工程, 2023, 49 (5): 231- 238. doi: 10.19678/j.issn.1000-3428.0064243
	DING Z X , YU L , ZHANG J , et al. Image super-resolution reconstruction based on depth residual adaptive attention network. Computer Engineering, 2023, 49 (5): 231- 238. doi: 10.19678/j.issn.1000-3428.0064243
3	JEEVAN P, SRINIDHI A, PRATHIBA P, et al. WaveMixSR: resource-efficient neural network for image super-resolution[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington D.C., USA: IEEE Press, 2024: 5884-5892.
4	王晔, 孙志宽, 李征. 基于生成对抗网络与噪声分布的图像超分辨率重建方法. 四川大学学报(自然科学版), 2023, 60 (3): 45- 54. doi: 10.19907/j.0490-6756.2023.032001
	WANG Y , SUN Z K , LI Z . An image super-resolution reconstruction method based on generative adversarial network and noise distribution. Journal of Sichuan University (Natural Science Edition), 2023, 60 (3): 45- 54. doi: 10.19907/j.0490-6756.2023.032001
5	DONG C, LOY C C, HE K M, et al. Learning a deep convolutional network for image super-resolution[EB/OL]. [2024-03-05]. https://link.springer.com/chapter/10.1007/978-3-319-10593-2_13.
6	KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2016: 1646-1654.
7	LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C., USA: IEEE Press, 2017: 136-144.
8	柳聪, 屈丹, 司念文, 等. 基于深度可分离卷积的轻量级图像超分辨率重建. 计算机工程, 2022, 48 (6): 228- 234. doi: 10.19678/j.issn.1000-3428.0061892
	LIU C , QU D , SI N W , et al. Lightweight image super-resolution reconstruction based on depthwise separable convolution. Computer Engineering, 2022, 48 (6): 228- 234. doi: 10.19678/j.issn.1000-3428.0061892
9	AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[EB/OL]. [2024-03-05]. https://arxiv.org/abs/1803.08664.
10	HUI Z, WANG X M, GAO X B. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 723-731.
11	HUI Z, GAO X B, YANG Y C, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York, USA: ACM Press, 2019: 2024-2032.
12	LI W, ZHOU K, QI L, et al. LAPAR: linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond[EB/OL]. [2024-03-05]. https://arxiv.org/abs/2105.10422.
13	WANG L, DONG X, WANG Y, et al. Exploring sparsity in image super-resolution for efficient inference[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2021: 4917-4926.
14	GAO Q Q, ZHAO Y, LI G, et al. Image super-resolution using knowledge distillation[EB/OL]. [2024-03-05]. https://link.springer.com/chapter/10.1007/978-3-030-20890-5_34.
15	ZHANG X M, LI T R, ZHAO X L. Boosting single image super-resolution via partial channel shifting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2023: 13223-13232.
16	DING X H, GUO Y C, DING G G, et al. ACNet: strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2019: 1911-1920.
17	KONG F Y, LI M X, LIU S W, et al. Residual local feature network for efficient super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C., USA: IEEE Press, 2022: 766-776.
18	AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on single image super-resolution: dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C., USA: IEEE Press, 2017: 126-135.
19	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2024-03-05]. https://arxiv.org/abs/1412.6980.
20	BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[EB/OL]. [2024-03-05]. http://eprints.imtlucca.it/2412/1/Bevilacqua_2012.pdf.
21	ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[EB/OL]. [2024-03-05]. https://link.springer.com/chapter/10.1007/978-3-642-27413-8_47.
22	MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2001: 416-423.
23	HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2015: 5197-5206.
24	MATSUI Y , ITO K , ARAMAKI Y , et al. Sketch-based manga retrieval using Manga109 dataset. Multimedia Tools and Applications, 2017, 76 (20): 21811- 21838. doi: 10.1007/s11042-016-4020-z
25	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 586-595.
26	LIU J, TANG J, WU G S. Residual feature distillation network for lightweight image super-resolution[EB/OL]. [2024-03-05]. https://arxiv.org/abs/2009.11551.
27	DU Z C, LIU D, LIU J, et al. Fast and memory-efficient network towards efficient image super-resolution[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C., USA: IEEE Press, 2022: 853-862.
28	ZHANG X D, ZENG H, GUO S, et al. Efficient long-range attention network for image super-resolution[EB/OL]. [2024-03-05]. https://arxiv.org/abs/2203.06697.
29	LI X, DONG J X, TANG J H, et al. DLGSANet: lightweight dynamic local and global self-attention network for image super-resolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2023: 12792-12801.

[1]	WEI Mingkang, LI Jianan, HAN Lin, GAO Wei, ZHAO Rongcai, WANG Hongsheng. Support and Optimization of Multi-Granularity Quantization Framework for Deep Learning Compiler [J]. Computer Engineering, 2025, 51(5): 62-72.
[2]	ZHUANG Ziwei, ZHU Junguo. Vietnamese Text Error Detection Method for Multi-source Text [J]. Computer Engineering, 2025, 51(5): 93-102.
[3]	HAO Zhifeng, LI Yanglin, XU Boyan, CAI Ruichu. Hypergraph Neural Networks for Cross-domain Text-to-SQL [J]. Computer Engineering, 2025, 51(5): 114-123.
[4]	LI Dandan, LI Zhi, ZHENG Long, ZHANG Li. Robust Reversible Watermarking Algorithm for Diffusion Tensor Images [J]. Computer Engineering, 2025, 51(5): 279-287.
[5]	LIU Yunxiang, LIANG Zhichao. A Highly Efficient Traffic Prediction Model for Continuous Time-series Graph Attention Networks [J]. Computer Engineering, 2025, 51(4): 350-359.
[6]	DU Chenyang, ZHANG Xueying, HUANG Lixia, LI Juan. Multi-Feature Speech Emotion Recognition Based on Improved Efficient Channel Attention Mechanism [J]. Computer Engineering, 2025, 51(4): 97-106.
[7]	JIANG Jieping, WANG Mingwen. Residual Behavior Recognition Model Based on Spatio-Temporal Shuffle Attention Mechanism [J]. Computer Engineering, 2025, 51(4): 119-128.
[8]	SUN Ting, YANG Jie, LI Jiaxuan, WANG Yaozong. Optimization of YOLOv7 Road Sign Detection Algorithm for Low-Light Traffic Scenes [J]. Computer Engineering, 2025, 51(3): 342-351.
[9]	DAI Kangjia, XU Huiying, ZHU Xinzhong, LI Xiyu, HUANG Xiao, CHEN Guoqiang, ZHANG Zhixiong. YGL-SLAM: Point and Line Based Semantic SLAM System for Dynamic Scenes [J]. Computer Engineering, 2025, 51(3): 95-104.
[10]	HAN Peng, HUANG Yunzhi, REN Caiyue, CHENG Jingyi, XU Jun. Assessment of Neoadjuvant Chemotherapy Efficacy in Breast Cancer Using Dual-Branch Network with PET Imaging [J]. Computer Engineering, 2025, 51(3): 293-299.
[11]	HU Chaoju, GUO Fengyi. MODF Port State Detection Algorithm Based on Improved YOLOv7 [J]. Computer Engineering, 2025, 51(2): 78-85.
[12]	SUN Haomiao, LI Zongmin, XIAO Qian, SUN Wenjie, ZHANG Wenxin. AI-Curling: An On-Site Curling Analysis and Decision-Making Method [J]. Computer Engineering, 2025, 51(2): 102-110.
[13]	SUN Yikang, GAO Jianhua. Dead Code Detection Method Based on Convolutional Neural Network and Long Short-Term Memory [J]. Computer Engineering, 2025, 51(2): 223-237.
[14]	WANG Yang, SONG Shijia, WANG Heqin, YUAN Zhenyu, ZHAO Lijun, WU Qilin. Estimation of Local Illumination Consistency Based on Improved Vision Transformer [J]. Computer Engineering, 2025, 51(2): 312-321.
[15]	ZHOU Yu, XIE Wei, Kwong Tak Wu, JIANG Jianmin. Reconstruction of Video Snapshot Compressive Imaging Based on Triple Self-Attention [J]. Computer Engineering, 2025, 51(1): 20-30.

Please choose a citation manager

Content to export