基于特征差异的多尺度特征融合去雾网络研究

doi:10.19678/j.issn.1000-3428.0068583

摘要/Abstract

摘要：

雾霾是大气污染物在逆温等气象条件下聚集和累积而形成的混浊物质, 其可见性十分有限。图像去雾技术能够消除由雾霾导致的模糊、低对比度等问题, 提高图像的清晰度和可见性, 但也存在图像细节信息丢失等问题。为此, 提出一种基于特征差异的多尺度特征融合去雾(FD-CA dehaze)网络。对FFA-Net的基本块结构进行改进, 分别从特征差异维度、坐标维度和通道维度提取中间特征信息。提出有效坐标注意力(ECA)模块, 将全局池化、最大池化与坐标位置信息相结合, 用于减轻特征融合过程中的位置信息丢失问题; 将通道注意力与ECA模块相结合, 构建双注意力(D-CA)模型, 更好地利用空间信息和通道信息, 进一步提升模型在图像去雾任务中的表现。在此基础上, 改进损失函数, 将L1损失与感知损失相结合。在综合目标测试集(SOTS)和混合主观测试集(HSTS)中进行实验, 结果表明, FD-CA dehaze网络在峰值信噪比、结构相似度2个指标上分别达到37.93 dB和0.990 5, 相较于FFA-Net、GridDehazeNet等经典去雾网络, FD-CA dehaze的去雾效果得到明显提升。

关键词: 图像去雾, 特征融合, 特征差异, 坐标注意力, 通道注意力

Abstract:

Haze, formed by the accumulation and concentration of atmospheric pollutants under meteorological conditions, such as temperature inversion, severely limits visibility. Image dehazing techniques aim to eliminate issues caused by haze, such as image blur and low contrast, thereby enhancing image clarity and visibility. However, challenges persist regarding the loss of image details. To address this issue, a feature difference-based multi-scale feature fusion dehazing network known as FD-CA dehaze is proposed in this study. In this network, the basic block structure of the FFA-Net is enhanced by extracting intermediate feature information from the feature difference, coordinate, and channel dimensions. An Effective Coordinate Attention (ECA) module that combines global pooling, max pooling, and coordinate positional information is introduced. This module mitigates the positional information loss during feature fusion. By integrating channel attention with the ECA module, a Dual Attention (D-CA) model that enables better utilization of spatial and channel information is constructed. Consequently, the model exhibits enhanced performance in image dehazing tasks. Furthermore, the loss function is improved by combining L1 loss function with perceptual loss. Experimental results on the Synthetical Objective Test Set (SOTS) and Hybrid Subjective Test Set (HSTS) demonstrate that the FD-CA dehaze network achieves a Peak Signal-to-Noise Ratio (PSNR) of 37.93 dB and a Structural Similarity Index (SSIM) of 0.990 5. Experimental results demonstrate that compared to classic dehazing networks such as FFA-Net and GridDehazeNet, FD-CA dehaze achieves significant improvement and better dehazing performance.

Key words: image dehazing, feature fusion, feature difference, coordinate attention, channel attention

刘彦红, 杨秋翔, 胡帅. 基于特征差异的多尺度特征融合去雾网络研究[J]. 计算机工程, 2024, 50(4): 247-257.

Yanhong LIU, Qiuxiang YANG, Shuai HU. Research on Multi-Scale Feature Fusion Dehazing Network Based on Feature Differences[J]. Computer Engineering, 2024, 50(4): 247-257.

http://www.ecice06.com/CN/Y2024/V50/I4/247

图/表 21

图1 大气散射模型成像原理

Fig.1 Imaging principles of atmospheric scattering model

图2 FD-CA dehaze网络结构

Fig.2 Network structure of FD-CA dehaze

图3 RC模块结构

Fig.3 RC module structure

图4 ECA模块结构

Fig.4 ECA module structure

图5 D-CA模块结构

Fig.5 D-CA module structure

图6 学习率对比结果

Fig.6 Learning rate comparison results

图7 SOTS测试集上的去雾效果

Fig.7 The dehazing effect on the SOTS test set

图8 HSTS测试集上的去雾效果

Fig.8 The dehazing effect on the HSTS test set

图9 去雾效果1

Fig.9 Dehazing effect 1

图10 去雾效果2

Fig.10 Dehazing effect 2

图11 网络结构消融实验对比

Fig.11 Comparison of network structure ablation experiments

图12 网络结构消融实验中的PSNR指标

Fig.12 PSNR index in network structure ablation experiments

图13 网络结构消融实验中的SSIM指标

Fig.13 SSIM index in network structure ablation experiments

图14 网络结构消融实验中的训练loss

Fig.14 Training loss in network structure ablation experiments

参考文献 25

1	贾童瑶, 卓力, 李嘉锋, 等. 基于深度学习的单幅图像去雾研究进展. 电子学报, 2023, 51(1): 231- 245. URL
	JIA T Y, ZHUO L, LI J F, et al. Research advances on deep learning based single image dehazing. Acta Electronica Sinica, 2023, 51(1): 231- 245. URL
2	魏轶伦, 徐海文. 基于深度学习的图像去雾方法综述研究. 信息技术与信息化, 2023,(4): 214-216, 220. doi: 10.3969/j.issn.1672-9528.2023.04.052
	WEI Y L, XU H W. Overview of image defogging methods based on deep learning. Information Technology and Informatization, 2023,(4): 214-216, 220. doi: 10.3969/j.issn.1672-9528.2023.04.052
3	QIN X, WANG Z L, BAI Y C, et al. FFA-Net: feature fusion attention network for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11908- 11915. doi: 10.1609/aaai.v34i07.6865
4	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11534-11542.
5	MCCARTNEYE J. Optics of the atmosphere: scattering by molecules and particles[EB/OL]. [2023-09-05]. https://ui.adsabs.harvard.edu/abs/1976nyjw. book..... M/abstract.
6	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341- 2353. doi: 10.1109/TPAMI.2010.168
7	ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24(11): 3522- 3533. doi: 10.1109/TIP.2015.2446191
8	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25(11): 5187- 5198. doi: 10.1109/TIP.2016.2598681
9	LI Y N, MIAO Q G, OUYANG W L, et al. LAP-Net: level-aware progressive network for image dehazing[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 3275-3284.
10	LI B, PENG X, WANG Z, et al. An all-in-one network for dehazing and beyond[EB/OL]. [2023-09-05]. https://arxiv.org/pdf/1707.06543.pdf.
11	ZHAO S Y, ZHANG L, SHEN Y, et al. RefineDNet: a weakly supervised refinement framework for single image dehazing. IEEE Transactions on Image Processing, 2021, 30, 3391- 3404. doi: 10.1109/TIP.2021.3060873
12	WU R Q, DUAN Z P, GUO C L, et al. RIDCP: revitalizing real image dehazing via high-quality codebook priors[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2023: 22282-22291.
13	CHEN D D, HE M M, FAN Q N, et al. Gated context aggregation network for image dehazing and deraining[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2019: 1375-1383.
14	杨振舰, 尚佳美, 张众维, 等. 基于残差注意力机制的图像去雾算法. 西北工业大学学报, 2021, 39(4): 901- 908. URL
	YANG Z J, SHANG J M, ZHANG Z W, et al. A new end-to-end image dehazing algorithm based on residual attention mechanism. Journal of Northwestern Polytechnical University, 2021, 39(4): 901- 908. URL
15	LIU X H, MA Y R, SHI Z H, et al. GridDehazeNet: attention-based multi-scale network for image dehazing[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 7313-7322.
16	肖进胜, 周景龙, 雷俊锋, 等. 面向图像场景转换的改进型生成对抗网络. 软件学报, 2021, 32(9): 2755- 2768. URL
	XIAO J S, ZHOU J L, LEI J F, et al. Improved generative adversarial network for image scene transformation. Journal of Software, 2021, 32(9): 2755- 2768. URL
17	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. [2023-09-05]. https://arxiv.org/abs/1706.03762v4.
18	SONG Y D, HE Z Q, QIAN H, et al. Vision transformers for single image dehazing. IEEE Transactions on Image Processing, 2023, 32, 1927- 1941. doi: 10.1109/TIP.2023.3256763
19	ZHAO H Y, KONG X T, HE J W, et al. Efficient image super-resolution using pixel attention[EB/OL]. [2023-09-05]. http://link.springer.com/chapter/10.1007/978-3-030-67070-2_3.
20	SONG M, LIM S, KIM W. Monocular depth estimation using Laplacian pyramid-based depth residuals. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(11): 4381- 4393. doi: 10.1109/TCSVT.2021.3049869
21	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7132-7141.
22	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-19.
23	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 13713-13722.
24	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2023-09-05]. http://arxiv.org/pdf/1409.1556.
25	LI B, REN W, FU D, et al. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 2018, 28(1): 492- 505.

[1]	崔丽群, 曹华维. 基于改进YOLOv5的遥感图像目标检测[J]. 计算机工程, 2024, 50(4): 228-236.
[2]	李振鲁, 黄威, 孙锴. 复杂环境下的轻量化道路目标识别算法研究[J]. 计算机工程, 2024, 50(4): 219-227.
[3]	安玉, 葛海波, 何文昊, 马赛, 程梦洋. 基于补偿注意力机制的Siamese网络跟踪算法[J]. 计算机工程, 2024, 50(4): 187-196.
[4]	谢帅康, 熊风光, 朱新杰, 宋宁栋, 李文清, 王廷凤. 基于空间可变形Transformer的三维点云配准方法[J]. 计算机工程, 2024, 50(3): 224-232.
[5]	倪林, 刘子辉, 张帅, 韩久江, 鲜明. 基于灰度图谱分析的IP软核硬件木马检测方法[J]. 计算机工程, 2024, 50(3): 44-51.
[6]	冯妍舟, 刘建霞, 王海翼, 冯国昊, 白宇. 基于多级残差信息蒸馏的真实图像去噪方法[J]. 计算机工程, 2024, 50(3): 216-223.
[7]	徐浩宸, 刘满华. 基于多层次自注意力网络的人脸特征点检测[J]. 计算机工程, 2024, 50(2): 239-246.
[8]	李伟健, 胡慧君. 基于潜在特征增强网络的视频描述生成方法[J]. 计算机工程, 2024, 50(2): 266-272.
[9]	王正家, 胡飞飞, 张成娟, 雷卓, 何涛. 引入轻量级Transformer的自适应窗口立体匹配算法[J]. 计算机工程, 2024, 50(2): 256-265.
[10]	杨瑞君, 秦晋京, 程燕. 基于生成对抗网络的自然场景低照度增强模型[J]. 计算机工程, 2024, 50(1): 279-288.
[11]	范文卓, 吴涛, 许俊平, 李庆庆, 张建林, 李美惠, 魏宇星. 基于多分辨率特征融合的任意尺度图像超分辨率重建[J]. 计算机工程, 2023, 49(9): 217-225.
[12]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[13]	包善书, 车波, 邓林红. 基于双源域迁移学习的肺音信号识别[J]. 计算机工程, 2023, 49(9): 295-302, 312.
[14]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[15]	卢昂, 储珺, 冷璐. 基于高低频特征增强的图像去雾[J]. 计算机工程, 2023, 49(8): 174-181.

选择文件类型/文献管理软件名称

选择包含的内容