结合对比感知损失和融合注意力的图像去雾模型

doi:10.19678/j.issn.1000-3428.0065255

摘要/Abstract

摘要：

基于深度学习的图像去雾方法主要是通过增加网络的深度或宽度来提升算法的性能，但是这样需要更多的计算资源，并且现有的去雾模型仅将无雾图像作为正样本来指导网络的训练，未能充分利用负样本即雾天图像。为了进一步利用雾天图像和无雾图像对之间的特征差异，并且更灵活地处理不同尺度、位置、范围、角度等区域特征，对重要的特征赋予更大的权重，在DehazeFormer-T模型基础上加入对比感知损失和融合注意力机制，提出改进的CFFormer模型。以L1损失函数度量真实图像和预测图像之间的重建损失，采用对比感知损失函数提取固定的预训练网络VGG16的权重，提升对比学习的能力，并将真实无雾图像和雾天图像分别作为正负样本，拉近预测图像和清晰图像，同时推远有雾图像。此外，将尺度注意力、空间注意力和通道注意力进行融合，在特征图的不同维度上分别应用注意力机制，使网络关注更重要的信息。实验结果表明，CFFormer在RESIDE的ITS数据集上PSNR和SSIM指标比DehazeFormer-T分别提高9.4%和0.6%，验证了模型的有效性。

关键词: 图像去雾, 深度学习, DehazeFormer模型, CFFormer模型, 对比感知损失, 融合注意力

Abstract:

Existing deep learning-based image dehazing methods mainly expand the depth or width of a network to improve its performance, requiring more computational resources. In addition, current dehazing models only adopt haze-free images as positives to guide the training of the network, leaving the negatives, that is hazy images, unattended.Therefore, it is crucial to further exploit the feature differences between hazy and haze-free image pairs in mainstream image dehazing models and deal with regional features of different scales, locations, ranges, and angles more flexibly, thus giving more attention and weight to important features.To this end, this study proposes CFFormer, an improved DehazeFormer-T model that adopts the Contrastive Perceptual Loss(CPL) and Fusion Attention(FA) mechanism.The L1 loss function is used as the reconstruction loss between the ground-truth image and the predicted image. Moreover, the CPL function is introduced because, using fixed pre-trained weights of VGG16, CPL increases the network's ability to learn contrastively. Furthermore, CPL considers real haze-free and a hazy images as positive and negative samples, respectively, bringing the predicted image closer to the clear image while pushing it away from the hazy image. The FA fuses three types of attention: scale, spatial, and channel attentions.These attention mechanisms are separately applied to different dimensions of the feature maps to ensure that the network pays more attention to important information. On the ITS dataset of RESIDE, the experimental results show that with respect to the PSNR and SSIM metrics, the proposed model is 9.4% and 0.6% higher than DehazeFormer-T, respectively, confirming the effectiveness of the proposed model.

Key words: image dehazing, deep learning, DehazeFormer model, CFFormer model, Contrastive Perceptual Loss(CPL), Fusion Attention(FA)

王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.

Kezheng WANG, Yufeng XU, Shangbo ZHOU. Image Dehazing Model Combined with Contrastive Perceptual Loss and Fusion Attention[J]. Computer Engineering, 2023, 49(8): 207-214.

https://www.ecice06.com/CN/Y2023/V49/I8/207

图/表 7

图1 CFFormer网络架构

Fig.1 Architecture of CFFormer network

图2 融合注意力块的实现方式

Fig.2 Implementation method of fusion attention block

表1 融合注意力块数n取值实验结果

Table 1 Experimental results of number n values of fusion attention block

n	PSNR/dB	SSIM	参数量/10⁶
0	35.15	0.989	0.686
1	35.46	0.990	1.160
2	36.54	0.992	1.160
3	36.84	0.992	1.730

图3 4组SOTS室内图像在不同模型上的去雾结果

Fig.3 Defogging results of different models on four sets of SOTS indoor images

图4 4组SOTS室外图像在不同模型上的去雾结果

Fig.4 Defogging results of different models on four sets of SOTS outdoor images

表2 各模型与CFFormer的评价指标与网络规模对比

Table 2 Comparison of evaluation indicator and network scale between each model and CFFormer

模型	PSNR/dB	SSIM	参数量/10⁶
DCP^[2]	16.62	0.818	—
DehazeNet^[5]	19.82	0.821	0.009
MSCNN^[6]	19.84	0.833	0.008
AOD-Net^[7]	20.51	0.816	0.002
FFA-Net^[8]	36.39	0.989	4.456
DehazeFormer-T^[11]	35.15	0.989	0.686
DehazeFormer-S^[11]	36.82	0.992	1.283
DehazeFormer-B^[11]	37.84	0.994	2.514
DehazeFormer-M^[11]	38.46	0.994	4.634
DehazeFormer-L^[11]	40.05	0.996	25.440
CFFormer	38.45	0.995	1.160

表3 对比感知损失和融合注意力的有效性验证

Table 3 Effectiveness validation of contrastive perceptual loss and fusion attention

CPL	FA	PSNR/dB	SSIM
		35.15	0.989
√		37.03	0.993
	√	36.54	0.992
√	√	38.45	0.995

参考文献 26

1	NAYAR S K, NARASIMHAN S G. Vision in bad weather[C]//Proceedings of the 7th IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2002: 820-827.
2	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (12): 2341- 2353. doi: 10.1109/TPAMI.2010.168
3	ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2015, 24 (11): 3522- 3533. doi: 10.1109/TIP.2015.2446191
4	赵慧, 魏伟波, 潘振宽, 等. 基于暗原色先验与变分正则化的图像去雾研究. 计算机工程, 2021, 47 (10): 214- 220. URL
	ZHAO H, WEI W B, PAN Z K, et al. Research on image dehazing based on dark channel prior and variational regularization. Computer Engineering, 2021, 47 (10): 214- 220. URL
5	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25 (11): 5187- 5198. doi: 10.1109/TIP.2016.2598681
6	REN W Q, PAN J S, ZHANG H, et al. Single image dehazing via multi-scale convolutional neural networks with holistic edges. International Journal of Computer Vision, 2020, 128 (1): 240- 259. doi: 10.1007/s11263-019-01235-8
7	LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: all-in-one dehazing network[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 4780-4788.
8	QIN X, WANG Z L, BAI Y C, et al. FFA-Net: feature fusion attention network for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34 (7): 11908- 11915. doi: 10.1609/aaai.v34i07.6865
9	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2010.11929.
10	LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical Vision Transformer using shifted windows[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2022: 9992-10002.
11	SONG Y D, HE Z Q, QIAN H, et al. Vision Transformers for single image dehazing[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2204.03883.
12	LI X, WANG W H, HU X L, et al. Selective kernel networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 510-519.
13	HENDRYCKS D, GIMPEL K. Gaussian Error Linear Units (GELUs)[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1606.08415.
14	PARK N, KIM S. How do Vision Transformers work?[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2202.06709.
15	JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 694-711.
16	RAD M S, BOZORGTABAR B, MARTI U V, et al. SROBB: targeted perceptual loss for single image super-resolution[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 2710-2719.
17	KHOSLA P, TETERWAK P, WANG C, et al. Supervised contrastive learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2020: 18661-18673.
18	CHOI H, SOM A, TURAGA P. AMC-Loss: angular margin contrastive loss for improved explainability in image classification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3659-3666.
19	XIE E Z, DING J, WANG W H, et al. DetCo: unsupervised contrastive learning for object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2022: 8372-8381.
20	SUN B, LI B H, CAI S C, et al. FSCE: few-shot object detection via contrastive proposal encoding[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 7348-7358.
21	WANG X L, ZHANG R F, SHEN C H, et al. Dense contrastive learning for self-supervised visual pre-training[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 3023-3032.
22	HE K M, FAN H Q, WU Y X, et al. Momentum contrast for unsupervised visual representation learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 9726-9735.
23	WU Z R, XIONG Y J, YU S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3733-3742.
24	WANG L G, WANG Y Q, DONG X Y, et al. Unsupervised degradation representation learning for blind super-resolution[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 10576-10585.
25	ZOU Y H, FU Y. Estimating fine-grained noise model via contrastive learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 12672-12681.
26	MO Y Z, LI C F, ZHENG Y H, et al. DCA-CycleGAN: unsupervised single image dehazing using dark channel attention optimized CycleGAN. Journal of Visual Communication and Image Representation, 2022, 82, 103431. doi: 10.1016/j.jvcir.2021.103431

[1]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[2]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[3]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.
[4]	徐明亮, 李芳媛, 马浩然, 何飞. 大规模神经记录的峰电位聚类算法(特邀)[J]. 计算机工程, 2024, 50(6): 1-34.
[5]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[6]	李子杰, 周菊香, 韩晓瑜, 甘健侯, 鹿泽光, 王俊. 序列特征与学习过程融合的知识追踪模型[J]. 计算机工程, 2024, 50(6): 77-85.
[7]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.
[8]	李致金, 汤佳辉, 闫金凤. 基于边缘计算的轻量化识别方法[J]. 计算机工程, 2024, 50(6): 287-295.
[9]	李雪, 王雅文, 张前进. 基于信息检索的源代码自动命名[J]. 计算机工程, 2024, 50(6): 304-310.
[10]	孙帮勇, 马铭, 于涛. 基于区域特征强化的多尺度伪装目标检测方法[J]. 计算机工程, 2024, 50(5): 209-219.
[11]	郑娟毅, 张庆珏, 董嘉豪, 郭梦月, 杨溥江. 一种深度学习的波束空间信道估计算法[J]. 计算机工程, 2024, 50(5): 298-305.
[12]	赵逸飞, 张俊华. 虚拟脊柱侧凸病例腰椎自动生成方法[J]. 计算机工程, 2024, 50(5): 323-329.
[13]	刘仲民, 严梁. 融合动态特征与注意力的敦煌壁画修复模型[J]. 计算机工程, 2024, 50(5): 342-353.
[14]	隗昊, 刁宏悦, 孔亮宸, 邓耀臣. 东北亚舆情文本细粒度命名实体识别方法研究[J]. 计算机工程, 2024, 50(5): 354-362.
[15]	隗昊, 刁宏悦, 孔亮宸, 邓耀臣. 东北亚舆情文本细粒度命名实体识别方法研究[J]. 计算机工程, 2024, 50(5): 354-362.

选择文件类型/文献管理软件名称

选择包含的内容