结合对比感知损失和融合注意力的图像去雾模型

doi:10.19678/j.issn.1000-3428.0065255

摘要/Abstract

摘要：

基于深度学习的图像去雾方法主要是通过增加网络的深度或宽度来提升算法的性能，但是这样需要更多的计算资源，并且现有的去雾模型仅将无雾图像作为正样本来指导网络的训练，未能充分利用负样本即雾天图像。为了进一步利用雾天图像和无雾图像对之间的特征差异，并且更灵活地处理不同尺度、位置、范围、角度等区域特征，对重要的特征赋予更大的权重，在DehazeFormer-T模型基础上加入对比感知损失和融合注意力机制，提出改进的CFFormer模型。以L1损失函数度量真实图像和预测图像之间的重建损失，采用对比感知损失函数提取固定的预训练网络VGG16的权重，提升对比学习的能力，并将真实无雾图像和雾天图像分别作为正负样本，拉近预测图像和清晰图像，同时推远有雾图像。此外，将尺度注意力、空间注意力和通道注意力进行融合，在特征图的不同维度上分别应用注意力机制，使网络关注更重要的信息。实验结果表明，CFFormer在RESIDE的ITS数据集上PSNR和SSIM指标比DehazeFormer-T分别提高9.4%和0.6%，验证了模型的有效性。

关键词: 图像去雾, 深度学习, DehazeFormer模型, CFFormer模型, 对比感知损失, 融合注意力

Abstract:

Existing deep learning-based image dehazing methods mainly expand the depth or width of a network to improve its performance, requiring more computational resources. In addition, current dehazing models only adopt haze-free images as positives to guide the training of the network, leaving the negatives, that is hazy images, unattended.Therefore, it is crucial to further exploit the feature differences between hazy and haze-free image pairs in mainstream image dehazing models and deal with regional features of different scales, locations, ranges, and angles more flexibly, thus giving more attention and weight to important features.To this end, this study proposes CFFormer, an improved DehazeFormer-T model that adopts the Contrastive Perceptual Loss(CPL) and Fusion Attention(FA) mechanism.The L1 loss function is used as the reconstruction loss between the ground-truth image and the predicted image. Moreover, the CPL function is introduced because, using fixed pre-trained weights of VGG16, CPL increases the network's ability to learn contrastively. Furthermore, CPL considers real haze-free and a hazy images as positive and negative samples, respectively, bringing the predicted image closer to the clear image while pushing it away from the hazy image. The FA fuses three types of attention: scale, spatial, and channel attentions.These attention mechanisms are separately applied to different dimensions of the feature maps to ensure that the network pays more attention to important information. On the ITS dataset of RESIDE, the experimental results show that with respect to the PSNR and SSIM metrics, the proposed model is 9.4% and 0.6% higher than DehazeFormer-T, respectively, confirming the effectiveness of the proposed model.

Key words: image dehazing, deep learning, DehazeFormer model, CFFormer model, Contrastive Perceptual Loss(CPL), Fusion Attention(FA)

王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.

Kezheng WANG, Yufeng XU, Shangbo ZHOU. Image Dehazing Model Combined with Contrastive Perceptual Loss and Fusion Attention[J]. Computer Engineering, 2023, 49(8): 207-214.

http://www.ecice06.com/CN/Y2023/V49/I8/207

图/表 7

图1 CFFormer网络架构

Fig.1 Architecture of CFFormer network

图2 融合注意力块的实现方式

Fig.2 Implementation method of fusion attention block

表1 融合注意力块数n取值实验结果

Table 1 Experimental results of number n values of fusion attention block

n	PSNR/dB	SSIM	参数量/10⁶
0	35.15	0.989	0.686
1	35.46	0.990	1.160
2	36.54	0.992	1.160
3	36.84	0.992	1.730

图3 4组SOTS室内图像在不同模型上的去雾结果

Fig.3 Defogging results of different models on four sets of SOTS indoor images

图4 4组SOTS室外图像在不同模型上的去雾结果

Fig.4 Defogging results of different models on four sets of SOTS outdoor images

表2 各模型与CFFormer的评价指标与网络规模对比

Table 2 Comparison of evaluation indicator and network scale between each model and CFFormer

模型	PSNR/dB	SSIM	参数量/10⁶
DCP^[2]	16.62	0.818	—
DehazeNet^[5]	19.82	0.821	0.009
MSCNN^[6]	19.84	0.833	0.008
AOD-Net^[7]	20.51	0.816	0.002
FFA-Net^[8]	36.39	0.989	4.456
DehazeFormer-T^[11]	35.15	0.989	0.686
DehazeFormer-S^[11]	36.82	0.992	1.283
DehazeFormer-B^[11]	37.84	0.994	2.514
DehazeFormer-M^[11]	38.46	0.994	4.634
DehazeFormer-L^[11]	40.05	0.996	25.440
CFFormer	38.45	0.995	1.160

表3 对比感知损失和融合注意力的有效性验证

Table 3 Effectiveness validation of contrastive perceptual loss and fusion attention

CPL	FA	PSNR/dB	SSIM
		35.15	0.989
√		37.03	0.993
	√	36.54	0.992
√	√	38.45	0.995

参考文献 26

1	NAYAR S K, NARASIMHAN S G. Vision in bad weather[C]//Proceedings of the 7th IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2002: 820-827.
2	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (12): 2341- 2353. doi: 10.1109/TPAMI.2010.168
3	ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2015, 24 (11): 3522- 3533. doi: 10.1109/TIP.2015.2446191
4	赵慧, 魏伟波, 潘振宽, 等. 基于暗原色先验与变分正则化的图像去雾研究. 计算机工程, 2021, 47 (10): 214- 220. URL
	ZHAO H, WEI W B, PAN Z K, et al. Research on image dehazing based on dark channel prior and variational regularization. Computer Engineering, 2021, 47 (10): 214- 220. URL
5	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25 (11): 5187- 5198. doi: 10.1109/TIP.2016.2598681
6	REN W Q, PAN J S, ZHANG H, et al. Single image dehazing via multi-scale convolutional neural networks with holistic edges. International Journal of Computer Vision, 2020, 128 (1): 240- 259. doi: 10.1007/s11263-019-01235-8
7	LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: all-in-one dehazing network[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 4780-4788.
8	QIN X, WANG Z L, BAI Y C, et al. FFA-Net: feature fusion attention network for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34 (7): 11908- 11915. doi: 10.1609/aaai.v34i07.6865
9	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2010.11929.
10	LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical Vision Transformer using shifted windows[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2022: 9992-10002.
11	SONG Y D, HE Z Q, QIAN H, et al. Vision Transformers for single image dehazing[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2204.03883.
12	LI X, WANG W H, HU X L, et al. Selective kernel networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 510-519.
13	HENDRYCKS D, GIMPEL K. Gaussian Error Linear Units (GELUs)[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1606.08415.
14	PARK N, KIM S. How do Vision Transformers work?[EB/OL]. [2022-06-10]. https://arxiv.org/abs/2202.06709.
15	JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 694-711.
16	RAD M S, BOZORGTABAR B, MARTI U V, et al. SROBB: targeted perceptual loss for single image super-resolution[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 2710-2719.
17	KHOSLA P, TETERWAK P, WANG C, et al. Supervised contrastive learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2020: 18661-18673.
18	CHOI H, SOM A, TURAGA P. AMC-Loss: angular margin contrastive loss for improved explainability in image classification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3659-3666.
19	XIE E Z, DING J, WANG W H, et al. DetCo: unsupervised contrastive learning for object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2022: 8372-8381.
20	SUN B, LI B H, CAI S C, et al. FSCE: few-shot object detection via contrastive proposal encoding[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 7348-7358.
21	WANG X L, ZHANG R F, SHEN C H, et al. Dense contrastive learning for self-supervised visual pre-training[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 3023-3032.
22	HE K M, FAN H Q, WU Y X, et al. Momentum contrast for unsupervised visual representation learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 9726-9735.
23	WU Z R, XIONG Y J, YU S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3733-3742.
24	WANG L G, WANG Y Q, DONG X Y, et al. Unsupervised degradation representation learning for blind super-resolution[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 10576-10585.
25	ZOU Y H, FU Y. Estimating fine-grained noise model via contrastive learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 12672-12681.
26	MO Y Z, LI C F, ZHENG Y H, et al. DCA-CycleGAN: unsupervised single image dehazing using dark channel attention optimized CycleGAN. Journal of Visual Communication and Image Representation, 2022, 82, 103431. doi: 10.1016/j.jvcir.2021.103431

[1]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[2]	李泽水, 冀俊忠, 杨翠翠. 基于边权重信息深度网络嵌入的PPIN功能模块检测[J]. 计算机工程, 2023, 49(8): 69-76.
[3]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[4]	卢昂, 储珺, 冷璐. 基于高低频特征增强的图像去雾[J]. 计算机工程, 2023, 49(8): 174-181.
[5]	闫兴亚, 匡娅茜, 白光睿, 李月. 基于深度学习的学生课堂行为识别方法[J]. 计算机工程, 2023, 49(7): 251-258.
[6]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[7]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[8]	席建锐, 唐红梅, 梁春阳, 刘鑫. 基于改进隐函数的点云物体重建[J]. 计算机工程, 2023, 49(7): 214-222.
[9]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[10]	张博旭, 蒲智, 程曦. 基于提示学习的维吾尔语文本分类研究[J]. 计算机工程, 2023, 49(6): 292-299,313.
[11]	于海洋, 景鹏, 张文涛, 谢赛飞, 滑志华, 宋草原. 基于残差与注意力机制的道路裂缝检测U-Net改进模型[J]. 计算机工程, 2023, 49(6): 265-273.
[12]	朱钦权, 王同罕, 贾惠珍. 基于雾密度感知增强的去雾图像质量评价方法[J]. 计算机工程, 2023, 49(6): 208-216,226.
[13]	谌雨章, 黄逸姿, 张钧涵. 基于多速率空洞卷积的多尺度水下小目标检测[J]. 计算机工程, 2023, 49(6): 257-264.
[14]	宋羽凯, 谢江. 基于多任务学习的轻量级语音情感识别模型[J]. 计算机工程, 2023, 49(5): 122-128.
[15]	王爱玲, 马文臻, 邹自明, 钟佳. 基于领域自适应的卫星工程参数异常检测[J]. 计算机工程, 2023, 49(5): 29-37,47.

选择文件类型/文献管理软件名称

选择包含的内容