Single Image Dehazing Network Based on Double Scale Feature Fusion

doi:10.19678/j.issn.1000-3428.0062925

Abstract

Abstract: Learning-based dehazing methods perform well with a synthetic dataset but have certain problems, such as residual haze and color distortion, in a real scene.To solve these problems, a single-image dehazing network based on double-scale feature fusion is proposed in this paper.The proposed network consists of two parts:feature extraction and multiscale feature fusion.In the feature extraction stage, the local and global features of the input image are extracted through the residual dense and feature extraction blocks, respectively, using a spatial attention mechanism.In the feature fusion stage, the local and global feature maps are channel weighted using the channel attention mechanism and fused through a convolution operation.Finally, the gated network adaptively combines three feature maps from different depths to restore the high-quality haze-free images.The experimental results show that the Peak Signal-to-Noise Ratio (PSNR) of the proposed network under indoor dataset is 33.04 dB, and the Structural Similarity (SSIM) is 0.983.And the PSNR and SSIM are 1.33 dB and 0.041 higher than that of the GirdDehazeNet network under HAZERD dataset, respectively.In addition, there are 0.34M model parameters, and the Floating Point Operations (FLOPs) of the network occur at a rate of 16.06×10⁹ frames/s, it shows that the proposed network achieves ideal dehazing results for both synthetic and real images with low complexity.

Key words: deep learning, image dehazing, attention mechanism, feature fusion, gated network

摘要： 基于深度学习的图像去雾方法在合成数据集上表现良好，但在真实场景中应用时存在去雾不彻底、颜色失真等问题。提出一种新的单幅图像去雾网络，该网络包含特征提取、特征融合2个模块。在特征提取模块中，通过残差密集块和具有空间注意机制的特征提取块分别提取图像的局部特征和全局特征。在特征融合模块中，利用通道注意力机制对局部特征图和全局特征图进行通道加权，并通过卷积操作融合加权后的局部特征图与全局特征图。最后，采用门控网络自适应结合3个不同深度的融合特征图，以恢复高质量的去雾图像。实验结果表明，所提网络在室内数据集下的峰值信噪比（PSNR）和结构相似度（SSIM）分别为33.04 dB、0.983，在HAZERD数据集下的PSNR和SSIM分别比GridDehazeNet网络高出1.33 dB和0.041。同时，该网络的模型参数量和浮点运算数分别为0.34M和16.06×10⁹ frame/s，具有较低复杂度，对合成图像和真实图像均可取得理想的去雾效果。

关键词: 深度学习, 图像去雾, 注意机制, 特征融合, 门控网络

CLC Number:

TP391.41

LAN Yunwei, CUI Zhigao, SU Yanzhao, WANG Bo, WANG Nian, LI Aihua. Single Image Dehazing Network Based on Double Scale Feature Fusion[J]. Computer Engineering, 2022, 48(11): 231-239.

兰云伟, 崔智高, 苏延召, 汪波, 王念, 李艾华. 基于双尺度特征融合的单幅图像去雾网络[J]. 计算机工程, 2022, 48(11): 231-239.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0062925

http://www.ecice06.com/EN/Y2022/V48/I11/231

Figures/Tables 10

References

[1] 蒲亨飞, 黄治勇.图像去雾算法研究综述[J].软件工程, 2021, 24(5):2-6. PU H F, HUANG Z Y.An overview of research on image dehazing algorithms[J].Software Engineering, 2021, 24(5):2-6.(in Chinese)
[2] 王道累, 张天宇.图像去雾算法的综述及分析[J].图学学报, 2020, 41(6):861-870. WANG D L, ZHANG T Y.Review and analysis of image defogging algorithm[J].Journal of Graphics, 2020, 41(6):861-870.(in Chinese)
[3] BERMAN D, TREIBITZ T, AVIDAN S.Non-local image dehazing[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1674-1682.
[4] 陈子妍, 龙道银, 王霄, 等.基于逆通道与改进引导滤波的暗通道去雾算法[J].计算机工程, 2021, 47(6):245-252. CHEN Z Y, LONG D Y, WANG X, et al.Dark channel defogging algorithm based on inverse channel and improved guided filtering[J].Computer Engineering, 2021, 47(6):245-252.(in Chinese)
[5] 丁洁, 陆安江, 彭熙舜.结合暗通道和Retinex理论的图像去雾算法[J].激光杂志, 2021, 42(12):102-107. DING J, LU A J, PENG X S.Image dehazing algorithm combining dark channel and Retinex theory[J].Laser Journal, 2021, 42(12):102-107.(in Chinese)
[6] 杨燕, 梁小珍.基于光幕约束和分段调整的去雾算法[J].光子学报, 2021, 50(5):249-258. YANG Y, LIANG X Z.Dehazing algorithm based on atmospheric veil constraint and piecewise adjustment[J].Acta Photonica Sinica, 2021, 50(5):249-258.(in Chinese)
[7] 高涛, 刘梦尼, 陈婷, 等.结合暗亮通道先验的远近景融合去雾算法[J].西安交通大学学报, 2021, 55(10):78-86. GAO T, LIU M N, CHEN T, et al.A far and near scene fusion defogging algorithm based on the prior of dark-light channel[J].Journal of Xi'an Jiaotong University, 2021, 55(10):78-86.(in Chinese)
[8] 杨燕, 张金龙.基于雾度分布与自适应衰减的图像去雾算法[J].激光与光电子学进展, 2021, 58(22):210-219. YANG Y, ZHANG J L.Image dehazing algorithm based on haze distribution and adaptive attenuation[J].Laser & Optoelectronics Progress, 2021, 58(22):210-219.(in Chinese)
[9] REN W Q, LIU S, ZHANG H, et al.Single image dehazing via multi-scale convolutional neural networks[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:154-169.
[10] CAI B L, XU X M, JIA K, et al.DehazeNet:an end-to-end system for single image haze removal[J].IEEE Transactions on Image Processing, 2016, 25(11):5187-5198.
[11] LI B Y, PENG X L, WANG Z Y, et al.AOD-net:all-in-one dehazing network[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:4780-4788.
[12] ZHANG H, PATEL V M.Densely connected pyramid dehazing network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:3194-3203.
[13] ZHANG H, SINDAGI V, PATEL V M.Multi-scale single image dehazing using perceptual pyramid deep network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop.Washington D.C., USA:IEEE Press, 2018:1015-101509.
[14] 张婷, 赵杏, 陈文欣.基于条件生成对抗网络的图像去雾方法[J].计算机应用, 2021, 41(S2):248-253. ZHANG T, ZHAO X, CHEN W X.Image dehazing method based on conditional generative adversarial network[J].Journal of Computer Applications, 2021, 41(S2):248-253.(in Chinese)
[15] 刘杰平, 杨业长, 陈敏园, 等.结合卷积神经网络与动态环境光的图像去雾算法[J].光学学报, 2019, 39(11):120-131. LIU J P, YANG Y Z, CHEN M Y, et al.Image dehazing algorithm based on convolutional neural network and dynamic ambient light[J].Acta Optica Sinica, 2019, 39(11):120-131.(in Chinese)
[16] 杨爱萍, 刘瑾, 邢金娜, 等.基于内容特征和风格特征融合的单幅图像去雾网络[J].自动化学报, 2020, 46(7):1-9. YANG A P, LIU J, XING J N, et al.Content feature and style feature fusion network for single image dehazing[J].Acta Automatica Sinica, 2020, 46(7):1-9.(in Chinese)
[17] REN W Q, MA L, ZHANG J W, et al.Gated fusion network for single image dehazing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:3253-3261.
[18] QU Y Y, CHEN Y Z, HUANG J Y, et al.Enhanced Pix2pix dehazing network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:8152-8160.
[19] LIU X H, MA Y R, SHI Z H, et al.GridDehazeNet:attention-based multi-scale network for image dehazing[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:7313-7322.
[20] DONG H, PAN J S, XIANG L, et al.Multi-scale boosted dehazing network with dense feature fusion[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:2154-2164.
[21] SHAO Y J, LI L, REN W Q, et al.Domain adaptation for image dehazing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:2805-2814.
[22] CHEN D D, HE M M, FAN Q N, et al.Gated context aggregation network for image dehazing and deraining[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision.Washington D.C., USA:IEEE Press, 2019:1375-1383.
[23] LI B Y, REN W Q, FU D P, et al.Benchmarking single-image dehazing and beyond[J].IEEE Transactions on Image Processing, 2019, 28(1):492-505.
[24] ZHANG Y F, DING L, SHARMA G.HazeRD:an outdoor scene dataset and benchmark for single image dehazing[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2017:3205-3209.

Please choose a citation manager

Content to export