Image Dehazing Based on High-Frequency and Low-Frequency Feature Enhancement

doi:10.19678/j.issn.1000-3428.0064826

Abstract

Abstract:

Image dehazing is a typical ill-posed problem.Dehazing network architectures commonly use encoder-decoder networks consisting of encoders, decoders, and feature converters that connect the two.The quality of dehazing images generated by existing dehazing algorithms is generally low, with problems such as incomplete local detail dehazing, color distortion, or the introduction of noise.Dehazing algorithms based on an encoder-decoder network do not fully utilize small-scale features when designing feature converters, and only use the corresponding layer coding features in the decoding stage; therefore, this paper proposes a dehazing algorithm based on high-frequency and low-frequency feature enhancement.In the feature transformation stage, an expanded residual component is designed and formed into a context aggregation network, which makes full use of the low-resolution features of the large receptive field, extracts the remote correlation of feature maps, and enhances low-frequency small-scale features.Furthermore, a multi-level feature reuse network based on channel attention is designed to reuse shallow high-frequency features while deeply fusing decoded and reconstructed features to enhance the recovery of visual perceptual features.In the coding stage, a Visual Feature Perception Module(VFPM) is constructed to enhance the rich high-frequency visual features of the shallow layer by leveraging the advantages of residual blocks in local modeling.Experimental results show that compared with AOD-Net, PFF-Net, and other dehazing algorithms, the Peak Signal-to-Noise Ratio(PSNR) and Structural Similarity(SSIM) of the proposed algorithm have significant advantages.The PSNR and SSIM of the proposed algorithm are 0.77 dB and 0.000 7, and 0.40 dB and 0.037 1 higher than those of the other algorithms, respectively, on an indoor synthetic dataset, SOTS, and outdoor real dataset, Dense-Haze.

Key words: image dehazing, encoder-decoder network, dilated residual, feature enhancement, channel attention

摘要：

图像去雾是典型的不适定问题，编解码网络是常用的去雾网络架构，编解码网络由编码器、解码器和连接两者的特征转换器构成。已有去雾算法生成的去雾图像通常质量较低，存在局部细节去雾不彻底、颜色失真或引入噪声等问题。针对基于编解码网络的去雾算法在设计特征转换器时没有充分利用小尺度特征、解码阶段仅利用对应层编码特征的问题，提出一种基于高低频特征增强的去雾算法。在特征转换阶段，设计扩张残差组件并组成上下文聚合网络，充分利用大感受野的低分辨率特征，提取特征图远距离相关性，增强低频小尺度特征。设计基于通道注意力的多级特征重用网络，实现浅层高频特征的重利用，同时深度融合解码重建特征，增强视觉感知特征的恢复。在编码阶段构建视觉特性感知模块，利用残差块在局部建模方面的优势增强浅层丰富的高频视觉特征。实验结果表明，与AOD-Net、PFF-Net等去雾算法相比，该算法的PSNR和SSIM指标均有明显优势，在室内合成数据集SOTS和室外真实数据集Dense-Haze上，所提算法的PSNR和SSIM分别高出性能次优算法0.77 dB、0.000 7和0.40 dB、0.037 1。

关键词: 图像去雾, 编解码网络, 扩张残差, 特征增强, 通道注意力

Ang LU, Jun CHU, Lu LENG. Image Dehazing Based on High-Frequency and Low-Frequency Feature Enhancement[J]. Computer Engineering, 2023, 49(8): 174-181.

卢昂, 储珺, 冷璐. 基于高低频特征增强的图像去雾[J]. 计算机工程, 2023, 49(8): 174-181.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0064826

http://www.ecice06.com/EN/Y2023/V49/I8/174

Figures/Tables 15

References 33

1	MCCARTNEY E J, HALL F. Optics of the atmosphere: scattering by molecules and particles[EB/OL]. [2022-04-05]. https://iopscience.iop.org/article/10.1088/0031-9112/28/11/025/pdf.
2	NARASIMHAN S G, NAYAR S K. Vision and the atmosphere. International Journal of Computer Vision, 2002, 48(3): 233- 254.
3	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 1956-1963.
4	HE K M, SUN J, TANG X O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397- 1409. doi: 10.1109/TPAMI.2012.213
5	ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24(11): 3522- 3533.
6	BERMAN D, TREIBITZ T, AVIDAN S. Non-local image dehazing[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 1674-1682.
7	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25(11): 5187- 5198. doi: 10.1109/TIP.2016.2598681
8	REN W, LIU S, ZHANG H, et al. Single image dehazing via multi-scale convolutional neural networks[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 154-169.
9	LI R D, PAN J S, LI Z C, et al. Single image dehazing via conditional generative adversarial network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8202-8211.
10	QU Y Y, CHEN Y Z, HUANG J Y, et al. Enhanced Pix2pix dehazing network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 8152-8160.
11	SHAO Y J, LI L, REN W Q, et al. Domain adaptation for image dehazing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 2805-2814.
12	ZHANG H, PATEL V M. Densely connected pyramid dehazing network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3194-3203.
13	PANG Y W, NIE J, XIE J, et al. BidNet: binocular image dehazing without explicit disparity estimation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 5930-5939.
14	何涛, 俞舒曼, 徐鹤. 基于条件生成对抗网络与知识蒸馏的单幅图像去雾方法. 计算机工程, 2022, 48(4): 165- 172. URL
	HE T, YU S M, XU H. Single image dehazing method based on conditional generative adversarial network and knowledge distillation. Computer Engineering, 2022, 48(4): 165- 172. URL
15	DUDHANE A, BIRADAR K M, PATIL P W, et al. Varicolored image de-hazing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 4563-4572.
16	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[EB/OL]. [2022-04-05]. https://arxiv.org/pdf/1505.04597.pdf.
17	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640- 651. doi: 10.1109/TPAMI.2016.2572683
18	LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: all-in-one dehazing network[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 4780-4788.
19	DONG J X, PAN J S. Physics-based feature dehazing networks[EB/OL]. [2022-04-05]. https://jspan.github.io/projects/pfdn/index.html.
20	KAR A, DHARA S K, SEN D, et al. Progressive update guided interdependent networks for single image dehazing[EB/OL]. [2022-04-05]. https://arxiv.org/abs/2008.01701.
21	REN W Q, MA L, ZHANG J W, et al. Gated fusion network for single image dehazing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3253-3261.
22	CHEN D D, HE M M, FAN Q N, et al. Gated context aggregation network for image dehazing and deraining[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2019: 1375-1383.
23	LIU X H, MA Y R, SHI Z H, et al. GridDehazeNet: attention-based multi-scale network for image dehazing[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 7313-7322.
24	DONG H, PAN J S, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 2154-2164.
25	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
26	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2022-04-05]. https://arxiv.org/abs/1511.07122.
27	MEI K F, JIANG A W, LI J C, et al. Progressive feature fusion network for realistic image dehazing[EB/OL]. [2022-04-05]. https://arxiv.org/pdf/1810.02283v1.pdf.
28	LI B Y, REN W Q, FU D P, et al. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 2019, 28(1): 492- 505.
29	ANCUTI C O, ANCUTI C, TIMOFTE R. NH-HAZE: an image dehazing benchmark with non-homogeneous hazy and haze-free images[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D. C., USA: IEEE Press, 2020: 1798-1805.
30	ANCUTI C O, ANCUTI C, SBERT M, et al. Dense-Haze: a benchmark for image dehazing with dense-haze and haze-free images[C]//Proceedings of IEEE International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2019: 1014-1018.
31	HONG M, XIE Y, LI C H, et al. Distilling image dehazing with heterogeneous task imitation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 3459-3468.
32	郭梦琰, 张娟, 刘巧红, 等. 基于循环生成对抗网络的图像去雾算法. 计算机工程, 2022, 48(3): 280- 287. URL
	GUO M Y, ZHANG J, LIU Q H, et al. Image dehazing algorithm based on recurrent generative adversarial network. Computer Engineering, 2022, 48(3): 280- 287. URL
33	ZHENG Z R, REN W Q, CAO X C, et al. Ultra-high-definition image dehazing via multi-guided bilateral learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 16180-16189.

模块	C-In	C-Out	Size-In	Size-Out
C1	3	16	W×H	W×H
DL1	16	32	W×H	W/2×H/2
VFPM1	32	32	W/2×H/2	W/2×H/2
DL2	32	64	W/2×H/2	W/4×H/4
VFPM2	64	64	W/4×H/4	W/4×H/4
DL3	64	128	W/4×H/4	W/8×H/8
VFPM3	128	128	W/8×H/8	W/8×H/8

模块	C-In	C-Out	Size-In	Size-Out
C1	3	16	W×H	W×H
DL1	16	32	W×H	W/2×H/2
VFPM1	32	32	W/2×H/2	W/2×H/2
DL2	32	64	W/2×H/2	W/4×H/4
VFPM2	64	64	W/4×H/4	W/4×H/4
DL3	64	128	W/4×H/4	W/8×H/8
VFPM3	128	128	W/8×H/8	W/8×H/8

卷积层	Kernal; r	C-In	C-Out
cv1	1×1;1	128	32
cv2	3×3;3	32	32
cv3	3×3;5	32	32
cv4	1×1;1	32	128

卷积层	Kernal; r	C-In	C-Out
cv1	1×1;1	128	32
cv2	3×3;3	32	32
cv3	3×3;5	32	32
cv4	1×1;1	32	128

卷积层	C-In	C-Out	Size-In	Size-Out
FD1	16	32	W×H	W/2×H/2
FC1	32	32	W/2×H/2	W/2×H/2
FU1	64	32	W/4×H/4	W/2×H/2

Please choose a citation manager

Content to export