基于U-Net的多尺度低照度图像增强网络

doi:10.19678/j.issn.1000-3428.0062333

摘要/Abstract

摘要： 低照度是夜晚拍摄时常见的一种现象，不充分的光照会使图像细节损失严重，降低图像视觉质量。针对现有低照度图像增强方法对不同尺度特征的感知和表达能力存在不足的问题，提出一种基于U-Net的多尺度低照度图像增强网络（MSU-LIIEN）。采用特征金字塔作为基本处理框架，实现对低照度图像的特征提取。在特征金字塔构建的3个分支结构中均使用U-Net作为骨干网，对提取到的浅层图像特征进行编码与解码操作，同时引入结构细节残差融合块以增强网络模型提取和表征低照度图像特征信息的能力。在此基础上，对提取到的特征信息逐层融合，恢复正常光照图像。实验结果表明，MSU-LIIEN在LOL-datasets和Brighting Train数据集中相比于性能排名第二的KinD模型，平均峰值信噪比分别提高16.21%和46.67%，且在主观视野感受和客观评价指标方面均优于所有对比的经典模型，不但能有效提升低照度图像的整体亮度，而且能很好地保持图像中的细节信息和清晰的物体边缘轮廓，使增强后的图像整体画面真实自然。

关键词: 低照度图像增强, 深度学习, U-Net网络, 多尺度特征图, 感受野

Abstract: Low light is a common phenomenon when shooting at night.Insufficient illumination causes serious loss of image details and reduces visual quality.The existing low-light image enhancement methods have insufficient perception and expression of features at different scales.To address the problem of existing low-light image enhancement methods being inadequate in their ability to perceive and express features at different scales, a multi-scale low-light image enhancement network based on U-Net(MSU-LIIEN) is proposed.Firstly, the feature pyramid is used as the basic processing framework of this article to achieve feature extraction for low-light images.Then, the U-Net is used as the backbone in all three branch structures of the feature pyramid construction to encode and decode the extracted shallow image features, while structural detail residual fusion blocks are introduced to enhance the network's ability to extract and characterize low-light image feature information.Finally, the extracted feature information is fused layer by layer to recover the final image.The experimental results show that, compared with the second-performing KinD algorithm in LOL-datasets, the average Peak Signal-to-Noise Ratio(PSNR) value increased by 16.21%, and compared with the second-performing model on the Brighting Train dataset, the average PSNR value increased by 49.67%.The proposed algorithm outperforms other classical low-light image enhancement algorithms in terms of both subjective visual field perception and objective evaluation metrics.Not only does it effectively enhance the overall brightness of low-light images, but it also maintains detailed information in the image and clear object outlines, making the overall picture of the enhanced image realistic and natural.

Key words: low-light image enhancement, deep learning, U-Net, multi-scale feature map, receptive field

中图分类号:

TP391.4

徐超越, 余映, 何鹏浩, 李淼, 马玉辉. 基于U-Net的多尺度低照度图像增强网络[J]. 计算机工程, 2022, 48(8): 215-223.

XU Chaoyue, YU Ying, HE Penghao, LI Miao, MA Yuhui. Multi-Scale Low-Light Image Enhancement Network Based on U-Net[J]. Computer Engineering, 2022, 48(8): 215-223.

http://www.ecice06.com/CN/Y2022/V48/I8/215

图/表 18

20220825092950

20220825092953

20220825092957

20220825093001

20220825093005

20220825093009

20220825093012

20220825093016

20220825093020

20220825093024

20220825093027

20220825093032

20220825093036

20220825093039

20220825093043

20220825093047

20220825093051

20220825093055

参考文献

[1] YUN S H, KIM J H, KIM S.Image enhancement using a fusion framework of histogram equalization and Laplacian pyramid[J].IEEE Transactions on Consumer Electronics, 2010, 56(4):2763-2771.
[2] SENTHILKUMARAN N, THIMMIARAJA J.Histogram equalization for image enhancement using MRI brain images[C]//Proceedings of World Congress on Computing and Communication Technologies.Washington D.C., USA:IEEE Press, 2014:80-83.
[3] SURESHA M, RAGHUKUMAR D S, KUPPA S.Kumaraswamy distribution based Bi-histogram equalization for enhancement of microscopic images[J].International Journal of Image and Graphics, 2022, 22(1):1-10.
[4] SINGH P, MUKUNDAN R, DE RYKE R.Feature enhancement in medical ultrasound videos using contrast-limited adaptive histogram equalization[J].Journal of Digital Imaging, 2020, 33(1):273-285.
[5] YE H M, SU K Q, HUANG S M.Image enhancement method based on bilinear interpolating and wavelet transform[C]//Proceedings of the 5th Advanced Information Technology, Electronic and Automation Control Conference.Washington D.C., USA:IEEE Press, 2021:1147-1150.
[6] KAUR K, JINDAL N, SINGH K.Fractional Fourier transform based Riesz fractional derivative approach for edge detection and its application in image enhancement[J].Signal Processing, 2021, 180:1-10.
[7] YANG M X, TANG G J, LIU X H, et al.Low-light image enhancement based on Retinex theory and dual-tree complex wavelet transform[J].Optoelectronics Letters, 2018, 14(6):470-475.
[8] RAHMAN Z, JOBSON D J, WOODELL G A.Multi-scale Retinex for color image enhancement[C]//Proceedings of the 3rd IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 1996:1003-1006.
[9] FU X Y, ZENG D L, HUANG Y, et al.A weighted variational model for simultaneous reflectance and illumination estimation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:2782-2790.
[10] FU X Y, ZENG D L, HUANG Y, et al.A fusion-based enhancing method for weakly illuminated images[J].Signal Processing, 2016, 129:82-96.
[11] 张聿, 刘世光.基于分区曝光融合的不均匀亮度视频增强[J].计算机辅助设计与图形学学报, 2017, 29(12):2317-2322. ZHANG Y, LIU S G.Non-uniform illumination video enhancement based on zone system and fusion[J].Journal of Computer-Aided Design &Computer Graphics, 2017, 29(12):2317-2322.(in Chinese)
[12] DONG X, WANG G, PANG Y, et al.Fast efficient algorithm for enhancement of low lighting video[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C., USA:IEEE Press, 2011:1-6.
[13] WANG S H, ZHENG J, HU H M, et al.Naturalness preserved enhancement algorithm for non-uniform illumination images[J].IEEE Transactions on Image Processing, 2013, 22(9):3538-3548.
[14] YING Z Q, LI G, GAO W.A bio-inspired multi-exposure fusion framework for low-light image enhancement[EB/OL].[2021-02-10].https://arxiv.org/abs/1711.00591.
[15] LORE K G, AKINTAYO A, SARKAR S.LLNet:a deep autoencoder approach to natural low-light image enhancement[J].Pattern Recognition, 2017, 61:650-662.
[16] 江泽涛, 伍旭, 张少钦.一种基于MR-VAE的低照度图像增强方法[J].计算机学报, 2020, 43(7):1328-1339. JIANG Z T, WU X, ZHANG S Q.Low-illumination image enhancement based on MR-VAE[J].Chinese Journal of Computers, 2020, 43(7):1328-1339.(in Chinese)
[17] WEI C, WANG W J, YANG W H, et al.Deep Retinex ecomposition for low-light enhancement[EB/OL].[2021-02-10].https://arxiv.org/abs/1808.04560.
[18] LÜF F, LU F, WU J H, et al.MBLLEN:low-light image/video enhancement using CNNs[C]//Proceedings of BMVC'18.Washington D.C., USA:IEEE Press, 2018:220.
[19] LIU S G, ZHANG Y.Detail-preserving underexposed image enhancement via optimal weighted multi-exposure fusion[J].IEEE Transactions on Consumer Electronics, 2019, 65(3):303-311.
[20] 马红强, 马时平, 许悦雷, 等.基于深度卷积神经网络的低照度图像增强[J].光学学报, 2019, 39(2):91-100. MA H Q, MA S P, XU Y L, et al.Low-light image enhancement based on deep convolutional neural network[J].Acta Optica Sinica, 2019, 39(2):91-100.(in Chinese)
[21] ZHANG Y H, ZHANG J W, GUO X J.Kindling the darkness:a practical low-light image enhancer[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York, USA:ACM Press, 2019:1632-1640.
[22] ZHANG Y H, GUO X J, MA J Y, et al.Beyond brightening low-light images[J].International Journal of Computer Vision, 2021, 129(4):1013-1037.
[23] LIU Y J, WANG Z N, ZENG Y, et al.PD-GAN:perceptual-details GAN for extremely noisy low light image enhancement[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing.Washington D.C., USA:IEEE Press, 2021:1840-1844.
[24] 陈榆琅, 高晶敏, 张科备, 等.基于生成对抗网络的空间卫星低照度图像增强[J].中国空间科学技术, 2021, 41(3):16-23. CHEN Y L, GAO J M, ZHANG K B, et al.Low-light image enhancement of space satellites based on GAN[J].Chinese Space Science and Technology, 2021, 41(3):16-23.(in Chinese)
[25] ZHU A Q, ZHANG L, SHEN Y, et al.Zero-shot restoration of underexposed images via robust retinex decomposition[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C., USA:IEEE Press, 2020:1-6.
[26] XU Y D, YANG C, SUN B B, et al.A novel multi-scale fusion framework for detail-preserving low-light image enhancement[J].Information Sciences, 2021, 548:378-397.
[27] LIN T Y, DOLLÁR P, GIRSHICK R, et al.Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:936-944.
[28] LU K, ZHANG L H.TBEFN:a two-branch exposure-fusion network for low-light image enhancement[J].IEEE Transactions on Multimedia, 2021, 23:4093-4105.

选择文件类型/文献管理软件名称

选择包含的内容