水下图像增强复原对深度学习目标检测精度的影响研究

doi:10.19678/j.issn.1000-3428.0066610

计算机工程 ›› 2023, Vol. 49 ›› Issue (11): 247-256. doi: 10.19678/j.issn.1000-3428.0066610

水下图像增强复原对深度学习目标检测精度的影响研究

杨谢柳¹, 门国文¹, 梁文峰¹, 王丹¹, 谢正义¹, 范慧杰²^,*

1. 沈阳建筑大学机械工程学院, 沈阳 110168
2. 中国科学院沈阳自动化研究所机器人学国家重点实验室, 沈阳 110016

收稿日期:2022-12-26 出版日期:2023-11-15 发布日期:2023-03-03
通讯作者: 范慧杰
作者简介:
杨谢柳（1985—），女，副教授、博士，主研方向为计算机视觉、图像处理
门国文，硕士研究生
梁文峰，教授、博士
王丹，副教授、博士
谢正义，副教授
基金资助:
国家自然科学基金(61973224); 兴辽英才计划"青年拔尖人才"(XLYC2007186); 辽宁省教育厅面上项目(LJKZ0571); 道路施工技术与装备教育部重点实验室开放基金(300102259506)

Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection

Xieliu YANG¹, Guowen MEN¹, Wenfeng LIANG¹, Dan WANG¹, Zhengyi XIE¹, Huijie FAN²^,*

1. School of Mechanical Engineering, Shenyang Jianzhu University, Shenyang 110168, China
2. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China

Received:2022-12-26 Online:2023-11-15 Published:2023-03-03
Contact: Huijie FAN

摘要/Abstract

摘要：

因水下环境的特殊性，水下光学图像往往存在色偏、模糊、对比度低等退化现象。为恢复颜色正常、清晰的水下图像，大量的水下图像增强复原方法已被提出，但是现有的水下图像增强复原方法主要以提高水下图像的视觉效果为直接目标，对基于深度学习的水下目标检测精度的影响尚不明确。因此，使用14种典型的水下图像增强复原方法和3种典型的基于深度学习的目标检测模型，在URPC2018和URPC2019数据集上从训练集与测试集的域差异、训练集的域数量、训练集的图像数量等方面，详细深入地探讨图像增强复原方法对基于深度学习的目标检测模型精度的影响，并自建数据集进行跨数据集测试。实验结果表明，在训练集和测试集均属同一数据集时，水下图像增强复原方法无论作为图像预处理方法还是数据增强方法，对深度学习目标检测精度的提升都无明显效果，但是在跨数据集检测时，借助水下图像增强复原方法能够大幅提升深度学习目标检测精度，mAP最高可提高13.6个百分点。

关键词: 水下图像, 图像增强, 图像复原, 目标检测, 跨数据集

Abstract:

Due to the particularity of the underwater environment, underwater optical images often suffer from degradation issues such as color cast, blur, and low contrast. To restore these underwater images to their natural and clear colors, numerous methods for enhancing and restoring underwater images have been proposed. However, the existing underwater image enhancement restoration techniques primarily focus on improving the visual quality of underwater images. Their impact on the accuracy of underwater object detection using deep learning methods remains uncertain. Therefore, this study conducts a detailed and comprehensive exploration of the influence of fourteen typical underwater image enhancement and restoration methods and three common deep learning-based object detection models on the accuracy of deep learning-based object detection models. The analysis includes URPC2018 and URPC2019 datasets and considers factors such as domain variations between training and testing sets, the number of domains in the training set, the quantity of images in the training set, and self-created datasets for cross-dataset testing. The experimental results show that when both the training and test sets belong to the same dataset, underwater image enhancement and restoration methods, whether used as image preprocessing methods or data enhancement methods, do not significantly improve the detection accuracy of deep learning objects. However, when detecting across datasets, using underwater image enhancement and restoration methods can significantly enhance the detection accuracy of deep learning objects, with mAP increasing by up to 13.6 percentage points.

Key words: underwater image, image enhancement, image restoration, object detection, cross-dataset

杨谢柳, 门国文, 梁文峰, 王丹, 谢正义, 范慧杰. 水下图像增强复原对深度学习目标检测精度的影响研究[J]. 计算机工程, 2023, 49(11): 247-256.

Xieliu YANG, Guowen MEN, Wenfeng LIANG, Dan WANG, Zhengyi XIE, Huijie FAN. Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection[J]. Computer Engineering, 2023, 49(11): 247-256.

http://www.ecice06.com/CN/Y2023/V49/I11/247

图/表 18

图1 原图1的增强复原图像

Fig.1 Enhancement restoration images for original image 1

图2 原图2的增强复原图像

Fig.2 Enhancement restoration images for original image 2

图3 交叉域测试示意图

Fig.3 Schematic diagram of cross-domain test

图4 YOLOv5同域检测结果的对比

Fig.4 Comparison of YOLOv5 in the same-domain detection results

图5 跨数据集测试图像示例

Fig.5 Example of cross-dataset testing images

参考文献 34

1	ANCUTI C O, ANCUTI C, DE VLEESCHOUWER C, et al. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27(1): 379- 393. doi: 10.1109/TIP.2017.2759252
2	GUO Y C, LI H Y, ZHUANG P X. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862- 870. doi: 10.1109/JOE.2019.2911447
3	DAI C G, LIN M X, WU X J, et al. Single underwater image restoration by decomposing curves of attenuating color. Optics & Laser Technology, 2020, 123, 1- 10.
4	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341- 2353. doi: 10.1109/TPAMI.2010.168
5	GALDRAN A, PARDO D, PICÓN A, et al. Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 2015, 26, 132- 145. doi: 10.1016/j.jvcir.2014.11.006
6	DREWS JR P, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]//Proceedings of International Conference on Computer Vision Workshops. Washington D. C., USA: IEEE Press, 2014: 825-830.
7	SONG W, WANG Y, HUANG D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]//Proceedings of the 19th Pacific Rim Conference on Multimedia. Berlin, Germany: Springer, 2018: 678-688.
8	PENG Y T, ZHAO X Y, COSMAN P C. Single underwater image enhancement using depth estimation based on blurriness[C]//Proceedings of International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2015: 4952-4956.
9	CARLEVARIS-BIANCO N, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing[C]//Proceedings of OCEANS 2010 MTS/IEEE SEATTLE. Washington D. C., USA: IEEE Press, 2010: 1-8.
10	PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing, 2017, 26(4): 1579- 1594. doi: 10.1109/TIP.2017.2663846
11	WANG K Y, HU Y, CHEN J, et al. Underwater image restoration based on a parallel convolutional neural network. Remote Sensing, 2019, 11(13): 1591. doi: 10.3390/rs11131591
12	BARBOSA W V, AMARAL H G B, ROCHA T L, et al. Visual-quality-driven learning for underwater vision enhancement[C]//Proceedings of the 25th International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2018: 3933-3937.
13	张清博, 张晓晖, 韩宏伟. 一种基于深度卷积神经网络的水下光电图像质量优化方法. 光学学报, 2018, 38(11): 96- 104. URL
	ZHANG Q B, ZHANG X H, HAN H W. Optimization of underwater photoelectric image quality based on deep convolutional neural networks. Acta Optica Sinica, 2018, 38(11): 96- 104. URL
14	EMBERTON S, CHITTKA L, CAVALLARO A. Underwater image and video dehazing with pure haze region segmentation. Computer Vision and Image Understanding, 2018, 168, 145- 156. doi: 10.1016/j.cviu.2017.08.003
15	YANG X L, YIN C Y, ZHANG Z Y, et al. Robust chromatic adaptation based color correction technology for underwater images. Applied Sciences, 2020, 10(18): 6392. doi: 10.3390/app10186392
16	IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method[C]//Proceedings of International Conference on Systems, Man and Cybernetics. Washington D. C., USA: IEEE Press, 2010: 1703-1709.
17	ABDUL GHANI A S, MAT ISA N A. Automatic system for improving underwater image contrast and color through recursive adaptive histogram modification. Computers and Electronics in Agriculture, 2017, 141, 181- 195. doi: 10.1016/j.compag.2017.07.021
18	王丹, 张子玉, 赵金宝, 等. 基于常散射假设和同态滤波的水下图像增强算法. 水下无人系统学报, 2021, 29(2): 210- 217. URL
	WANG D, ZHANG Z Y, ZHAO J B, et al. Underwater image enhancement algorithm based on constant scattering hypothesis and homomorphic filtering. Journal of Unmanned Undersea Systems, 2021, 29(2): 210- 217. URL
19	KHAN A, ALI S S A, MALIK A S, et al. Underwater image enhancement by wavelet based fusion[C]//Proceedings of International Conference on Underwater System Technology: Theory and Applications. Washington D. C., USA: IEEE Press, 2017: 83-88.
20	LI C Y, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognition, 2020, 98, 107038. doi: 10.1016/j.patcog.2019.107038
21	LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2020, 29, 4376- 4389. doi: 10.1109/TIP.2019.2955241
22	LI J, SKINNER K A, EUSTICE R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 2018, 3(1): 387- 394.
23	ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 2020, 5(2): 3227- 3234. doi: 10.1109/LRA.2020.2974710
24	UPLAVIKAR P, WU Z Y, WANG Z Y. All-in-one underwater image enhancement using domain-adversarial learning[EB/OL]. [2022-11-15]. https://arxiv.org/abs/1905.13342v1.
25	史朋飞, 韩松, 倪建军, 等. 结合数据增强和改进YOLOv4的水下目标检测算法. 电子测量与仪器学报, 2022, 36(3): 113- 121. URL
	SHI P F, HAN S, NI J J, et al. Underwater object detection algorithm combining dataenhancement and improved YOLOv4. Journal of Electronic Measurement and Instrumentation, 2022, 36(3): 113- 121. URL
26	HAN F L, YAO J Z, ZHU H T, et al. Marine organism detection and classification from underwater vision based on the deep CNN method. Mathematical Problems in Engineering, 2020, 2020, 1- 11.
27	YANG H B, LIU P, HU Y Z, et al. Research on underwater object recognition based on YOLOv3. Microsystem Technologies, 2021, 27(4): 1837- 1844.
28	CHEN W, FAN B J. Underwater object detection with mixed attention mechanism and multi-enhancement strategy[C]//Proceedings of Chinese Automation Congress. Washington D. C., USA: IEEE Press, 2021: 2821-2826.
29	CHEN X Y, LU Y, WU Z X, et al. Reveal of domain effect: how visual restoration contributes to object detection in aquatic scenes[EB/OL]. [2022-11-15]. https://arxiv.org/abs/2003.01913v1.
30	PENG X Image restoration contributes to underwater object detection[C]//Proceedings of International Conference on Consumer Electronics-Taiwan. Washington D. C., USA: IEEE Press, 2021: 1-2.
31	LI C Y, QUO J C, PANG Y W, et al. Single underwater image restoration by blue-green channels dehazing and red channel correction[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Washington D. C., USA: IEEE Press, 2016: 1731-1735.
32	IQBAL K, SALAM R A, OSMAN A, et al. Underwater image enhancement using an integrated colour model. IAENG International Journal of Computer Science, 2007, 34(2): 1- 10.
33	HUMMEL R. Image enhancement by histogram transformation. Computer Graphics and Image Processing, 1977, 6(2): 184- 195.
34	ZUIDERVELD K. Contrast limited adaptive histogram equalization. Graphics Gems, 1994, 474- 485.

[1]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[2]	龙玉江, 卫薇, 舒彧, 张正刚, 王道累, 李峰. 基于自适应关键点的破损旋转绝缘子检测方法[J]. 计算机工程, 2023, 49(9): 272-278.
[3]	徐春波, 闫娟, 杨慧斌, 王博, 吴晗. 基于目标检测和语义分割的视觉SLAM算法[J]. 计算机工程, 2023, 49(8): 199-206, 214.
[4]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[5]	王书朋, 何引弟. 融合特征注意力机制的非均匀光照图像增强算法[J]. 计算机工程, 2023, 49(8): 232-239.
[6]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[7]	李强龙, 周新文, 位梦恩, 甘阳洲. 基于条形池化和注意力机制的街道场景红外目标检测算法[J]. 计算机工程, 2023, 49(8): 310-320.
[8]	闫兴亚, 匡娅茜, 白光睿, 李月. 基于深度学习的学生课堂行为识别方法[J]. 计算机工程, 2023, 49(7): 251-258.
[9]	聂志勇, 阴宇薇, 汤佳欣, 涂志刚. 一种基于边界框关键点距离的框回归算法[J]. 计算机工程, 2023, 49(7): 65-75.
[10]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[11]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[12]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[13]	谌雨章, 黄逸姿, 张钧涵. 基于多速率空洞卷积的多尺度水下小目标检测[J]. 计算机工程, 2023, 49(6): 257-264.
[14]	常戬, 刘鑫姝. 空间转换与自适应灰度校正的低照度图像增强[J]. 计算机工程, 2023, 49(6): 193-200,207.
[15]	罗华峰, 沈奕菲, 阮黎翔, 杜奇伟, 郑翔, 陈智麒, 张胜. 边缘环境下面向实时目标检测的帧卸载调度算法[J]. 计算机工程, 2023, 49(5): 295-301,309.

选择文件类型/文献管理软件名称

选择包含的内容

水下图像增强复原对深度学习目标检测精度的影响研究

Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

水下图像增强复原对深度学习目标检测精度的影响研究

Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价