水下图像增强复原对深度学习目标检测精度的影响研究

doi:10.19678/j.issn.1000-3428.0066610

计算机工程 ›› 2023, Vol. 49 ›› Issue (11): 247-256. doi: 10.19678/j.issn.1000-3428.0066610

水下图像增强复原对深度学习目标检测精度的影响研究

杨谢柳¹, 门国文¹, 梁文峰¹, 王丹¹, 谢正义¹, 范慧杰²^,*

1. 沈阳建筑大学机械工程学院, 沈阳 110168
2. 中国科学院沈阳自动化研究所机器人学国家重点实验室, 沈阳 110016

收稿日期:2022-12-26 出版日期:2023-11-15 发布日期:2023-03-03
通讯作者: 范慧杰
作者简介:
杨谢柳（1985—），女，副教授、博士，主研方向为计算机视觉、图像处理
门国文，硕士研究生
梁文峰，教授、博士
王丹，副教授、博士
谢正义，副教授
基金资助:
国家自然科学基金(61973224); 兴辽英才计划"青年拔尖人才"(XLYC2007186); 辽宁省教育厅面上项目(LJKZ0571); 道路施工技术与装备教育部重点实验室开放基金(300102259506)

Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection

Xieliu YANG¹, Guowen MEN¹, Wenfeng LIANG¹, Dan WANG¹, Zhengyi XIE¹, Huijie FAN²^,*

1. School of Mechanical Engineering, Shenyang Jianzhu University, Shenyang 110168, China
2. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China

Received:2022-12-26 Online:2023-11-15 Published:2023-03-03
Contact: Huijie FAN

摘要/Abstract

摘要：

因水下环境的特殊性，水下光学图像往往存在色偏、模糊、对比度低等退化现象。为恢复颜色正常、清晰的水下图像，大量的水下图像增强复原方法已被提出，但是现有的水下图像增强复原方法主要以提高水下图像的视觉效果为直接目标，对基于深度学习的水下目标检测精度的影响尚不明确。因此，使用14种典型的水下图像增强复原方法和3种典型的基于深度学习的目标检测模型，在URPC2018和URPC2019数据集上从训练集与测试集的域差异、训练集的域数量、训练集的图像数量等方面，详细深入地探讨图像增强复原方法对基于深度学习的目标检测模型精度的影响，并自建数据集进行跨数据集测试。实验结果表明，在训练集和测试集均属同一数据集时，水下图像增强复原方法无论作为图像预处理方法还是数据增强方法，对深度学习目标检测精度的提升都无明显效果，但是在跨数据集检测时，借助水下图像增强复原方法能够大幅提升深度学习目标检测精度，mAP最高可提高13.6个百分点。

关键词: 水下图像, 图像增强, 图像复原, 目标检测, 跨数据集

Abstract:

Due to the particularity of the underwater environment, underwater optical images often suffer from degradation issues such as color cast, blur, and low contrast. To restore these underwater images to their natural and clear colors, numerous methods for enhancing and restoring underwater images have been proposed. However, the existing underwater image enhancement restoration techniques primarily focus on improving the visual quality of underwater images. Their impact on the accuracy of underwater object detection using deep learning methods remains uncertain. Therefore, this study conducts a detailed and comprehensive exploration of the influence of fourteen typical underwater image enhancement and restoration methods and three common deep learning-based object detection models on the accuracy of deep learning-based object detection models. The analysis includes URPC2018 and URPC2019 datasets and considers factors such as domain variations between training and testing sets, the number of domains in the training set, the quantity of images in the training set, and self-created datasets for cross-dataset testing. The experimental results show that when both the training and test sets belong to the same dataset, underwater image enhancement and restoration methods, whether used as image preprocessing methods or data enhancement methods, do not significantly improve the detection accuracy of deep learning objects. However, when detecting across datasets, using underwater image enhancement and restoration methods can significantly enhance the detection accuracy of deep learning objects, with mAP increasing by up to 13.6 percentage points.

Key words: underwater image, image enhancement, image restoration, object detection, cross-dataset

杨谢柳, 门国文, 梁文峰, 王丹, 谢正义, 范慧杰. 水下图像增强复原对深度学习目标检测精度的影响研究[J]. 计算机工程, 2023, 49(11): 247-256.

Xieliu YANG, Guowen MEN, Wenfeng LIANG, Dan WANG, Zhengyi XIE, Huijie FAN. Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection[J]. Computer Engineering, 2023, 49(11): 247-256.

https://www.ecice06.com/CN/Y2023/V49/I11/247

图/表 18

图1 原图1的增强复原图像

Fig.1 Enhancement restoration images for original image 1

图2 原图2的增强复原图像

Fig.2 Enhancement restoration images for original image 2

图3 交叉域测试示意图

Fig.3 Schematic diagram of cross-domain test

图4 YOLOv5同域检测结果的对比

Fig.4 Comparison of YOLOv5 in the same-domain detection results

图5 跨数据集测试图像示例

Fig.5 Example of cross-dataset testing images

参考文献 34

1	ANCUTI C O, ANCUTI C, DE VLEESCHOUWER C, et al. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27(1): 379- 393. doi: 10.1109/TIP.2017.2759252
2	GUO Y C, LI H Y, ZHUANG P X. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862- 870. doi: 10.1109/JOE.2019.2911447
3	DAI C G, LIN M X, WU X J, et al. Single underwater image restoration by decomposing curves of attenuating color. Optics & Laser Technology, 2020, 123, 1- 10.
4	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341- 2353. doi: 10.1109/TPAMI.2010.168
5	GALDRAN A, PARDO D, PICÓN A, et al. Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 2015, 26, 132- 145. doi: 10.1016/j.jvcir.2014.11.006
6	DREWS JR P, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]//Proceedings of International Conference on Computer Vision Workshops. Washington D. C., USA: IEEE Press, 2014: 825-830.
7	SONG W, WANG Y, HUANG D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]//Proceedings of the 19th Pacific Rim Conference on Multimedia. Berlin, Germany: Springer, 2018: 678-688.
8	PENG Y T, ZHAO X Y, COSMAN P C. Single underwater image enhancement using depth estimation based on blurriness[C]//Proceedings of International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2015: 4952-4956.
9	CARLEVARIS-BIANCO N, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing[C]//Proceedings of OCEANS 2010 MTS/IEEE SEATTLE. Washington D. C., USA: IEEE Press, 2010: 1-8.
10	PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing, 2017, 26(4): 1579- 1594. doi: 10.1109/TIP.2017.2663846
11	WANG K Y, HU Y, CHEN J, et al. Underwater image restoration based on a parallel convolutional neural network. Remote Sensing, 2019, 11(13): 1591. doi: 10.3390/rs11131591
12	BARBOSA W V, AMARAL H G B, ROCHA T L, et al. Visual-quality-driven learning for underwater vision enhancement[C]//Proceedings of the 25th International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2018: 3933-3937.
13	张清博, 张晓晖, 韩宏伟. 一种基于深度卷积神经网络的水下光电图像质量优化方法. 光学学报, 2018, 38(11): 96- 104. URL
	ZHANG Q B, ZHANG X H, HAN H W. Optimization of underwater photoelectric image quality based on deep convolutional neural networks. Acta Optica Sinica, 2018, 38(11): 96- 104. URL
14	EMBERTON S, CHITTKA L, CAVALLARO A. Underwater image and video dehazing with pure haze region segmentation. Computer Vision and Image Understanding, 2018, 168, 145- 156. doi: 10.1016/j.cviu.2017.08.003
15	YANG X L, YIN C Y, ZHANG Z Y, et al. Robust chromatic adaptation based color correction technology for underwater images. Applied Sciences, 2020, 10(18): 6392. doi: 10.3390/app10186392
16	IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method[C]//Proceedings of International Conference on Systems, Man and Cybernetics. Washington D. C., USA: IEEE Press, 2010: 1703-1709.
17	ABDUL GHANI A S, MAT ISA N A. Automatic system for improving underwater image contrast and color through recursive adaptive histogram modification. Computers and Electronics in Agriculture, 2017, 141, 181- 195. doi: 10.1016/j.compag.2017.07.021
18	王丹, 张子玉, 赵金宝, 等. 基于常散射假设和同态滤波的水下图像增强算法. 水下无人系统学报, 2021, 29(2): 210- 217. URL
	WANG D, ZHANG Z Y, ZHAO J B, et al. Underwater image enhancement algorithm based on constant scattering hypothesis and homomorphic filtering. Journal of Unmanned Undersea Systems, 2021, 29(2): 210- 217. URL
19	KHAN A, ALI S S A, MALIK A S, et al. Underwater image enhancement by wavelet based fusion[C]//Proceedings of International Conference on Underwater System Technology: Theory and Applications. Washington D. C., USA: IEEE Press, 2017: 83-88.
20	LI C Y, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognition, 2020, 98, 107038. doi: 10.1016/j.patcog.2019.107038
21	LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2020, 29, 4376- 4389. doi: 10.1109/TIP.2019.2955241
22	LI J, SKINNER K A, EUSTICE R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 2018, 3(1): 387- 394.
23	ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 2020, 5(2): 3227- 3234. doi: 10.1109/LRA.2020.2974710
24	UPLAVIKAR P, WU Z Y, WANG Z Y. All-in-one underwater image enhancement using domain-adversarial learning[EB/OL]. [2022-11-15]. https://arxiv.org/abs/1905.13342v1.
25	史朋飞, 韩松, 倪建军, 等. 结合数据增强和改进YOLOv4的水下目标检测算法. 电子测量与仪器学报, 2022, 36(3): 113- 121. URL
	SHI P F, HAN S, NI J J, et al. Underwater object detection algorithm combining dataenhancement and improved YOLOv4. Journal of Electronic Measurement and Instrumentation, 2022, 36(3): 113- 121. URL
26	HAN F L, YAO J Z, ZHU H T, et al. Marine organism detection and classification from underwater vision based on the deep CNN method. Mathematical Problems in Engineering, 2020, 2020, 1- 11.
27	YANG H B, LIU P, HU Y Z, et al. Research on underwater object recognition based on YOLOv3. Microsystem Technologies, 2021, 27(4): 1837- 1844.
28	CHEN W, FAN B J. Underwater object detection with mixed attention mechanism and multi-enhancement strategy[C]//Proceedings of Chinese Automation Congress. Washington D. C., USA: IEEE Press, 2021: 2821-2826.
29	CHEN X Y, LU Y, WU Z X, et al. Reveal of domain effect: how visual restoration contributes to object detection in aquatic scenes[EB/OL]. [2022-11-15]. https://arxiv.org/abs/2003.01913v1.
30	PENG X Image restoration contributes to underwater object detection[C]//Proceedings of International Conference on Consumer Electronics-Taiwan. Washington D. C., USA: IEEE Press, 2021: 1-2.
31	LI C Y, QUO J C, PANG Y W, et al. Single underwater image restoration by blue-green channels dehazing and red channel correction[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Washington D. C., USA: IEEE Press, 2016: 1731-1735.
32	IQBAL K, SALAM R A, OSMAN A, et al. Underwater image enhancement using an integrated colour model. IAENG International Journal of Computer Science, 2007, 34(2): 1- 10.
33	HUMMEL R. Image enhancement by histogram transformation. Computer Graphics and Image Processing, 1977, 6(2): 184- 195.
34	ZUIDERVELD K. Contrast limited adaptive histogram equalization. Graphics Gems, 1994, 474- 485.

[1]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[2]	贵向泉, 刘世清, 李立, 秦庆松, 李唐艳. 基于改进YOLOv8的景区行人检测算法[J]. 计算机工程, 2024, 50(7): 342-351.
[3]	董良振, 田建艳, 杨胜强, 陈海滨. 基于光照校正和图像融合的零件表面图像增强[J]. 计算机工程, 2024, 50(6): 245-254.
[4]	刘仕兵, 周诗涵. 高铁接触网绝缘子检测算法研究[J]. 计算机工程, 2024, 50(5): 200-208.
[5]	孙帮勇, 马铭, 于涛. 基于区域特征强化的多尺度伪装目标检测方法[J]. 计算机工程, 2024, 50(5): 209-219.
[6]	赵继达, 甄国涌, 储成群. 基于YOLOv8的无人机图像目标检测算法[J]. 计算机工程, 2024, 50(4): 113-120.
[7]	崔丽群, 曹华维. 基于改进YOLOv5的遥感图像目标检测[J]. 计算机工程, 2024, 50(4): 228-236.
[8]	冯妍舟, 刘建霞, 王海翼, 冯国昊, 白宇. 基于多级残差信息蒸馏的真实图像去噪方法[J]. 计算机工程, 2024, 50(3): 216-223.
[9]	周金涛, 高迪驹, 刘志全. 基于全景视觉的无人船水面障碍物检测方法[J]. 计算机工程, 2024, 50(2): 113-121.
[10]	何银银, 胡静, 陈志泊, 张荣国. 融合门控变换机制和GAN的低光照图像增强方法[J]. 计算机工程, 2024, 50(2): 247-255.
[11]	祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测[J]. 计算机工程, 2024, 50(1): 216-223.
[12]	杨康, 任愈, 吴学杰. 融合对比度拉伸的地铁隧道环境图像复原算法[J]. 计算机工程, 2024, 50(1): 224-231.
[13]	蒋心璐, 陈天恩, 王聪, 赵春江. 大田环境下的农业害虫图像小目标检测算法[J]. 计算机工程, 2024, 50(1): 232-241.
[14]	圣文顺, 余熊峰, 林佳燕, 陈欣. 融合注意力与特征金字塔的小尺度目标检测算法[J]. 计算机工程, 2024, 50(1): 242-250.
[15]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.

选择文件类型/文献管理软件名称

选择包含的内容

水下图像增强复原对深度学习目标检测精度的影响研究

Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

水下图像增强复原对深度学习目标检测精度的影响研究

Research on the Influence of Underwater Image Enhancement Restoration on the Accuracy of Deep Learning Object Detection

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价