基于卷积调制与空间协作的水下图像增强

doi:10.19678/j.issn.1000-3428.0067883

摘要/Abstract

摘要：

针对光线在水中的散射和吸收效应造成水下图像纹理和结构不清晰的问题, 提出一种基于卷积调制(CM)与空间协作(SC)的水下图像增强算法。以编码器-解码器作为基础网络, 使用RepVGG的浅层和深层网络分别提取水下图像的纹理和结构特征。首先, 特征主导网络将RepVGG中提取到的水下图像特征转化成具有不同尺度的纹理和结构特征, 使其与解码器中的特征图进行拼接融合。其次, 在编码器中使用卷积调制模块, 采用深度可分离卷积(DSConv)模拟自注意力机制的方式减少图像细节信息的丢失, 提高编码器特征提取的能力。最后, 在解码器中使用空间协作卷积(SCConv), 在空间维度上处理水下特征保留更多的位置信息, 以提高解码器对融合后特征的增强能力。实验结果表明, 该算法在视觉感知与性能指标上优于对比算法, 峰值信噪比(PSNR)和结构相似性(SSIM)指标最高达到23.446 5 dB和0.894 6, 水下彩色图像质量评价(UCIQE)和水下图像质量测量(UIQM)指标最高达到0.582 6和3.068 9, 进一步证明了该算法能够有效增强水下图像的纹理和结构特征, 具有较好的视觉感知效果。

关键词: 图像处理, 水下图像增强, 卷积调制, 空间协作, 编解码结构

Abstract:

To address the issue of unclear textures and structures in underwater images due to light scattering and absorption effects in water, this paper proposes an underwater image enhancement algorithm based on Convolutional Modulation (CM) and Spatial Collaboration (SC). Using an encoder-decoder as the base network, the textural and structural features of underwater images are extracted using the shallow and deep networks of Reparameterization Visual Geometry Group (RepVGG). Initially, the feature dominative network transforms the underwater image features extracted from RepVGG into textural and structural features of various scales, which are then concatenated and fused with feature maps in the decoder. Subsequently, a CM module is employed within the encoder, where Depth Separable Convolution (DSConv) is implemented to simulate a self-attention mechanism to reduce the loss of image detail information and enhance feature extraction. Finally, Spatially Collaborative Convolution (SCConv) is performed in the decoder to process underwater features in the spatial dimension, preserving more positional information and enhancing the merged features. Experimental results indicate that this algorithm outperforms comparable algorithms in terms of visual perception and performance metrics, achieving a Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) index as high as 23.446 5 dB and 0.894 6, respectively. The highest scores reached for Underwater Color Image Quality Evaluation (UCIQE) and Underwater Image Quality Measurement (UIQM) are 0.582 6 and 3.068 9, respectively, which further demonstrate the algorithm's effectiveness in enhancing textural and structural features of underwater images and providing superior visual perception.

Key words: image processing, underwater image enhancement, Convolutional Modulation(CM), Spatial Collaboration(SC), encoding and decoding structure

郭伟, 王欣哲, 王江达, 王春艳. 基于卷积调制与空间协作的水下图像增强[J]. 计算机工程, 2024, 50(8): 310-318.

Wei GUO, Xinzhe WANG, Jiangda WANG, Chunyan WANG. Underwater Image Enhancement Based on Convolutional Modulation and Spatial Collaboration[J]. Computer Engineering, 2024, 50(8): 310-318.

https://www.ecice06.com/CN/Y2024/V50/I8/310

图/表 12

图1 所提算法框架

Fig.1 Framework of the proposed algorithm

图2 多路径特征细化模块

Fig.2 Multi-path feature refinement module

图3 特征变换层

Fig.3 Feature transformation layer

图4 卷积调制模块

Fig.4 Convolutional modulation module

图5 空间协作模块

Fig.5 Spatial collaborative module

图6 EUVP数据集中不同算法的视觉效果对比

Fig.6 Comparison of visual effect of different algorithms in EUVP dataset

图7 UIEB数据集中泛化能力对比

Fig.7 Comparison of generalization ability in UIEB dataset

参考文献 28

1	王红茹, 张弓, 卢道华, 等. 基于背景光估计与颜色修正的水下图像增强. 计算机工程, 2020, 46(10): 253- 258. URL
	WANG H R, ZHANG G, LU D H, et al. Underwater image enhancement based on background light estimation and color correction. Computer Engineering, 2020, 46(10): 253- 258. URL
2	DREWS P, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]//Proceedings of IEEE International Conference on Computer Vision Workshops. Washington D. C., USA: IEEE Press, 2013: 825-830.
3	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341- 2353. doi: 10.1109/TPAMI.2010.168
4	李莉, 王新强, 银珊. 基于衰减补偿与直方图拉伸的水下图像增强算法. 计算机工程, 2022, 48(6): 222- 227. URL
	LI L, WANG X Q, YIN S. Underwater image enhancement algorithm based on attenuation compensation and histogram stretching. Computer Engineering, 2022, 48(6): 222- 227. URL
5	HUMMEL R. Image enhancement by histogram transformation. Computer Graphics and Image Processing, 1977, 6(2): 184- 195. doi: 10.1016/S0146-664X(77)80011-7
6	HUANG D M, WANG Y, SONG W, et al. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition[C]//Proceedings of International Conference on Multimedia Modeling. Berlin, Germany: Springer, 2018: 453-465.
7	徐超越, 余映, 何鹏浩, 等. 基于U-Net的多尺度低照度图像增强网络. 计算机工程, 2022, 48(8): 215- 223. URL
	XU C Y, YU Y, HE P H, et al. Multi-scale low-light image enhancement network based on U-Net. Computer Engineering, 2022, 48(8): 215- 223. URL
8	WANG Y, ZHANG J, CAO Y, et al. A deep CNN method for underwater image enhancement[C]//Proceedings of 2017 IEEE International Conference on Image Processing (ICIP). New York, USA: ACM Press, 2017: 1382-1386.
9	陈清江, 解亚丽. 基于稠密级联卷积神经网络的水下图像增强. 激光与光电子学进展, 2022, 59(22): 2215004. URL
	CHEN Q J, XIE Y L. Underwater image enhancement based on dense cascaded convolutional neural network. Laser & Optoelectronics Progress, 2022, 59(22): 2215004. URL
10	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision (ICCV). Washington D. C., USA: IEEE Press, 2017: 2223-2232.
11	CHOI Y, CHOI M, KIM M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8789-8797.
12	DING X H, ZHANG X Y, MA N N, et al. RepVGG: making VGG-style ConvNets great again[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2021: 13733-13742.
13	DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 248-255.
14	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241.
15	HOU Q, LU C Z, CHENG M M, et al. Conv2Former: a simple Transformer-style ConvNet for visual re-cognition[EB/OL]. [2023-05-11]. https://arxiv.org/abs/2211.11943.
16	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2021: 13713-13722.
17	WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers. Washington D. C., USA: IEEE Press, 2003: 1398-1402.
18	ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 2020, 5(2): 3227- 3234. doi: 10.1109/LRA.2020.2974710
19	LI C, GUO C, REN W, et al. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2019, 29, 4376- 4389.
20	HOU G J, LI J M, WANG G D, et al. A novel dark channel prior guided variational framework for underwater image restoration. Journal of Visual Communication and Image Representation, 2020, 66, 102732. doi: 10.1016/j.jvcir.2019.102732
21	CARLEVARIS-BIANCO N, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing[EB/OL]. [2023-05-11]. https://www.semanticscholar.org/paper/Initial-results-in-underwater-single-image-dehazing-Carlevaris-Bianco-Mohan/6554d59867907742259235d2647f37d71a592f19.
22	FU X Y, FAN Z W, LING M, et al. Two-step approach for single underwater image enhancement[C]//Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS). Washington D. C., USA: IEEE Press, 2017: 789-794.
23	SONG W, WANG Y, HUANG D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]//Proceedings of the 19th Pacific-Rim Conference on Multimedia. Berlin, Germany: Springer, 2018: 678-688.
24	WANG D, MA L, LIU R S, et al. Semantic-aware texture-structure feature collaboration for underwater image enhancement[C]//Proceedings of International Conference on Robotics and Automation (ICRA). Washington D. C., USA: IEEE Press, 2022: 4592-4598.
25	KORHONEN J, YOU J Y. Peak signal-to-noise ratio revisited: is simple beautiful? [C]//Proceedings of the 4th International Workshop on Quality of Multimedia Experience. Washington D. C., USA: IEEE Press, 2012: 37-38.
26	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600- 612. doi: 10.1109/TIP.2003.819861
27	PANETTA K, GAO C, AGAIAN S. Human-visual-system-inspired underwater image quality measures. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541- 551.
28	YANG M, SOWMYA A. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 2015, 24(12): 6062- 6071. doi: 10.1109/TIP.2015.2491020

[1]	杨太龙, 赵红朋, 张磊. 基于国产异构平台的奇异值分解法[J]. 计算机工程, 2024, 50(9): 216-225.
[2]	王安政, 党建武, 岳彪, 杨景玉. 基于位置信息和注意力机制的路面裂缝检测[J]. 计算机工程, 2024, 50(4): 303-312.
[3]	杨晶晶, 谢海燕, 薛妮妮, 张傲明. 基于双通道残差网络的水下图像去噪研究[J]. 计算机工程, 2023, 49(4): 188-198.
[4]	罗嗣卿, 陈慧. 基于生成对抗网络的图像场景转换[J]. 计算机工程, 2023, 49(4): 217-225.
[5]	余嘉昕, 王春媛, 韩华, 高燕. 基于融合代价和优化引导滤波的立体匹配算法[J]. 计算机工程, 2023, 49(3): 257-262,270.
[6]	邓天民, 谭思奇, 蒲龙忠. 基于改进YOLOv5s的交通信号灯识别方法[J]. 计算机工程, 2022, 48(9): 55-62.
[7]	高海韬, 李丹宁, 王彬, 唐鑫鑫. 运动模糊图像PSF参数估计方法改进及图像复原[J]. 计算机工程, 2022, 48(9): 197-203,212.
[8]	林志洁, 郑秋岚, 梁涌, 邢卫. 基于内卷U-Net的医学图像分割模型[J]. 计算机工程, 2022, 48(8): 180-186.
[9]	李莉, 王新强, 银珊. 基于衰减补偿与直方图拉伸的水下图像增强算法[J]. 计算机工程, 2022, 48(6): 222-227.
[10]	刘金硕, 黄朔, 邓娟. 面向PMVS算法的自动两级并行翻译方法[J]. 计算机工程, 2022, 48(12): 16-23.
[11]	王宇, 李涛, 邢立冬, 冯臻夫. OpenVX高效能并行可重构运算通路的设计与实现[J]. 计算机工程, 2021, 47(12): 236-248.
[12]	张一飞, 李新福, 田学东. 融合边缘特征的SAD立体匹配算法[J]. 计算机工程, 2020, 46(4): 236-240,246.
[13]	王红茹, 张弓, 卢道华, 王佳. 基于背景光估计与颜色修正的水下图像增强[J]. 计算机工程, 2020, 46(10): 253-258.
[14]	苗续芝,陈伟,毕方明,房卫东,张武雄. 基于改进FOA-SVM的矿井火灾图像识别[J]. 计算机工程, 2019, 45(4): 267-274.
[15]	李超凡,陈庆奎. 基于传感器的GPU集群功耗收集监控系统[J]. 计算机工程, 2019, 45(3): 65-72.

选择文件类型/文献管理软件名称

选择包含的内容