基于全景视觉的无人船水面障碍物检测方法

doi:10.19678/j.issn.1000-3428.0067238

摘要/Abstract

摘要：

无人船航行时水面障碍物检测因视角不足，导致漏检或误检，同时为满足无人船安全正常作业的需求，提出基于全景视觉的无人船水面障碍物目标检测方法。与传统的单目和双目视觉相比，全景视觉具有水平方向大视场监控的优点。基于多目全景视觉系统获得待拼接图像，在加速稳健特征(SURF)算法的基础上进行图像配准，引入k维树来构建数据索引，实现搜索空间级分类并进行快速匹配。通过M估计样本一致算法对匹配点进行优化，剔除误匹配点。对于图像融合中重叠区域出现的拼接缝隙或重影问题，设计一种基于圆弧函数的加权融合算法。提出改进的水面障碍物目标检测模型DS-YOLOv5s，将拼接好的全景图像作为训练好的模型作为输入，从而检测目标障碍物。实验结果表明，改进后的SURF算法与SURF算法相比特征点的匹配正确率提高11.47个百分点，在匹配时间上比SURF、RANSAC算法缩短5.83 s，DS-YOLOv5s模型的mAP@0.5达到95.7%，检测速度为51帧/s，符合实时目标检测标准。

关键词: 全景视觉, 图像拼接, 无人船, 改进YOLOv5, 目标检测

Abstract:

During the navigation of an unmanned ship, an inadequate perspective may cause obstacles on the water-surface to be missed.To enable the safe and normal operation of unmanned ships, this study proposes a panoramic vision-based method for the detection of water-surface obstacles.Panoramic vision is employed because it has the advantage of horizontal large-field monitoring, in contrast to traditional monocular and binocular vision. In the proposed approach, a multi-camera panoramic vision system acquires an image to be stitched.The Speeded-Up Robust Feature(SURF) algorithm then performs image registration. A k-dimensional tree constructs a data index, facilitating search-space level classification and fast matching.The M-estimation-based sample consensus algorithm optimizes the matching points and eliminates the mismatched points.A specifically designed arc function-based weighted fusion algorithm stitches the gaps and overcomes ghosting in the overlapping areas during image fusion. Finally, this study proposes an improved water-surface obstacle target detection model DS-YOLOv5s. This model takes the stitched panoramic image as input to detect target obstacles. In experiment, the improved SURF algorithm improved the feature-point matching accuracy by 11.47 percentage points compared to the SURF algorithm, and shorten the matching time by 5.83 seconds compared to the SURF, RANSAC algorithm. The DS-YOLOv5s model mAP@0.5 reached 95.7%, with a detection speed of 51 frames/s, conforming to real-time object detection standards.

Key words: panoramic vision, image stitching, unmanned ship, improved YOLOv5, target detection

周金涛, 高迪驹, 刘志全. 基于全景视觉的无人船水面障碍物检测方法[J]. 计算机工程, 2024, 50(2): 113-121.

Jintao ZHOU, Diju GAO, Zhiquan LIU. Detection Method of Water-Surface Obstacles for Unmanned Ships Based on Panoramic Vision[J]. Computer Engineering, 2024, 50(2): 113-121.

http://www.ecice06.com/CN/Y2024/V50/I2/113

图/表 17

图1 基于圆弧函数加权融合的图像拼接算法示意图

Fig.1 Schematic diagram of image stitching algorithm based on arc function weighted fusion

图2 DS-YOLOv5s算法的总体框架

Fig.2 Overall framework of the DS-YOLOv5s algorithm

图3 深度可分离卷积过程

Fig.3 Process of depthwise separable convolution

图4 水面障碍物类别图

Fig.4 lmages of obstacles categories on the water-surface

图5 在各种扰动下的水面障碍物图像

Fig.5 Images of water-surface obstacles under various disturbances

图6 待配准原图

Fig.6 Original images to be matched

图7 不同配准算法的配准图

Fig.7 Registration images among different registration algorithms

图8 较高分辨率待拼接原图

Fig.8 Original images to be stitched in higher resolution

图9 融合前后的拼接图

Fig.9 Stitching images before and after fusion

图10 面向复杂场景的对比实验结果

Fig.10 Comparison experimental results for complex scenarios

图11 YOLOv5s模型改进前后的P-R曲线

Fig.11 P-R curves before and after the improvement of the YOLOv5s model

图12 参与测试的多目视觉图像

Fig.12 Multi-eye visual images of the participants in the test

图13 多目视觉全景拼接图

Fig.13 Multi-eye vision panorama stitching image

图14 障碍物检测图

Fig.14 Obstacle detection image

参考文献 30

1	林涛. 基于深度学习的无人舰艇目标检测方法研究[D]. 沈阳: 沈阳工业大学, 2020.
	LIN T. Research on target detection method of unmanned warship based on deep learning[D]. Shenyang: Shenyang University of Technology, 2020. (in Chinese)
2	GAO S, YANG K, SHI H, et al. Review on panoramic imaging and its applications in scene understanding. IEEE Transactions on Instrumentation and Measurement, 2022, 71, 1- 34. URL
3	李盛辉, 周俊, 姬长英, 等. 基于全景视觉的智能农业车辆运动障碍目标检测. 农业机械学报, 2013, 44 (12): 239- 244. doi: 10.6041/j.issn.1000-1298.2013.12.040
	LI S H, ZHOU J, JI C Y, et al. Moving obstacle detection based on panoramic vision for intelligent agricultural vehicle. Transactions of the Chinese Society for Agricultural Machinery, 2013, 44 (12): 239- 244. doi: 10.6041/j.issn.1000-1298.2013.12.040
4	翁翔宇. 三目混合立体视觉系统检测技术研究[D]. 哈尔滨: 哈尔滨工程大学, 2019.
	WENG X Y. Research on detection technology of three-eye hybrid stereo vision system[D]. Harbin: Harbin Engineering University, 2019. (in Chinese)
5	LOWE D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
6	BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Berlin, Germany: Springer, 2006: 404-417.
7	HAST A, NYSJÖ J, MARCHETTI A. Optimal RANSAC-towards a repeatable algorithm for finding the optimal set. Journal of WSCG, 2013, 21 (1): 21- 30. URL
8	FISCHER C, WETZL J, SCHAEFFTER T, et al. Fully automated background phase correction using M⁃estimate sample consensus(MSAC)—application to 2D and 4D flow. Magnetic Resonance in Medicine, 2022, 88 (6): 2709- 2717. doi: 10.1002/mrm.29363
9	WEN L W, DING J S, XU Z. Multiframe detection of sea-surface small target using deep convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, 1- 16.
10	GUO Z X, SHUI P L. Anomaly based sea-surface small target detection using K-nearest neighbor classification. IEEE Transactions on Aerospace and Electronic Systems, 2020, 56 (6): 4947- 4964. doi: 10.1109/TAES.2020.3011868
11	ZHANG Y, GUO Z Y, WU J Q, et al. Real-time vehicle detection based on improved YOLOv5. Sustainability, 2022, 14 (19): 12274. doi: 10.3390/su141912274
12	CHEN L A, YANG Y Y, WANG Z H, et al. Underwater target detection lightweight algorithm based on multi-scale feature fusion. Journal of Marine Science and Engineering, 2023, 11 (2): 320. doi: 10.3390/jmse11020320
13	ZHENG Z H, ZHAO J, LI Y E. Research on detecting bearing-cover defects based on improved YOLOv3. IEEE Access, 2021, 9, 10304- 10315. doi: 10.1109/ACCESS.2021.3050484
14	李松江, 耿兰兰, 王鹏. 基于改进YOLOv4的车辆目标检测. 计算机工程, 2023, 49 (4): 272- 280. URL
	LI S J, GENG L L, WANG P. Vehicle target detection based on improved YOLOv4. Computer Engineering, 2023, 49 (4): 272- 280. URL
15	李嘉新, 侯进, 盛博莹, 等. 基于改进YOLOv5的遥感小目标检测网络. 计算机工程, 2023, 49 (9): 256- 264. URL
	LI J X, HOU J, SHENG B Y, et al. Remote sensing small object detection network based on improved YOLOv5. Computer Engineering, 2023, 49 (9): 256- 264. URL
16	CHEN X, LIU Y C, ACHUTHAN K. WODIS: water obstacle detection network based on image segmentation for autonomous surface vehicles in maritime environments. IEEE Transactions on Instrumentation and Measurement, 2021, 70, 1- 10. doi: 10.1109/TIM.2021.3092070
17	ZHANG R, LIU J Y, LI H Y, et al. Real-time obstacle detection based on monocular vision for unmanned surface vehicles[C]//Proceedings of International Conference on Bio-inspired Information and Communication Technologies. Berlin, Germany: Springer, 2020: 166-180.
18	SHI B H, SU Y X, LIAN C, et al. Obstacle type recognition in visual images via dilated convolutional neural network for unmanned surface vehicles. Journal of Navigation, 2022, 75 (2): 437- 454. doi: 10.1017/S0373463321000941
19	MA L Y, XIE W, HUANG H B. Convolutional neural network based obstacle detection for unmanned surface vehicle. Mathematical Biosciences and Engineering, 2020, 17 (1): 845- 861. doi: 10.3934/mbe.2020045
20	修春波, 马云菲, 潘肖楠. 基于距离融合的图像特征点匹配方法. 计算机应用, 2019, 39 (11): 3158- 3162. doi: 10.11772/j.issn.1001-9081.2019051180
	XIU C B, MA Y F, PAN X N. Image feature point matching method based on distance fusion. Journal of Computer Applications, 2019, 39 (11): 3158- 3162. doi: 10.11772/j.issn.1001-9081.2019051180
21	WANG C Y, MARK LIAO H Y, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]//Proceedings of Conference on Computer Vision and Pattern Recognition Workshops. Washington D. C., USA: IEEE Press, 2020: 390-391.
22	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2980-2988.
23	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8759-8768.
24	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 4510-4520.
25	GE Z, LIU S T, WANG F, et al. YOLOx: exceeding YOLO series in 2021[EB/OL]. [2023-02-18]. https://arxiv.org/abs/2107.08430.
26	陈科圻, 朱志亮, 邓小明等. 多尺度目标检测的深度学习研究综述. 软件学报, 2021, 32 (4): 1201- 1227. doi: 10.13328/j.cnki.jos.006166
	CHEN K Q, ZHU Z L, DENG X M, et al. Deep learning for multi-scale object detection: a survey. Journalof Soft-ware, 2021, 32 (4): 1201- 1227. doi: 10.13328/j.cnki.jos.006166
27	陈方杰, 韩军, 王祖武等. 基于改进GMS和加权投影变换的图像配准算法. 激光与光电子学进展, 2018, 55 (11): 180- 186. doi: 10.3788/LOP55.111006
	CHEN F J, HAN J, WANG Z W, et al. Image registration algorithm based on improved GMS and weighted pro-jection Transformation. Laser & Optoelectronics Progress, 2018, 55 (11): 180- 186. doi: 10.3788/LOP55.111006
28	CATHERWOOD D F, EDGAR G K, NIKOLLA D, et al. Mapping brain activity during loss of situation awareness: an EEG investigation of a basis for top-down influence on perception. Human Factors, 2014, 56 (8): 1428- 1452. doi: 10.1177/0018720814537070
29	REDMON J, FARHADI A. YOLOv3: an incremental improve-ment[EB/OL]. [2023-02-18]. https://arxiv.org/pdf/1804.02767.pdf.
30	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2023-02-18]. https://arxiv.org/abs/2004.10934.

[1]	圣文顺, 余熊峰, 林佳燕, 陈欣. 融合注意力与特征金字塔的小尺度目标检测算法[J]. 计算机工程, 2024, 50(1): 242-250.
[2]	祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测[J]. 计算机工程, 2024, 50(1): 216-223.
[3]	蒋心璐, 陈天恩, 王聪, 赵春江. 大田环境下的农业害虫图像小目标检测算法[J]. 计算机工程, 2024, 50(1): 232-241.
[4]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[5]	龙玉江, 卫薇, 舒彧, 张正刚, 王道累, 李峰. 基于自适应关键点的破损旋转绝缘子检测方法[J]. 计算机工程, 2023, 49(9): 272-278.
[6]	徐春波, 闫娟, 杨慧斌, 王博, 吴晗. 基于目标检测和语义分割的视觉SLAM算法[J]. 计算机工程, 2023, 49(8): 199-206, 214.
[7]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[8]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[9]	李强龙, 周新文, 位梦恩, 甘阳洲. 基于条形池化和注意力机制的街道场景红外目标检测算法[J]. 计算机工程, 2023, 49(8): 310-320.
[10]	闫兴亚, 匡娅茜, 白光睿, 李月. 基于深度学习的学生课堂行为识别方法[J]. 计算机工程, 2023, 49(7): 251-258.
[11]	聂志勇, 阴宇薇, 汤佳欣, 涂志刚. 一种基于边界框关键点距离的框回归算法[J]. 计算机工程, 2023, 49(7): 65-75.
[12]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[13]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[14]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[15]	谌雨章, 黄逸姿, 张钧涵. 基于多速率空洞卷积的多尺度水下小目标检测[J]. 计算机工程, 2023, 49(6): 257-264.

选择文件类型/文献管理软件名称

选择包含的内容