动态环境下融合轻量级YOLOv5s的视觉SLAM

doi:10.19678/j.issn.1000-3428.0062294

摘要/Abstract

摘要： 移动机器人在未知环境下依靠同步定位与地图构建（SLAM）实现自身的精确定位，目前大多数视觉SLAM系统在运行时均假设外部环境是静态的，但在实际应用场景下该假设并不成立，传统的视觉SLAM系统在动态环境下易受移动目标的影响，导致系统定位精度下降。提出一种新的视觉SLAM系统，将轻量级网络MobileNetV3作为目标检测网络YOLOv5s的主干网络，从而减少模型参数量，提高算法在CPU上的推理速度。将目标检测网络、光流法与ORB-SLAM系统相结合，使SLAM系统前端提取ORB特征点的同时能够有效剔除动态特征点。仅利用静态目标上的特征点进行帧间匹配，从而求解相机位姿，提高系统在动态环境下的定位精度。在TUM动态数据集上的实验结果表明，与ORB-SLAM3系统相比，该系统的位姿估计精度提升了80.16%，与DS-SLAM、DVO-SLAM系统等动态SLAM系统相比，该系统在定位精度上有大幅提升，相比使用MASK-RCNN的DynaSLAM系统，在保持相近ATE指标的情况下，该系统具有更高的实时性。

关键词: 同步定位与地图构建, 动态环境, 目标检测, 轻量级网络, 光流法

Abstract: Simultaneous Localization and Mapping (SLAM) is indispensable for mobile robots to achieve accurate localization in an unknown environment. Currently, most visual SLAM systems assume that the external environment is static, but this assumption cannot be satisfied in practical application scenarios.The traditional visual SLAM system is easily affected by moving targets in a dynamic environment, decreasing the system localization accuracy.This study proposes a new visual SLAM systems, uses a lightweight network, MobileNetV3, as the backbone network of target detection network YOLOv5s, to reduce the number of model parameters and improve the inference speed of the algorithm on the CPU.The combination of the target detection network, optical flow method, and ORB-SLAM system enables the front-end of SLAM system to extract ORB feature points and effectively eliminate dynamic feature points. Only feature points on static targets are used for frame matching to solve camera pose and improve the positioning accuracy of the system in a dynamic environment.The experimental results for TUM dynamic dataset show that the pose estimation accuracy of the proposed system is improved by 80.16% compared with the ORB-SLAM3 system.Compared to other dynamic SLAM systems, such as DS-SLAM and DVO-SLAM, the positioning accuracy of the proposed system improved significantly.The proposed system achieved better real-time performance than the DynaSLAM system with MASK-RCNN, while maintaining similar ATE metrics.

Key words: Simultaneous Localization and Mapping(SLAM), dynamic environment, target detection, lightweight network, optical flow method

中图分类号:

TP391

伍子嘉, 陈航, 彭勇, 宋威. 动态环境下融合轻量级YOLOv5s的视觉SLAM[J]. 计算机工程, 2022, 48(8): 187-195,205.

WU Zijia, CHEN Hang, PENG Yong, SONG Wei. Visual SLAM with Lightweight YOLOv5s in Dynamic Environment[J]. Computer Engineering, 2022, 48(8): 187-195,205.

https://www.ecice06.com/CN/Y2022/V48/I8/187

图/表 17

20220825092545

20220825092548

20220825092552

20220825092556

20220825092600

20220825092603

20220825092607

20220825092610

20220825092614

20220825092617

20220825092621

20220825092632

20220825092636

20220825092640

20220825092644

20220825092648

20220825092651

参考文献

[1] 张晨阳, 黄腾, 吴壮壮.基于K-Means聚类与深度学习的RGB-D SLAM算法[J].计算机工程, 2022, 48(1):236-244, 252. ZHANG C Y, HUANG T, WU Z Z.RGB-D SLAM algorithm based on K-means clustering and deep learning[J].Computer Engineering, 2022, 48(1):236-244, 252.(in Chinese)
[2] CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al.ORB-SLAM 3:An accurate open-source library for visual, visual-inertial and multi-map SLAM[BE/OL].[2021-07-07].https://arxiv.org/pdf/2007.11898.
[3] MUR-ARTAL R, TARDÓS J D.ORB-SLAM2:an open-source SLAM system for monocular, stereo, and RGB-D cameras[J].IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[4] NEWCOMBE R A, LOVEGROVE S J, DAVISON A J.DTAM:Dense tracking and mapping in real-time[C]//Proceedings of International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2011:2320-2327.
[5] ENGEL J, SCHÖPS T, CREMERS D.LSD-SLAM:large-scale direct monocular SLAM[M].Berlin, Germany:Springer, 2014.
[6] SALAS-MORENO R F, NEWCOMBE R A, STRASDAT H, et al.SLAM:simultaneous localisation and mapping at the level of objects[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:1352-1359.
[7] CHEN W, FANG M, LIU Y H, et al.Monocular semantic SLAM in dynamic street scene based on multiple object tracking[C]//Proceedings of IEEE International Conference on Cybernetics and Intelligent Systems and IEEE Conference on Robotics, Automation and Mechatronics.Washington D.C., USA:IEEE Press, 2017:599-604.
[8] BESCOS B, FÁCIL J M, CIVERA J, et al.DynaSLAM:tracking, mapping, and inpainting in dynamic scenes[J].IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083.
[9] JOHNSON J W.Adapting mask-RCNN for automatic nucleus segmentation[EB/OL].[2021-07-07].https://arxiv.org/abs/1805.00500.
[10] YU C, LIU Z X, LIU X J, et al.DS-SLAM:a semantic visual SLAM towards dynamic environments[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C., USA:IEEE Press, 2018:1168-1174.
[11] BADRINARAYANAN V, KENDALL A, CIPOLLA R.SegNet:a deep convolutional encoder-decoder architecture for imagesegmentation[EB/OL].[2021-07-07].https://arxiv.org/abs/1511.00561.
[12] HOWARD A, SANDLER M, CHU G, et al.Searching for MobileNetV3[EB/OL].[2021-07-07].https://arxiv.org/abs/1905.02244.
[13] HOWARD A G, ZHU M L, CHEN B, et al.MobileNets:efficient convolutional neural networks for mobile vision applications[EB/OL].[2021-07-07].https://arxiv.org/abs/1704.04861.
[14] SANDLER M, HOWARD A, ZHU M L, et al.MobileNetV2:inverted residuals and linear bottlenecks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:4510-4520.
[15] LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision, 2004, 60(2):91-110.
[16] BAY H, TUYTELAARS T, VAN GOOL L.SURF:speeded up robust features[M].Berlin, Germany:Springer, 2006.
[17] RUBLEE E, RABAUD V, KONOLIGE K, et al.ORB:an efficient alternative to SIFT or SURF[C]//Proceedings of International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2011:2564-2571.
[18] 高翔, 张涛, 刘毅.视觉SLAM十四讲从理论到实践[M].北京:电子工业出版社, 2017. GAO X, ZHANG T, LIU Y.Lecture 14 of visual SLAM:from theory to practice[M].Beijing:Publishing House of Electronics Industry, 2017.(in Chinese)
[19] BOUGUET J Y.Pyramidal implementation of the affine lucas-kanade feature tracker description of the algorithm[EB/OL].[2021-07-07].https://www.doc88.com/p-7038947449312.html?r=1.
[20] KERL C, STURM J, CREMERS D.Dense visual SLAM for RGB-D cameras[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C., USA:IEEE Press, 2013:2100-2106.
[21] CHENG J Y, SUN Y X, MENG M Q H.Improving monocular visual SLAM in dynamic environments:an optical-flow-based approach[J].Advanced Robotics, 2019, 33(12):576-589.
[22] SUN Y X, LIU M, MENG M Q H.Improving RGB-D SLAM in dynamic environments:a motion removal approach[J].Robotics and Autonomous Systems, 2017, 89:110-122.

选择文件类型/文献管理软件名称

选择包含的内容