基于K-Means聚类与深度学习的RGB-D SLAM算法

doi:10.19678/j.issn.1000-3428.0060052

摘要/Abstract

摘要： 传统的RGB-D视觉同时定位与制图（SLAM）算法在动态场景中识别动态特征时会产生数据错误关联，导致视觉SLAM估计姿态精度退化。提出一种适用于动态场景的RGB-D SLAM算法，利用全新的跨平台神经网络深度学习框架检测场景中的动态语义特征，并分割提取对应的动态语义特征区域。结合深度图像的K均值聚类算法和动态语义特征区域对点特征深度值进行聚类，根据聚类结果剔除动态特征点，同时通过剩余特征点计算RGB-D相机的位姿。实验结果表明，相比ORB-SLAM2、OFD-SLAM、MR-SLAM等算法，该算法能够减小动态场景下的跟踪误差，提高相机位姿估计的精度和鲁棒性，其在TUM动态数据集上相机绝对轨迹的均方根误差约为0.019 m。

关键词: 同时定位与制图, 动态场景, 深度学习, 目标检测, K均值聚类

Abstract: The traditional RGB-D visual Simultaneous Localization and Mapping(SLAM) algorithms often generate wrong data associations when recognizing dynamic features in dynamic scenarios, which leads to a loss in the accuracy of posture estimation.To address the problem, a new RGB-D SLAM algorithm is proposed for dynamic scenarios.The naive convolutional neural network deep learning framework is used to detect the dynamic semantic features in the scenario, and then segment and extract the corresponding dynamic semantic feature areas.Next, the K-Means clustering algorithm and the dynamic semantic feature areas are both used to cluster the depth values of point features.Based on the clustering results, the dynamic feature points are removed, and the remaining feature points are used to calculate the posture of the RGB-D camera.The experimental results show that compared with ORB-SLAM2, OFD-SLAM, MR-SLAM and other algorithms, the proposed algorithm can reduce the tracking errors in dynamic scenarios, and improves the accuracy and robustness of camera posture estimation.Its root mean square error of camera absolute trajectory is 0.019 m on the TUM dataset.

Key words: Simultaneous Localization and Mapping(SLAM), dynamic scenario, Deep Learning(DL), target detection, K-Means clustering

中图分类号:

TP391

张晨阳, 黄腾, 吴壮壮. 基于K-Means聚类与深度学习的RGB-D SLAM算法[J]. 计算机工程, 2022, 48(1): 236-244,252.

ZHANG Chenyang, HUANG Teng, WU Zhuangzhuang. RGB-D SLAM Algorithm Based on K-Means Clustering and Deep Learning[J]. Computer Engineering, 2022, 48(1): 236-244,252.

https://www.ecice06.com/CN/Y2022/V48/I1/236

图/表 13

20220108123502

20220108123507

20220108123511

20220108123515

20220108123519

20220108123523

20220108123526

20220108123529

20220108123533

20220108123537

20220108123542

20220108123547

20220108123550

参考文献

[1] 邸凯昌, 万文辉, 赵红颖, 等.视觉SLAM技术的进展与应用[J].测绘学报, 2018, 47(6):770-779. DI K C, WAN W H, ZHAO H Y, et al.Progress and applications of visual SLAM[J].Acta Geodaetica et Cartographica Sinica, 2018, 47(6):770-779.(in Chinese)
[2] SMITH R, SELF M, CHEESEMAN P.Estimating uncertain spatial relationships in robotics[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C., USA:IEEE Press, 1987:850-850.
[3] 赵洋, 刘国良, 田国会, 等.基于深度学习的视觉SLAM综述[J].机器人, 2017, 39(6):889-896. ZHAO Y, LIU G L, TIAN G H, et al.A survey of visual SLAM based on deep learning[J].ROBOT, 2017, 39(6):889-896.(in Chinese)
[4] DI K, XU F L, WANG J, et al.Photogrammetric processing of rover imagery of the 2003 mars exploration rover mission[J].ISPRS Journal of Photogramm and Remote Sensing, 2008, 63(2):181-201.
[5] WANG B F, ZHOU J L, TANG G S.Research on visual localization method of lunar rover[J].Science China-Information Sciences, 2014, 44(4):452-260.
[6] LIU Z Q, DI K C, LI J, et al.Landing site topographic mapping and rover localization for Chang'e-4 mission[J].Science China-Information Sciences, 2020, 63(4):1-12.
[7] ZHANG Z.Microsoft kinect sensor and its effect[J].IEEE Multimedia, 2012, 19(2):4-10.
[8] 付梦印, 吕宪伟, 刘彤.基于RGB-D数据的实时SLAM算法[J].机器人, 2015, 37(6):683-692. FU M Y, LÜ X W, LIU T.Real-time SLAM algorithm based on RGB-D data[J].ROBOT, 2015, 37(6):683-692.(in Chinese)
[9] HAN J, SHAO L, XU D, et al.Enhanced computer vision with microsoft kinect sensor:a review[J].IEEE Transactions on Cybernetics, 2013, 43(5):1318-1334.
[10] GUEVARA D C, VIETRI G, PRABAKAR M, et al.Robotic exoskeleton system controlled by kinect and haptic sensors for physical therapy[C]//Proceedings of the 29th Southern Biomedical Engineering Conference.Washington D.C., USA:IEEE Press, 2013:71-72.
[11] MUR-ARTAL R, TARDÓS J D.ORB-SLAM2:an open-source SLAM system for monocular, stereo, and RGB-D cameras[J].IEEE Transaction on Robotics, 2017, 33(5):1255-1262.
[12] ENGEL J, SCHÖPS T, CREMERS D.LSD-SLAM:large-scale direct monocular SLAM[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2014:834-849.
[13] KLEIN G, MURRAY D.Parallel tracking and mapping for small AR workspaces[C]//Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.Washington D.C., USA:IEEE Press, 2007:225-234.
[14] NEWCOMBE R A, LOVEGROVE S J, DAVISON A J.DTAM:dense tracking and mapping in real-time[C]//Proceedings of International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2011:2320-2327.
[15] WANG C C, THORPE C, THRUN S, et al.Simultaneous localization and mapping with detection and tracking of moving objects[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C., USA:IEEE Press, 2002:2918-2924.
[16] ALCANTARILLA P, YEBES J, ALMAZN J, et al.On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C., USA:IEEE Press, 2012:1290-1297.
[17] WANG Y, HUANG S.Towards dense moving object segmentation based robust dense RGB-D SLAM in dynamic scenarios[C]//Proceedings of the 13th International Conference on Control Automation Robotics and Vision.Washington D.C., USA:IEEE Press, 2014:1841-1846.
[18] BAKKAY M C, ARAFA M, ZAGROUBA E.Dense 3D SLAM in dynamic scenes using kinect[C]//Proceedings of the 7th Iberian Conference on Pattern Recognition and Image Analysis.Berlin, Germany:Springer, 2015:121-129.
[19] SUN Y, LIU M, MENG Q H.Improving RGB-D SLAM in dynamic environments:a motion removal approach[J].Robotics and Autonomous Systems, 2017, 89:110-122.
[20] SUN Y X, LIU M A, MAX Q H, et al.Motion removal for reliable RGB-D SLAM in dynamic environments[J].Robotics and Autonomous Systems, 2018, 108:115-128.
[21] MORATUWAGE D, VO B N, WANG D.Collaborative multi-vehicle SLAM with moving object tracking[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C., USA:IEEE Press, 2013:5702-5708.
[22] ZOU D, TAN P.CoSLAM:collaborative visual SLAM in dynamic environments[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013, 35(2):354-366.
[23] LEE D, MYUNG H.Solution to the SLAM problem in low dynamic environments using a pose graph and an RGB-D sensor[J].Sensors, 2014, 14(7):12467-12496.
[24] KIM D H, KIM J H.Effective background model-based RGB-D dense visual odometry in a dynamic environment[J].IEEE Transactions on Robotics, 2017, 32(6):1565-1573.
[25] LI S, LEE D.RGB-D SLAM in dynamic environments using static point weighting[J].IEEE Robotics and Automation Letters, 2017, 2(4):2263-2270.
[26] CUI L, MA C.SDF-SLAM:semantic depth filter SLAM for dynamic environments[J].IEEE Access, 2020, 8:95301-95311.
[27] WANG R, WAN W, WANG Y, et al.A new RGB-D SLAM method with moving object detection for dynamic indoor scenes[J].Remote Sensing, 2019, 11(10):1143-1148.
[28] SHIMAMURA J, MORIMOTO M, KOIKE H.Robust vSLAM for dynamic scenes[C]//Proceedings of the MVA2011 IAPR Conference on Machine Vision Applications.Washington D.C., USA:IEEE Press, 2011:344-347.
[29] CHENG J Y, SUN Y X, MENG M Q H.Improving monocular visual SLAM in dynamic environments:an optical-flow-based approach[J].Advanced Robotics, 2019, 33(12):576-589.
[30] LIU H, LIU G, TIAN G, et al.Visual SLAM based on dynamic object removal[C]//Proceedings of IEEE International Conference on Robotics and Biomimetics.Washington D.C., USA:IEEE Press, 2019:596-601.
[31] KIM D H, HAN S B, KIM J H.Visual odometry algorithm using an RGB-D sensor and IMU in a highly dynamic environment[C]//Proceedings of the 3rd Conference on Robot Intelligence Technology and Applications.Berlin, Germany:Springer, 2015:11-26.
[32] YANG S, WANG J, WANG G, et al.Robust RGB-D SLAM in dynamic environment using faster R-CNN[C]//Proceedings of the 3rd IEEE International Conference on Computer and Communications.Washington D.C., USA:IEEE Press, 2017:5702-5708.
[33] HE K, GKIOXARI G, DOLLÁR P, et al.Mask R-CNN[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2961-2969.
[34] ZHONG F, WANG S, ZHANG Z, et al.Detect-SLAM:making object detection and SLAM mutually beneficial[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision.Washington D.C., USA:IEEE Press, 2018:1001-1010.
[35] 张金凤, 石朝侠, 王燕清.动态场景下基于视觉特征的SLAM方法[J].计算机工程, 2020, 46(10):95-102. ZHANG J F, SHI C X, WANG Y Q.SLAM method based on visual features in dynamic scene[J].Computer Engineering, 2020, 46(10):95-102.(in Chinese)
[36] WANG R, WANG Y, WAN W, et al.A point-line feature based visual SLAM method in dynamic indoor scene[C]//Proceedings of 2018 Ubiquitous Positioning, Indoor Navigation and Location-Based Services.Washington D.C., USA:IEEE Press, 2018:1-6.
[37] BESCOS B, FÁCIL J M, CIVERA J, et al.DynaSLAM:tracking, mapping, and inpainting in dynamic scenes[J].IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083.
[38] CHENG J, WANG Z, ZHOU H, et al.DM-SLAM:a feature-based SLAM system for rigid dynamic scenes[J].ISPRS International Journal of Geo-Information, 2020, 9:1-11.
[39] 余东应, 刘桂华, 曾维林, 等.一种自适应窗隔匹配与深度学习相结合的RGB-D SLAM算法[J]. 计算机工程, 2020, 47(8):224-233. YU D Y, LIU G H, ZENG W L, et al.A RGB-D SLAM algorithm combining adaptive window interval matching and deep learning[J].Computer Engineering, 2020, 47(8):224-233.(in Chinese)
[40] YU C, LIU Z, LIU X J, et al.DS-SLAM:a semantic visual SLAM towards dynamic environments[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C., USA:IEEE Press, 2018:1168-1174.
[41] CHENG J, ZHANG H, MENG M Q.Improving visual localization accuracy in dynamic environments based on dynamic region removal[J].IEEE Transactions on Automation Science and Engineering, 2020, 17(3):1585-1596.
[42] RUBLEE E, RABAUD V, KONOLIGE K, et al.ORB:an efficient alternative to SIFT or SURF[C]//Proceedings of 2011 International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2011:2564-2571.
[43] HOWARD A G, ZHU M, CHEN B, et al.MobileNets:efficient conovolutional neural networks for mobilevision application[EB/OL].[2020-10-15].https://arxiv.org/pdf/1704.04861.pdf.
[44] STURM J, ENGELHARD N, ENDRES F, et al.A benchmark for the evaluation of RGB-D SLAM systems[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C., USA:IEEE Press, 2012:573-580.

选择文件类型/文献管理软件名称

选择包含的内容