基于超点图网络的三维点云室内场景分割方法

doi:10.19678/j.issn.1000-3428.0059789

摘要/Abstract

摘要： 三维点云语义分割在自动驾驶、室内导航等领域有着广泛应用。针对目前点云数据集样本不均衡和类 PointNet 网络不能充分利用点云邻域信息问题，提出一种基于超点图网络（SPG）的三维点云场景分割方法。该方法通过最小割理论，根据几何信息将原始点云块同质分割为超点。利用小型 PointNet，将点云原始特征映射到高维空间挖掘场景中深层语义信息。为有效提高点云上下文语义分割，构建一种自归一化属性门控单元及采用二维图像领域针对样本不均衡的 Focal Loss 损失函数来实现点云的场景分割。实验结果表明，该方法能够显著地提高三维点云场景语义分割的准确率和分割效果。在 S3DIS 数据集上的平均交并比、总体精度和平均精度识别准确率分别为 63.8%、86.4%和 74.3%，较 SPG 模型分别提升 1.7%、0.9%和 1.3%。

Abstract: Three-dimensional(3D) semantic segmentation is widely applied in various fields such as automatic driving and indoor navigation. Aiming at the problems that the samples of current point cloud dataset are unbalanced and the similar pointnet network can not effectively utilize the neighborhood information around the point cloud.This paper proposes a 3D point cloud scene segmentation method based on Super Point Graph network. By using the min-cut theory, the point cloud of the scene is divided into several blocks homogeneously according to the geometric information. Each block is denoted as a super point.Utilizing the mini-pointnet, the original features of point cloud are mapped to high-dimensional features fully mining the deep fine-grained geometric features in scenes.By constructing a self-normalization Gate Recurrent Unit and the Focal Loss aimed to the imbalance samples, this model can effectively extract the context features of point cloud. The experimental results show that the proposed method can improve the accuracy and segmentation effect of 3D point cloud semantic segmentation effectively. The MIOU,OA and mAcc on S3DIS dataset reach 63.8%, 86.4% and 74.3% respectively, which have the increases of 1.7% , 0.9% and 1.3% compared with the SPG model.

霍占强, 王勇杰, 雒芬, 乔应旭. 基于超点图网络的三维点云室内场景分割方法[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0059789.

Huo Zhanqiang , Wang Yongjie , Luo Fen , Qiao Yingxu. Three-dimensional Point Cloud Indoor Segmentation Model based on Super Point Graph Network[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0059789.

参考文献

[1] JIANG Mingyang. Classification and segmentation of 3D point clouds[D]. Shanghai: Shanghai Jiao Tong University, 2019. 姜名扬. 三维点云图形的分类和分割[D]. 上海: 上海交通大学,2019.
[2] CHEN Xiaozhi, MA Huimin, WAN Ji, et al. Multi-view 3d object detection network for autonomous driving[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE Computer Society Press, 2017: 1907-1915.
[3] ZHU Yuke, MOTTAGHI R, KOLVE E, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//IEEE international conference on robotics and automation. Marina Bay Sands, Singapore: IEEE Computer Society Press, 2017: 3357-3364.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[5] ZENG Bi, HUANG Wen. An Indoor Point Cloud Segmentation Method Fusing with Multi-feature Cluster Ensemble[J].Computer Engineering, 2018, 44(3): 281-286. 曾碧, 黄文. 一种融合多特征聚类集成的室内点云分割方法[J]. 计算机工程, 2018, 44(3): 281-286.
[6] SU Hang, MAJI S, KALOGERAKIS E, et al. Multi-view Convolutional Neural Networks for 3D Shape Recognition[C]//IEEE International Conference on Computer Vision. Santiago, Chile: IEEE Press, 2015: 945-953.
[7] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. arXiv preprint arXiv：1409.1556, 2014.
[8] BOULCH A , GUERRY J , LE Saux B , et al. SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks[J]. Computers & Graphics, 2017, 71(APR.):189-198.
[9] MATURANA D, SCHERER S. VoxNet: A 3D Convolutional Neural Network for real-time object recognition[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg, Germany: IEEE Press, 2015: 922-928.
[10] LE T, DUAN Ye. PointGrid: A deep network for 3D shape understanding[C]//IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Computer Society Press, 2018: 9204– 9214.
[11] KLOKOV R, LEMPITSKY V. Escape from cells: Deep kd-networks for the recognition of 3d point cloud models[C]//Proceedings of the IEEE International Conference on Computer Vision. NewYork, USA: IEEE Press, 2017: 863-872.
[12] GU Li, JI Yi, LIU Chunping. A Classification Model of 3D Point Cloud Based On Multimodal Feature Fusion[J/OL]. Computer Engineering:1-7[2020-09-25].https://doi.org/10.19678/j.issn.1000-3428.0057147. 顾砾 , 季怡 , 刘纯平 . 基于多模态特征融合的三维点云分类方法 [J/OL]. 计算机工程 :1-7[2020-09-25]. https://doi.org/10.19678/j.issn.1000-3428.0057147.
[13] LI Meijia, YU Zekuan, LIU Xiao, et al. Progress of point cloud algorithm in medical field[J]. Journal of Image and Graphics, 2020, 25(10): 2013-2023. 李美佳,于泽宽,刘晓,等.点云算法在医学领域的研究进展[J]. 中国图象图形学报, 2020, 25(10): 2013-2023.
[14] ZHANG Rui, LI Jintao. A Survey on Algorithm Research of Scene Parsing Based on Deep Learning[J]. Journal of Computer Research and Development, 2020, 057(004): 859-875. 张蕊, 李锦涛. 基于深度学习的场景分割算法研究综述[J]. 计算机研究与发展, 2020, 057(004): 859-875.
[15] QI C R, SU Hao, MO K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 652-660.
[16] QI C R, YI L, SU Hao, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metricspace[C]//Advances in Neural Information Processing Systems. California, USA: IEEE Press, 2017: 5099-5108.
[17] LANDRIEU L, OBOZINSKI G. Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs[J]. SIAM Journal on Imaging Sciences, 2017, 10(4): 1724-1766.
[18] LANDRIEU L, SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Computer Society Press,2018: 4558-4567.
[19] CHO K, VAN Merriënboer B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv preprint arXiv:1406.1078, 2014.
[20] LIN Tsung-Yi, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE international conference on computer vision. Venice, Italy: IEEE Computer Society Press,2017: 2980-2988.
[21] ARMENI I, SAX S, ZAMIR A R, et al. Joint 2D-3D-semantic data for indoor scene understanding[J]. arXiv preprint arXiv:1702.01105,2017.
[22] ACHANTA R , SHAJI A , SMITH K , et al. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2012, 34(11):2274-2282.
[23] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Advances in neural information processing systems. Montreal, Canada: MIT Press, 2014: 3104-3112.
[24] SIMONOVSKY M, KOMODAKIS N. Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE Computer Society Press, 2017: 3693-3702.
[25] DEMANTKÉ J, MALLET C, DAVID N, et al. Dimensionality based scale selection in 3D lidar point clouds[J]. ISPRS - International Archives of the Photogrammetry Remote Sensing and Spatial Information Sciences, 2011, 38(5): 97-102.
[26] KLAMBAUER G, UNTERTHINER T, MAYR A, et al. Self-normalizing neural networks[C]//Advances in neural information processing systems. California, USA: MIT Press, 2017: 971-980.
[27] GRAVES A. Long Short-Term Memory: Supervised Sequence Labelling with Recurrent Neural Networks[J]. Springer, 2012, 9(8): 1735-1780.
[28] ENGELMANN F, KONTOGIANNI T, HERMANS A, et al. Exploring spatial context for 3D semantic segmentation of point clouds[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. New York, USA: IEEE Press, 2017: 716-724.
[29] HUANG Qiangui, WANG Weiyue, NEUMANN U. Recurrent slice networks for 3d segmentation of point clouds[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Computer Society Press,2018: 2626-2635.