Indoor Scene Segmentation Model Using Three-Dimensional Point Cloud Based on Super Point Graph Network

doi:10.19678/j.issn.1000-3428.0059789

Abstract

Abstract: The samples of the existing current point cloud datasets are unbalanced, and PointNet can not effectively utilize the neighborhood information of the point cloud.To address the problem, a three-dimensional scene segmentation method using point cloud is proposed based on super point graph network. Based on geometric information, the original point cloud is divided into several blocks homogeneously.Utilizing small-sized PointNet, the original features of point cloud are mapped to high-dimensional space to mine the deep fine-grained semantic information in the scene.By constructing a kind of self-normalization gate recurrent units, the contextual semantic segmentation performance of the point cloud is improved, and the Focal Loss function in the two-dimensional image field is used to segment the scenes of the point cloud.The experimental results on the S3DIS dataset show that the proposed model exhibits a MIOU of 63.8%, OA of 86.4% and mAcc of 74.3%, which are respectively 1.7%, 0.9% and 1.3% higher than the SPG model.The proposed model significantly improves the semantic segmentation performance of point cloud for three-dimensional scenes.

Key words: scene segmentation, 3D point cloud, contextual information, homogeneous segmentation, deep learning

摘要： 针对点云数据集样本不均衡及PointNet网络无法充分利用点云邻域信息的问题，提出一种三维点云场景分割模型。根据几何信息将原始点云块同质分割为超点，利用小型PointNet网络将点云原始特征映射到高维空间中，并挖掘场景中深层语义信息。在此基础上，构建自归一化属性门控单元优化点云上下文语义分割效果，采用二维图像领域中的Focal Loss损失函数实现点云场景分割。实验结果表明，该模型在S3DIS数据集上的平均交并比、总体精度、平均精度分别达到63.8%、86.4%、74.3%，较SPG模型分别提升1.7、0.9、1.3个百分点。

关键词: 场景分割, 三维点云, 上下文信息, 同质分割, 深度学习

CLC Number:

TP183

HUO Zhanqiang, WANG Yongjie, LUO Fen, QIAO Yingxu. Indoor Scene Segmentation Model Using Three-Dimensional Point Cloud Based on Super Point Graph Network[J]. Computer Engineering, 2021, 47(12): 308-315.

霍占强, 王勇杰, 雒芬, 乔应旭. 基于超点图网络的三维点云室内场景分割模型[J]. 计算机工程, 2021, 47(12): 308-315.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0059789

http://www.ecice06.com/EN/Y2021/V47/I12/308

Figures/Tables 6

References

[1] 姜名扬.三维点云图形的分类和分割[D].上海:上海交通大学, 2019. JIANG M Y.Classification and segmentation of 3D point clouds[D].Shanghai:Shanghai Jiao Tong University, 2019.(in Chinese)
[2] CHEN X Z, MA H M, WAN J, et al.Multi-view 3D object detection network for autonomous driving[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:1907-1915.
[3] ZHU Y K, MOTTAGHI R, KOLVE E, et al.Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C., USA:IEEE Press, 2017:3357-3364.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM, 2017, 60(6):84-90.
[5] 曾碧, 黄文.一种融合多特征聚类集成的室内点云分割方法[J].计算机工程, 2018, 44(3):281-286. ZENG B, HUANG W.An indoor point cloud segmentation method fusing with multi-feature cluster ensemble[J].Computer Engineering, 2018, 44(3):281-286.(in Chinese)
[6] CHO K, VAN MERRIËNBOER B, GULCEHRE C, et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL].[2020-09-02].https://arxiv.org/abs/1406.1078v3.
[7] LIN T Y, GOYAL P, GIRSHICK R, et al.Focal loss for dense object detection[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2980-2988.
[8] ARMENI I, SAX S, ZAMIR A R, et al.Joint 2D-3D-semantic data for indoor scene understanding[EB/OL].[2020-09-02].https://arxiv.org/abs/1406.1078v3.
[9] SU H, MAJI S, KALOGERAKIS E, et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of IEEE International Conference on Computer Vision.Santiago, Chile:IEEE Press, 2015:945-953.
[10] 李浩, 霍雯, 裴春营, 等.融合VGG与FCN的智能出租车订单预测模型[J].计算机工程, 2020, 46(12):276-282. LI HAO, HUO W, PEI C Y, et al.Intelligent taxi order forecasting model fusing VGG with FCN[J].Computer Engineering, 2020, 46(12):276-282.(in Chinese)
[11] BOULCH A, GUERRY J, LE S B, et al.SnapNet:3D point cloud semantic labeling with 2D deep segmentation networks[J].Computers and Graphics, 2017, 71:189-198.
[12] MATURANA D, SCHERER S.VoxNet:a 3D convolutional neural network for real-time object recognition[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C., USA:IEEE Press, 2015:922-928.
[13] LE T, DUAN Y.PointGrid:a deep network for 3D shape understanding[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:9204-9214.
[14] KLOKOV R, LEMPITSKY V.Escape from cells:deep KD-networks for the recognition of 3D point cloud models[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:863-872.
[15] 顾砾, 季怡, 刘纯平.基于多模态特征融合的三维点云分类方法[J].计算机工程, 2021, 47(2):279-284. GU L, JI Y, LIU C P.A classification model of 3D point cloud based on multimodal feature fusion[J].Computer Engineering, 2021, 47(2):279-284.(in Chinese)
[16] 李美佳, 于泽宽, 刘晓, 等.点云算法在医学领域的研究进展[J].中国图象图形学报, 2020, 25(10):2013-2023. LI M J, YU Z K, LIU X, et al.Progress of point cloud algorithm in medical field[J].Journal of Image and Graphics, 2020, 25(10):2013-2023.(in Chinese)
[17] 张蕊, 李锦涛.基于深度学习的场景分割算法研究综述[J].计算机研究与发展, 2020, 57(4):859-875. ZHANG R, LI J T.A survey on algorithm research of scene parsing based on deep learning[J].Journal of Computer Research and Development, 2020, 57(4):859-875.(in Chinese)
[18] QI C R, SU H, MO K, et al.Pointnet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:652-660.
[19] QI C R, YI L, SU H, et al.PointNet++:deep hierarchical feature learning on point sets in a metric space[EB/OL].[2020-09-02].https://www.researchgate.net/publication/317426798_PointNet_Deep_Hierarchical_Feature_Learning_on_Point_Sets_in_a_Metric_Space.
[20] LANDRIEU L, OBOZINSKI G.Cut pursuit:fast algorithms to learn piecewise constant functions on general weighted graphs[J].SIAM Journal on Imaging Sciences, 2017, 10(4):1724-1766.
[21] LANDRIEU L, SIMONOVSKY M.Large-scale point cloud semantic segmentation with superpoint graphs[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:4558-4567.
[22] ACHANTA R, SHAJI A, SMITH K, et al.SLIC superpixels compared to state-of-the-art superpixel methods[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11):2274-2282.
[23] SUTSKEVER I, VINYALS O, LE Q V.Sequence to sequence learning with neural networks[EB/OL].[2020-09-02].https://wenku.baidu.com/view/9dad28f2c281e 53a5902ffe9.html.
[24] SIMONOVSKY M, KOMODAKIS N.Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:3693-3702.
[25] DEMANTKÉ J, MALLET C, DAVID N, et al.Dimensionality based scale selection in 3D lidar point clouds[J].ISPRS-International Archives of the Photogrammetry Remote Sensing and Spatial Information Sciences, 2011, 38(5):97-102.
[26] KLAMBAUER G, UNTERTHINER T, MAYR A, et al.Self-normalizing neural networks[EB/OL].[2020-09-02].https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE07457319.
[27] GRAVES A.Long short-term memory:supervised sequence labelling with recurrent neural networks[J].Springer, 2012, 9(8):1735-1780.
[28] ENGELMANN F, KONTOGIANNI T, HERMANS A, et al.Exploring spatial context for 3D semantic segmentation of point clouds[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Washington D.C., USA:IEEE Press, 2017:716-724.
[29] HUANG Q G, WANG W Y, NEUMANN U.Recurrent slice networks for 3D segmentation of point clouds[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:2626-2635.

Please choose a citation manager

Content to export