Point Cloud Semantic Segmentation Method Based on Block Feature Fusion

doi:10.19678/j.issn.1000-3428.0063863

Abstract

Abstract: A semantic segmentation method integrating block features is proposed to reduce the computational complexity of semantic segmentation of multi-class 3D objects in outdoor large-scale point cloud scenes.The square grid segmentation method is used to divide, sample, and combine the 3D point cloud, obtain the simplified point cloud combination block set, and input it into the block feature extraction and fusion network, to obtain the feature correction vector of each block.In order to correct the wrong feature caused by segmentation, the global feature correction network of the point cloud block is designed, and the feature correction vector is fused with the global feature of the original point cloud in the form of a residual.Based on this, the square mesh segmentation size is introduced into the back-propagation process as a parameter of the neural network for optimization, to establish an efficient point cloud semantic segmentation network.The experimental results show that the back propagation algorithm can optimize the segmentation size to near the optimal value, and the global feature correction method in the proposed network can improve the semantic segmentation accuracy.The semantic segmentation accuracy of this method on the Semantic3D dataset is 78.7%, which is 1.3% higher than the RandLA-Net method, and its point cloud preprocessing computational complexity and network computing time are significantly reduced on the premise of ensuring segmentation accuracy.When processing large-scale point clouds with 100 000‒1 million points, the semantic segmentation speed of point clouds is 2‒4 times higher than that of SPG, KPConv, and other methods.

Key words: point cloud semantic segmentation, block feature fusion, point cloud feature extraction, deep learning, point cloud preprocessing

摘要： 为降低室外大规模点云场景中多类三维目标语义分割的计算复杂度，提出一种融合区块特征的语义分割方法。采用方形网格分割方法对三维点云进行区块划分、采样以及组合，求取简化的点云组合区块集，将其输入至区块特征提取和融合网络中从而获得每个区块的特征修正向量。设计点云区块全局特征修正网络，以残差的方式融合特征修正向量与原始点云全局特征，修正因分割造成的错误特征。在此基础上，将方形网格分割尺寸作为神经网络的参数引入反向传播过程中进行优化，从而建立高效的点云语义分割网络。实验结果表明，反向传播算法可以优化分割尺寸至最佳值附近，所提网络中的全局特征修正方法能够提高语义分割精度，该方法在Semantic3D数据集上的语义分割精度达到78.7%，较RandLA-Net方法提升1.3%，且在保证分割精度的前提下其点云预处理计算复杂度和网络计算时间明显降低，在处理点数为10万~100万的大规模点云时，点云语义分割速度较SPG、KPConv等方法提升2~4倍。

关键词: 点云语义分割, 区块特征融合, 点云特征提取, 深度学习, 点云预处理

CLC Number:

TP393

GAO Qingji, LI Tianhao, XING Zhiwei, LIU Peipei. Point Cloud Semantic Segmentation Method Based on Block Feature Fusion[J]. Computer Engineering, 2022, 48(9): 37-44,54.

高庆吉, 李天昊, 邢志伟, 刘佩佩. 基于区块特征融合的点云语义分割方法[J]. 计算机工程, 2022, 48(9): 37-44,54.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0063863

http://www.ecice06.com/EN/Y2022/V48/I9/37

Figures/Tables 13

References

[1] 李娇娇, 孙红岩, 董雨, 等.基于深度学习的3维点云处理综述[J].计算机研究与发展, 2022, 59(5):1160-1179. LI J J, SUN H Y, DONG Y, et al.Survey of 3-dimensional point cloud processing based on deep learning[J].Journal of Computer Research and Development, 2022, 59(5):1160-1179.(in Chinese)
[2] 霍占强, 王勇杰, 雒芬, 等.基于超点图网络的三维点云室内场景分割模型[J].计算机工程, 2021, 47(12):308-315. HUO Z Q, WANG Y J, LUO F, et al.Indoor scene segmentation model using three-dimensional point cloud based on super point graph network[J].Computer Engineering, 2021, 47(12):308-315.(in Chinese)
[3] 张诚, 黄丹丹, 刘智.动态环境下的语义SLAM算法[J].信息与控制, 2021, 50(3):297-307. ZHANG C, HUANG D D, LIU Z.Semantic SLAM algorithm based on dynamic environment[J].Information and Control, 2021, 50(3):297-307.(in Chinese)
[4] CHARLES R Q, HAO S, MO K C, et al.PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:77-85.
[5] HU Q Y, YANG B, XIE L H, et al.RandLA-net:efficient semantic segmentation of large-scale point clouds[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:11105-11114.
[6] WANG Y, SUN Y B, LIU Z W, et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics, 2019, 38(5):1-12.
[7] HUA B S, TRAN M K, YEUNG S K.Pointwise convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:984-993.
[8] 郑乐乐, 韩慧妍, 韩燮.基于显著性与弱凸性的三维点云模型分割[J].计算机工程, 2018, 44(4):299-304. ZHENG L L, HAN H Y, HAN X.Three-dimensional point cloud model segmentation based on significance and weak convexity[J].Computer Engineering, 2018, 44(4):299-304.(in Chinese)
[9] 顾砾, 季怡, 刘纯平.基于多模态特征融合的三维点云分类方法[J].计算机工程, 2021, 47(2):279-284. GU L, JI Y, LIU C P.Classification method of three-dimensional point cloud based on multiple modal feature fusion[J].Computer Engineering, 2021, 47(2):279-284.(in Chinese)
[10] CHEN S H, NIU S F, LAN T, et al.PCT:large-scale 3D point cloud representations via graph inception networks with applications to autonomous driving[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2019:4395-4399.
[11] LANDRIEU L, SIMONOVSKY M.Large-scale point cloud semantic segmentation with superpoint graphs[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:4558-4567.
[12] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[13] OH S H.Generalization of the cross-entropy error function to improve the error backpropagation algorithm[C]//Proceedings of International Conference on Neural Networks.Washington D.C., USA:IEEE Press, 1997:1856-1861.
[14] BEHLEY J, GARBADE M, MILIOTO A, et al.SemanticKITTI:a dataset for semantic scene understanding of LiDAR sequences[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:9296-9306.
[15] CAO Y, ZHANG X J, DUAN B H, et al.An improved method to build the KD-tree based on presorted results[C]//Proceedings of IEEE International Conference on Software Engineering and Service Science.Washington D.C., USA:IEEE Press, 2020:71-75.
[16] SU H, JAMPANI V, SUN D Q, et al.SPLATNet:sparse lattice networks for point cloud processing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:2530-2539.
[17] QI C R, YI L, SU H, et al.PointNet++:deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems.Washington D.C., USA:IEEE Press, 2017:5100-5109.
[18] TATARCHENKO M, PARK J, KOLTUN V, et al.Tangent convolutions for dense prediction in 3D[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:3887-3896.
[19] HACKEL T, SAVINOV N, LADICKY L, et al.Semantic3D net:a new large-scale point cloud classification benchmark[EB/OL].[2021-12-05].https://arxiv.org/abs/1704.03847.
[20] BOULCH A, GUERRY J, LE SAUX B, et al.SnapNet:3D point cloud semantic labeling with 2D deep segmentation networks[J].Computers & Graphics, 2018, 71:189-198.
[21] TCHAPMI L, CHOY C, ARMENI I, et al.SEGCloud:semantic segmentation of 3D point clouds[C]//Proceedings of International Conference on 3D Vision.Washington D.C., USA:IEEE Press, 2017:537-547.
[22] THOMAS H, GOULETTE F, DESCHAUD J E, et al.Semantic classification of 3D point clouds with multiscale spherical neighborhoods[C]//Proceedings of International Conference on 3D Vision.Washington D.C., USA:IEEE Press, 2018:390-398.
[23] ROYNARD X, DESCHAUD J E, GOULETTE F.Classification of point cloud scenes with multiscale voxel deep network[EB/OL].[2021-12-05].https://arxiv.org/pdf/1804.03583.pdf.
[24] ZHANG Z Y, HUA B S, YEUNG S K.ShellNet:efficient point cloud convolutional neural networks using concentric shells statistics[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:1607-1616.
[25] WANG L, HUANG Y C, HOU Y L, et al.Graph attention convolution for point cloud semantic segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:10288-10297.
[26] THOMAS H, QI C R, DESCHAUD J E, et al.KPConv:flexible and deformable convolution for point clouds[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:6410-6419.
[27] 李少飞, 史泽林, 庄春刚.基于深度学习的物体点云六维位姿估计方法[J].计算机工程, 2021, 47(8):216-223. LI S F, SHI Z L, ZHUANG C G.Deep learning-based 6D object pose estimation method from point clouds[J].Computer Engineering, 2021, 47(8):216-223.(in Chinese)

Please choose a citation manager

Content to export