基于可变形三维图卷积的轻量级点云分类研究

doi:10.19678/j.issn.1000-3428.0067589

摘要/Abstract

摘要：

现有深度学习方法在处理点云分类任务时, 依赖于点的绝对坐标, 存在模型复杂度较大的问题。对此, 提出一种轻量级的点云分类网络DMGCN-3D。使用自适应空洞K近邻(KNN)算法构造图结构, 尽可能捕捉局部更广泛空间的几何结构信息, 并减少计算开支; 构造可变形三维图卷积, 引入可学习的点与点之间的方向向量来获取相对特性, 在特征提取过程中保证点云的置换不变性与尺度不变性; 构建多头自注意力模块, 通过残差结构将分组变换注意力(GSA)与多层感知机(MLP)相结合, MLP有助于保持原始点云信息的完整性, GSA使得网络能够学习特征内部的自相关性, 在提高特征表达能力的同时降低参数总量; 使用空间变换网络结合MLP来学习点云特征; 对所提取的特征进行融合以得到更综合的特征, 将其用于点云分类。实验结果表明, DMGCN-3D在ModelNet10、ModelNet40、ScanObjectNN数据集上的总体精度分别达到96.5%、94.7%、81.9%, 比DGCNN分别提高2.9、2.1、3.8个百分点, 参数总量相比DGCNN、LDGCNN、3DGCN模型分别降低52.9%、23.9%、3.3%, 且DMGCN-3D能够保持较高的鲁棒性。

关键词: 点云分类, 可变形三维图卷积, 自适应, 多头自注意力, 轻量级网络

Abstract:

Existing deep learning methods rely on absolute point coordinates when addressing point cloud classification tasks, which encounter the large model complexity problem. To address this challenge, a lightweight point cloud classification network called DMGCN-3D is proposed herein. The adaptive hollow K-Nearest Neighbor (KNN) algorithm is used to construct the graph structure, capture geometric structure information regarding the local wider space, and reduce calculation costs. A deformable 3-Dimensional (3D) graph convolution is constructed, and the learnable direction vector between points is introduced to obtain relative characteristics between points. The displacement and scale invariances of point clouds are guaranteed during the feature extraction process. A multi-head self-attention module is constructed, and the residual structure is combined with Group Shift Attention (GSA) and the Multi-Layer Perceptron (MLP) network. The MLP assists in maintaining the integrity of original point cloud information, and the GSA enables the network to learn the internal autocorrelation of features, which improves feature expression capability and reduces the total number of model parameters. A spatial transformation network combined with the MLP is used to learn point cloud features. Finally, the extracted features are fused to obtain more comprehensive point cloud classification features. The experimental results demonstrate that the overall accuracies of DMGCN-3D on ModelNet10, ModelNet40, and ScanObjectNN are 96.5%, 94.7%, and 81.9%, respectively, which is 2.9, 2.1, and 3.8 percentage points higher than those of the DGCNN. Compared with DGCNN, LDGCNN, and 3DGCN, the total number of parameters is reduced by 52.9%, 23.9%, and 3.3%, respectively. Additionally, high robustness is maintained, which demonstrates an improvement on that of existing advanced methods.

Key words: point cloud classification, deformable 3D graph convolution, self-adaption, multiple head self-attention, lightweight network

蔡俊民, 梁正友, 孙宇, 陈子奥. 基于可变形三维图卷积的轻量级点云分类研究[J]. 计算机工程, 2024, 50(9): 255-265.

CAI Junmin, LIANG Zhengyou, SUN Yu, CHEN Ziao. Research on Lightweight Point Cloud Classification Based on Deformable 3D Graph Convolution[J]. Computer Engineering, 2024, 50(9): 255-265.

https://www.ecice06.com/CN/Y2024/V50/I9/255

图/表 19

图1 STN网络结构

Fig.1 The structure of STN network

图2 DMGCN-3D模型整体框架

Fig.2 Overall framework of DMGCN-3D model

图3 自适应空洞KNN算法

Fig.3 Adaptive hollow KNN algorithm

图4 感受野示意图

Fig.4 Schematic diagram of receptive field

图5 DTGC算子

Fig.5 DTGC operator

图6 MGSA模块的结构

Fig.6 The structure of MGSA module

图7 MGSA模块的头部分支结构

Fig.7 The head branch structure of MGSA module

图8 损失收敛和总体精度的变化曲线

Fig.8 Curve of loss convergence and overall accuracy variation

图9 在ModelNet40上的T-SNE可视化结果

Fig.9 Visualization results of T-SNE on ModelNet40

图10 对点云密度的鲁棒性实验结果

Fig.10 Experimental results on the robustness of point cloud density

图11 对点云变换的鲁棒性实验结果

Fig.11 Experimental results on the robustness of point cloud transformation

参考文献 32

1	FAN T Y, ZHANG R J. Research on automatic lane line extraction method based on onboard lidar point cloud data[C]//Proceedings of the 2nd International Conference on Digital Signal and Computer Communications. Washington D. C., USA: IEEE Press, 2022: 161-169.
2	缪建起, 王宏涛, 田普光. 整合图卷积与PointNet的机载激光雷达点云分类. 激光与光电子学进展, 2022, 59 (22): 328- 334. URL
	MIAO J Q, WANG H T, TIAN P G. Airborne light detection and ranging point cloud classification via graph convolution and PointNet integration. Laser & Optoelectronics Progress, 2022, 59 (22): 328- 334. URL
3	郑维刚, 赵振威, 唐红, 等. 基于三维激光点云的隧道电缆敷设质量参数自动检测方法. 半导体光电, 2023, 44 (3): 460- 466. URL
	ZHENG W G, ZHAO Z W, TANG H, et al. Automatic detection method for tunnel cable laying quality parameters based on three-dimensional laser point cloud. Semiconductor Optoelectronics, 2023, 44 (3): 460- 466. URL
4	李美佳, 于泽宽, 刘晓, 等. 点云算法在医学领域的研究进展. 中国图象图形学报, 2020, 25 (10): 2013- 2023. doi: 10.11834/jig.200253
	LI M J, YU Z K, LIU X, et al. Progress of point cloud algorithm in medical field. Journal of Image and Graphics, 2020, 25 (10): 2013- 2023. doi: 10.11834/jig.200253
5	SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 945-953.
6	MATURANA D, SCHERER S. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington D. C., USA: IEEE Press, 2015: 922-928.
7	CHARLES R Q, HAO S, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1612.00593.
8	QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1706.02413.
9	YAN X, ZHENG C D, LI Z, et al. PointASNL: robust point clouds processing using nonlocal neural networks with adaptive sampling[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 5589-5598.
10	ZHAO H S, JIANG L, FU C W, et al. PointWeb: enhancing local neighborhood features for point cloud processing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5565-5573.
11	LI J, CHEN B M, LEE G H. So-Net: self-organizing network for point cloud analysis[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1803.04249.
12	KOMARICHEV A, ZHONG Z C, HUA J. A-CNN: annularly convolutional neural networks on point clouds[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 7421-7430.
13	LI Y, BU R, SUN M, et al. PointCNN: convolution on X-transformed points[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1801.07791.
14	WU W X, QI Z A, LI F X. PointConv: deep convolutional networks on 3D point clouds[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 9621-9630.
15	LIU Y, FAN B, XIANG S, et al. Relation-shape convolutional neural network for point cloud analysis[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1904.07601.
16	THOMAS H, QI C R, DESCHAUD J E, et al. KPConv: flexible and deformable convolution for point clouds[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6411-6420.
17	HOANG L, LEE S H, LEE E J, et al. GSV-NET: a multi-modal deep learning network for 3D point cloud classification. Applied Sciences, 2022, 12 (1): 483. doi: 10.3390/app12010483
18	SIMONOVSKY M, KOMODAKIS N. Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3693-3702.
19	WANG Y, SUN Y B, LIU Z W, et al. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 2019, 38 (5): 1- 12.
20	ZHANG K G, HAO M, WANG J, et al. Linked dynamic graph CNN: learning on point cloud via linking hierarchical features[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1904.10014.
21	LIN Z H, HUANG S Y, WANG Y C F. Convolution in the cloud: learning deformable kernels in 3D graph convolution networks for point cloud analysis[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 1800-1809.
22	LI R H, LI X Z, HENG P A, et al. PointAugment: an auto-augmentation framework for point cloud classification[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 6378-6387.
23	YANG J C, ZHANG Q, NI B B, et al. Modeling point clouds with self-attention and gumbel subset sampling[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 3323-3332.
24	GUO M H, CAI J X, LIU Z N, et al. PCT: point cloud transformer. Computational Visual Media, 2021, 7 (2): 187- 199. doi: 10.1007/s41095-021-0229-5
25	LIU Y H, TIAN B, LÜ Y S, et al. Point cloud classification using content-based transformer via clustering in feature space. CAA Journal of Automatica Sinica, 2024, 11 (1): 231- 239.
26	HUANG C Q, JIANG F, HUANG Q H, et al. Dual-graph attention convolution network for 3-D point cloud classification. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35 (4): 4813- 4825. doi: 10.1109/TNNLS.2022.3162301
27	李维刚, 陈婷, 田志强. 基于孪生自适应图卷积算法的点云分类与分割. 计算机应用, 2023, 43 (11): 3396- 3402. URL
	LI W G, CHEN T, TIAN Z Q. Point cloud classification and segmentation based on siamese adaptive graph convolution algorithm. Journal of Computer Applications, 2023, 43 (11): 3396- 3402. URL
28	蒋玉英, 陈心雨, 李广明, 等. 图神经网络及其在图像处理领域的研究进展. 计算机工程与应用, 2023, 59 (7): 15- 30. URL
	JIANG Y Y, CHEN X Y, LI G M, et al. Graph neural network and its research progress in field of image processing. Computer Engineering and Applications, 2023, 59 (7): 15- 30. URL
29	张学典, 方慧. BTDGCNN: 面向三维点云拓扑结构的BallTree动态图卷积神经网络. 小型微型计算机系统, 2022, 43 (11): 2342- 2347. URL
	ZHANG X D, FANG H. BTDGCNN: BallTree dynamic graph convolution neural network for 3D point cloud topology. Journal of Chinese Computer Systems, 2022, 43 (11): 2342- 2347. URL
30	ZHANG T, QI G J, XIAO B, et al. Interleaved group convolutions[EB/OL]. [2023-04-05]. https://ieeexplore.ieee.org/document/8237731.
31	WU Z R, SONG S R, KHOSLA A, et al. 3D ShapeNets: a deep representation for volumetric shapes[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 1912-1920.
32	MAATEN L, HINTON G. Visualizing data using T-SNE. Journal of Machine Learning Research, 2008, 9, 2579- 2605.

[1]	李维刚, 厉许昌, 田志强, 李金灵. 基于自蒸馏框架的点云分类及其鲁棒性研究[J]. 计算机工程, 2024, 50(9): 72-81.
[2]	李俊仪, 李向阳, 龙朝勋, 李海燕, 李红松, 余鹏飞. 基于多级区域选择与跨层特征融合的野生菌分类[J]. 计算机工程, 2024, 50(9): 179-188.
[3]	陈瀚, 赵春蕾, 蒋昊达, 王春东. 基于融合模型与语义网络的App用户意图识别研究[J]. 计算机工程, 2024, 50(8): 50-63.
[4]	王富平, 刘鸿玮, 张锲石, 段冠庄. 基于深度特征抑制的遮挡人脸识别网络[J]. 计算机工程, 2024, 50(8): 259-269.
[5]	高爽, 史轶伦, 徐巧枝, 于磊. 基于对比学习的非对称编解码结构的心脏MRI分割研究[J]. 计算机工程, 2024, 50(8): 290-300.
[6]	曾碧卿, 陈鹏飞, 姚勇涛. 融合思维链和低秩自适应微调的方面情感三元组抽取[J]. 计算机工程, 2024, 50(7): 53-62.
[7]	杨郅树, 梁佳楠, 曹永军, 钟震宇, 何永伦. 基于局部分离与多尺度融合的图像超分辨率重建[J]. 计算机工程, 2024, 50(7): 314-323.
[8]	秦媛, 张杭, 朱宏鹏, 李炯, 胡航. 卫星MIMO通信系统抗移动无人机集群干扰的在线BSS算法[J]. 计算机工程, 2024, 50(6): 65-76.
[9]	李田芳, 普园媛, 赵征鹏, 徐丹, 钱文华. 基于CLIP和双空间自适应归一化的图像翻译[J]. 计算机工程, 2024, 50(5): 229-240.
[10]	黄君泽, 吴文渊, 李轶, 石明全, 王正江. 面向动态公交的离散分层记忆粒子群优化算法[J]. 计算机工程, 2024, 50(4): 20-30.
[11]	张毅恒, 刘以安, 宋海凌. 基于增强型龙格库塔优化算法的跳频序列设计[J]. 计算机工程, 2024, 50(4): 267-276.
[12]	李宝莹, 李志淮, 王成爱, 杨锋. 自适应节点规模的区块链分片可扩展模型[J]. 计算机工程, 2024, 50(3): 137-147.
[13]	单永航, 张希, 胡川, 丁涛军, 姚远. 基于集成学习的交通事故严重程度预测研究与应用[J]. 计算机工程, 2024, 50(2): 33-42.
[14]	王丽娟, 邢津萍, 尹明, 郝志峰, 蔡瑞初, 温雯. 基于一致性图的权重自适应多视角谱聚类算法[J]. 计算机工程, 2024, 50(2): 122-131.
[15]	王正家, 胡飞飞, 张成娟, 雷卓, 何涛. 引入轻量级Transformer的自适应窗口立体匹配算法[J]. 计算机工程, 2024, 50(2): 256-265.

选择文件类型/文献管理软件名称

选择包含的内容