基于空间可变形Transformer的三维点云配准方法

doi:10.19678/j.issn.1000-3428.0067566

摘要/Abstract

摘要：

针对低重叠场景下点云配准方法鲁棒性差、配准精度低的问题，提出一种基于空间可变形Transformer(SDT)的三维点云配准方法。设计多级分辨率特征的提取与融合方法，显式计算点云的局部空间关系。利用SDT模块增强点云空间特征的表达能力，聚合局部与全局的特征得到特征矩阵。计算两个特征矩阵的相似度矩阵并额外地为其添加边缘松弛块，有效降低了不可行匹配对配准鲁棒性的影响，同时对相似度矩阵进行归一化等计算得到软对应置信度矩阵，根据预测的对应点空间特征是否一致来寻找点云在低重叠场景下更精确的对应关系，使用直接定义在对应关系上的损失来训练网络，将软对应关系转换为一对一的硬匹配关系，最终通过随机抽样一致性刚性变换求解器执行配准。实验结果表明，在重叠率低于30%的3DLoMatch场景中，该方法的特征匹配召回率和配准召回率相比于高度关注重叠区域的成对点云配准等方法至少提高了3.7和3.9个百分点，并且具有较强的鲁棒性。

关键词: 低重叠率, 多特征融合, 可变形自注意力, 边缘松弛块, 重叠对应预测

Abstract:

Aiming at the problem of poor robustness and low registration accuracy of point cloud registration algorithms in low overlap scenarios, this study proposes a Three-Dimensional (3D) point cloud registration method based on a Spatial Deformable Transformer (SDT). A multi-level resolution feature extraction and fusion method is designed to explicitly compute the local spatial relationships of point clouds. The SDT module is used to enhance the expressive power of the spatial features of the point cloud, and local and global features are aggregated to obtain the feature matrix. This method computes the similarity matrix of the two feature matrices and adds an edge slack block to it, effectively reducing the impact of infeasible matching on the robustness of registration. Additionally, the similarity matrix is normalized and calculated to obtain the soft correspondence confidence matrix, and the more accurate correspondence of the point cloud in the low overlap scenario is determined according to whether the spatial features of the corresponding points are consistent. The loss defined directly on the correspondence is used to train the network to convert the soft correspondence into a one-to-one hard matching relation, and finally, the registration is performed by the RANdom SAmple Consensus (RANSAC) rigid transformation solver. Experimental results demonstrate that the Feature Matching Recall (FMR) and Registration Recall (RR) of the proposed method are at least 3.7 and 3.9 percentage points higher than those of Pairwise pointcloud REgistration with Deep Attention To the Overlap Region (PREDATOR), respectively, including other methods in the 3DLoMatch scenario with less than 30% overlap, with strong robustness.

Key words: low overlap rate, multi-feature fusion, deformable self-attention, edge slack block, overlap correspondence prediction

谢帅康, 熊风光, 朱新杰, 宋宁栋, 李文清, 王廷凤. 基于空间可变形Transformer的三维点云配准方法[J]. 计算机工程, 2024, 50(3): 224-232.

Shuaikang XIE, Fengguang XIONG, Xinjie ZHU, Ningdong SONG, Wenqing LI, Tingfeng WANG. Three-Dimensional Point Cloud Registration Method Based on Spatial Deformable Transformer[J]. Computer Engineering, 2024, 50(3): 224-232.

http://www.ecice06.com/CN/Y2024/V50/I3/224

图/表 13

图1 三维点云配准网络模型结构

Fig.1 Structure of 3D point cloud registration network model

图2 特征提取网络结构

Fig.2 Feature extraction network structure

图3 局部空间关系

Fig.3 Local spatial relation

图4 可变形自注意力计算过程

Fig.4 Process of deformable self-attention calculation

图5 边缘松弛块示意图

Fig.5 Schematic diagram of edge slack block

图6 3DMatch和3DLoMatch数据集下的配准结果

Fig.6 Registration results under 3DMatch and 3DLoMatch datasets

图7 ModelNet40和ModelLoNet40数据集下的配准结果

Fig.7 Registration results under ModelNet40 and ModelLoNet40 datasets

参考文献 31

1	吴海滨, 徐若彤, 王爱丽, 等. 基于计算机视觉的人体内腔三维重建技术综述. 计算机工程, 2021, 47(10): 1- 15. doi: 10.3778/j.issn.1002-8331.2101-0402
	WU H B, XU R T, WANG A L, et al. Overview of 3D human lumen reconstruction technology based on computer vision. Computer Engineering, 2021, 47(10): 1- 15. doi: 10.3778/j.issn.1002-8331.2101-0402
2	张宇翔, 任爽. 定位技术在虚拟现实中的应用综述. 计算机科学, 2021, 48(1): 308- 318. URL
	ZHANG Y X, REN S. Overview of application of positioning technology in virtual reality. Computer Science, 2021, 48(1): 308- 318. URL
3	卞殷旭, 邢涛, 邓伟杰, 等. 基于深度学习的色彩迁移生物医学成像技术. 红外与激光工程, 2022, 51(2): 331- 348. URL
	BIAN Y X, XING T, DENG W J, et al. Deep learning-based color transfer biomedical imaging technology. Infrared and Laser Engineering, 2022, 51(2): 331- 348. URL
4	BESL P J, MCKAY N D. Method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992,(2): 239- 256.
5	YANG J L, LI H D, JIA Y D. Go-ICP: solving 3D registration efficiently and globally optimally[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2013: 1457-1464.
6	CHOY C, PARK J, KOLTUN V. Fully convolutional geometric features[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 8958-8966.
7	AO S, HU Q Y, YANG B, et al. SpinNet: learning a general surface descriptor for 3D point cloud registration[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 11753-11762.
8	BAI X Y, LUO Z X, ZHOU L, et al. D3Feat: joint learning of dense detection and description of 3D local features[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 6359-6367.
9	THOMAS H, QI C R, DESCHAUD J E, et al. KPConv: flexible and deformable convolution for point clouds[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6411-6420.
10	HUANG S Y, GOJCIC Z, USVYATSOV M, et al. PREDATOR: registration of 3D point clouds with low overlap[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 4267-4276.
11	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.
12	LI Y, HARADA T. Lepard: learning partial point cloud matching in rigid and deformable scenes[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 5554-5564.
13	SINKHORN R, KNOPP P. Concerning nonnegative matrices and doubly stochastic matrices. Pacific Journal of Mathematics, 1967, 21(2): 343- 348. doi: 10.2140/pjm.1967.21.343
14	KUHN H W. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 1955, 2(1/2): 83- 97.
15	FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981, 24(6): 381- 395. doi: 10.1145/358669.358692
16	XU M T, DING R Y, ZHAO H S, et al. PAConv: position adaptive convolution with dynamic kernel assembling on point clouds[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 3173-3182.
17	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
18	QIN Z, YU H, WANG C, et al. Geometric Transformer for fast and robust point cloud registration[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 11143-11152.
19	ZHU X, SU W, LU L, et al. Deformable DETR: deformable Transformers for end-to-end object detection[EB/OL]. [2023-04-11]. https://arxiv.org/abs/2010.04159.
20	XIA Z F, PAN X R, SONG S J, et al. Vision Transformer with deformable attention[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 4794-4803.
21	LIU Z, HU H, LIN Y T, et al. Swin Transformer V2: scaling up capacity and resolution[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 12009-12019.
22	LI J X, LEE G H. USIP: unsupervised stable interest point detection from 3D point clouds[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 361-370.
23	LI J X, CHEN B M, LEE G H. SO-Net: self-organizing network for point cloud analysis[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 9397-9406.
24	YANG Y Q, FENG C, SHEN Y R, et al. FoldingNet: point cloud auto-encoder via deep grid deformation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 206-215.
25	YEW Z J, LEE G H. REGTR: end-to-end point cloud correspondences with transformers[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 6677-6686.
26	WANG N. The use of bilinear interpolation filter to remove image noise. Journal of Physics: Conference Series, 2022, 2303(1): 012089. doi: 10.1088/1742-6596/2303/1/012089
27	CHEN Z L, CHEN H H, GONG L N, et al. UTOPIC: uncertainty-aware overlap prediction network for partial point cloud registration. Computer Graphics Forum, 2022, 41(7): 87- 98. doi: 10.1111/cgf.14659
28	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2980-2988.
29	YU H, LI F, SALEH M, et al. CoFiNet: Reliable coarse-to-fine correspondences for robust pointcloud registration[EB/OL]. [2023-04-11]. https://arxiv.org/abs/2110.14076.
30	WANG Y, SOLOMON J. Deep closest point: learning representations for point cloud registration[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 3523-3532.
31	YEW Z J, LEE G H. RPM-Net: robust point matching using learned features[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11824-11833.

[1]	陈文轩, 曾碧, 郭植星. 融合多特征与语义图卷积网络的摔倒检测方法[J]. 计算机工程, 2023, 49(5): 277-285,294.
[2]	张博熠, 者甜甜, 赵新旭, 刘庆华, 王家晨. 基于眼嘴状态识别网络的疲劳驾驶检测[J]. 计算机工程, 2023, 49(5): 310-320.
[3]	霍跃华, 赵法起. 基于Stacking与多特征融合的加密恶意流量检测[J]. 计算机工程, 2023, 49(5): 165-172,180.
[4]	王畅, 李雷孝, 杨艳艳. 基于面部多特征融合的疲劳驾驶检测综述[J]. 计算机工程, 2023, 49(11): 1-12.
[5]	周海赟, 项学智, 王馨遥, 任文凯. 多特征融合的端到端链式行人多目标跟踪网络[J]. 计算机工程, 2022, 48(9): 305-313.
[6]	张会云, 黄鹤鸣. 基于多操作网络的图式多域语音情感识别研究[J]. 计算机工程, 2022, 48(7): 59-65.
[7]	谭梦婕,吕鑫,陶飞飞. 基于多特征融合的财经新闻话题检测研究[J]. 计算机工程, 2019, 45(3): 293-299,308.
[8]	周明珠,周治平. 结合运动特征的目标跟踪方法[J]. 计算机工程, 2015, 41(3): 242-246.
[9]	许庆勇,江顺亮,黄伟,李菁,徐少平,叶发茂. 基于多特征融合的深度置信网络图像分类算法[J]. 计算机工程, 2015, 41(11): 245-252.
[10]	李星，郭晓松，郭君斌. 基于多特征融合的前向车辆检测方法[J]. 计算机工程, 2014, 40(2): 203-207.
[11]	刘锴,戴平阳,江晓莲,李翠华. 基于排序支持向量机的多特征融合目标跟踪算法[J]. 计算机工程, 2014, 40(11): 42-45,55.
[12]	于金霞,乔楠. 基于多特征融合与均值偏移的粒子滤波跟踪算法[J]. 计算机工程, 2014, 40(11): 14-17.
[13]	师庆敏, 许耀华, 胡艳军, 吴小培. 基于多级颜色累积和纹理融合的目标跟踪算法[J]. 计算机工程, 2013, 39(7): 209-213.
[14]	余胜, 曾接贤, 谢莉. 基于多特征融合的图像检索算法[J]. 计算机工程, 2012, 38(24): 216-219.
[15]	卞志国, 姚源源. 一种多特征融合的红外目标跟踪算法[J]. 计算机工程, 2012, 38(21): 189-192,196.

选择文件类型/文献管理软件名称

选择包含的内容