基于双分支卷积和深度插值的点云表面重建

doi:10.19678/j.issn.1000-3428.0069420

摘要/Abstract

摘要：

随着近年来深度学习的快速发展，基于深度学习重建稀疏、含有噪声的低质量点云表面成为当前的研究热点。目前已有的点云表面重建模型还存在难以重建复杂场景、局部细节重建不完整以及重建效率低等问题。为了进一步提高点云表面重建的效果，结合卷积占用网络模型，提出一种基于双分支卷积和深度插值的点云表面重建模型。首先，使用PointNet网络和双分支卷积构建的融合卷积编码模块进行特征提取，其中双分支卷积将点分支提取的点特征自适应地融入到体素分支的体积特征中，为体积特征提供更细粒的局部信息；然后，结合邻居点特征，通过一个多头注意力网络生成查询点特征，构建深度插值特征模块，使得用于特征解码的全连接层(FC)网络预测查询点的空间位置更加准确；最后，基于移动立方体(MC)算法提取高质量的网格表面模型。在对象级数据集ShapeNet以及场景级数据集Synthetic Rooms上的实验结果表明，所提模型在交并比(IoU)指标上分别达到了0.931和0.910，优于POCONet、ConvONet、DP-ConvONet等对比模型，在Synthetic Rooms上的平均重建时间上较POCONet大幅减少，且在视觉上表现出了良好的效果；在对象级数据集ABC上体现了模型优越的泛化性能，证明了所提模型的有效性。

关键词: 表面重建, 双分支卷积, 深度插值, 点云, 深度学习, 移动立方体算法

Abstract:

With the rapid development of deep learning in recent years, the reconstruction of sparse and noisy low-quality point cloud surfaces based on deep learning has attracted increasing attention from researchers. Existing point cloud surface reconstruction models present limitations such as difficulty in reconstructing complex scenes, incomplete local detail reconstruction, and low reconstruction efficiency. To address these limitations, this paper proposes a point cloud surface reconstruction model based on dual-branch convolution and deep interpolation combined with a convolution occupancy network model. First, a fusion convolution coding module constructed using a PointNet network and dual-branch convolution is used for feature extraction. The dual-branch convolution adaptively integrates point features extracted by the point branch into the volume feature of the voxel branch to provide more fine-grained local information for the volume feature. Subsequently, query point features in combination with the characteristics of neighbor points are generated through a multi-head attention network. Further, a deep interpolation feature module is constructed to improve the accuracy of the Fully Connected (FC) layer network for feature decoding in predicting the spatial location of query points. Finally, a high-quality mesh surface model is extracted based on the Marching Cubes (MC) algorithm. Results of experiments on the object-level dataset ShapeNet and the scene-level dataset Synthetic Rooms show that the proposed model achieves an Intersection over Union (IoU) metric of 0.931 and 0.910 respectively. It outperforms the comparative models such as POCONet, ConvONet, and DP-ConvONet. On the Synthetic Rooms dataset, the average reconstruction time of the proposed model is significantly reduced compared to that of the POCONet model, and it also demonstrates good visual performance. On the object-level dataset ABC, the proposed model exhibits superior generalization performance, which proves the effectiveness of the proposed model.

Key words: surface reconstruction, dual-branch convolution, deep interpolation, point cloud, deep learning, Marching Cubes (MC) algorithm

孟波, 史旭华, 张彬. 基于双分支卷积和深度插值的点云表面重建[J]. 计算机工程, 2025, 51(7): 119-126.

MENG Bo, SHI Xuhua, ZHANG Bin. Point Cloud Surface Reconstruction Based on Dual-Branch Convolution and Deep Interpolation[J]. Computer Engineering, 2025, 51(7): 119-126.

https://www.ecice06.com/CN/Y2025/V51/I7/119

图/表 12

图1 重建模型整体流程

Fig.1 The overall process of reconstruction model

图2 融合卷积编码模块

Fig.2 Fusion convolution coding module

图3 双分支卷积

Fig.3 Dual-branch convolution

图4 深度插值特征模块

Fig.4 Deep interpolation feature module

图5 在ShapeNet数据集上的重建结果比较

Fig.5 Comparison of reconstruction results on ShapeNet dataset

图6 在Synthetic Rooms数据集上的重建结果比较

Fig.6 Comparison of reconstruction results on Synthetic Rooms dataset

图7 在ABC数据集上的重建结果比较

Fig.7 Comparison of reconstruction results on ABC dataset

参考文献 32

1	BERNARDINI F , MITTLEMAN J , RUSHMEIER H , et al. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics, 1999, 5 (4): 349- 359. doi: 10.1109/2945.817351
2	GROUEIX T, FISHER M, KIM V G, et al. A papier-Mache approach to learning 3D surface generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2018: 216-224.
3	LIAO Y Y, DONNE S, GEIGER A. Deep marching cubes: learning explicit surface representations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2018: 2916-2925.
4	KAZHDAN M , HOPPE H . Screened poisson surface reconstruction. ACM Transactions on Graphics, 2013, 32 (3): 1- 13.
5	CHEN Z Q, ZHANG H. Learning implicit fields for generative shape modeling[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2019: 5939-5948.
6	梁春阳, 唐红梅, 席建锐, 等. 基于稀疏特征改进的单视图表面重建. 计算机应用研究, 2023, 40 (3): 925-931, 937. doi: 10.19734/j.issn.1001-3695.2022.06.0320
	LIANG C Y , TANG H M , XI J R , et al. 925-931, 937. Improved single-view surface reconstruction based on sparse feature. Application Research of Computers, 2023, 40 (3): 925- 931. doi: 10.19734/j.issn.1001-3695.2022.06.0320
7	张冀, 郑传哲. 基于多尺度CNN-RNN的单图三维重建网络. 计算机应用研究, 2020, 37 (11): 3487- 3491. doi: 10.19734/j.issn.1001-3695.2019.08.0251
	ZHANG J , ZHENG C Z . 3D reconstruction network based on multi-scale CNN-RNN. Application Research of Computers, 2020, 37 (11): 3487- 3491. doi: 10.19734/j.issn.1001-3695.2019.08.0251
8	MESCHEDER L, OECHSLE M, NIEMEYER M, et al. Occupancy networks: learning 3D reconstruction in function space[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2019: 4460-4470.
9	PARK J J, FLORENCE P, STRAUB J, et al. DeepSDF: learning continuous signed distance functions for shape representation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2019: 165-174.
10	席建锐, 唐红梅, 梁春阳, 等. 基于改进隐函数的点云物体重建. 计算机工程, 2023, 49 (7): 214- 222. URL
	XI J R , TANG H M , LIANG C Y , et al. Point cloud object reconstruction based on improved implicit function. Computer Engineering, 2023, 49 (7): 214- 222. URL
11	GROPP A, YARIV L, HAIM N, et al. Implicit geometric regularization for learning shapes[C]//Proceedings of International Conference on Machine Learning (PMLR). New York, USA: ACM Press, 2020: 3789-3799.
12	ZHOU J , MA B , LIU Y S , et al. Learning consistency-aware unsigned distance functions progressively from raw point clouds. Advances in Neural Information Processing Systems, 2022, 35, 16481- 16494.
13	CHABRA R, LENSSEN J E, ILG E, et al. Deep local shapes: learning local SDF priors for detailed 3D reconstruction[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2020: 608-625.
14	JIANG C Y, SUD A, MAKADIA A, et al. Local implicit grid representations for 3D scenes[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2020: 6001-6010.
15	PENG S Y, NIEMEYER M, MESCHEDER L, et al. Convolutional occupancy networks[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2020: 523-540.
16	LIONAR S, EMTSEV D, SVILARKOVIC D, et al. Dynamic plane convolutional occupancy networks[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV). Washington D. C., USA: IEEE Press, 2021: 1829-1838.
17	MAO Q , LI R , ZHU Y , et al. Enhancing 3D-2D representations for convolution occupancy networks. Pattern Recognition, 2023, 134, 109097.
18	WILLIAMS F, GOJCIC Z, KHAMIS S, et al. Neural fields as learnable kernels for 3D reconstruction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2022: 18500-18510.
19	CHIBANE J, ALLDIECK T, PONS-MOLL G. Implicit functions in feature space for 3D shape reconstruction and completion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2020: 6970-6981.
20	CHIBANE J, MIR A, PONS-MOLL G. Neural unsigned distance fields for implicit function learning[EB/OL]. [2023-12-25]. https://arxiv.org/abs/2010.13938v1.
21	YE J L, CHEN Y T, WANG N Y, et al. GIFS: neural implicit function for general shape representation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2022: 12829-12839.
22	CHEN W K, LIN C, LI W Y, et al. 3PSDF: three-pole signed distance function for learning surfaces with arbitrary topologies[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2022: 18522-18531.
23	BOULCH A, MARLET R. POCO: point convolution for surface reconstruction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2022: 6302-6314.
24	JIANG H Y, CAI J F, ZHENG J M, et al. Neighborhood-based neural implicit reconstruction from point clouds[C]// Proceedings of the International Conference on 3D Vision (3DV). Washington D. C., USA: IEEE Press, 2021: 1259-1268.
25	GIEBENHAIN S, GOLDLUCKE B. AIR-nets: an attention-based framework for locally conditioned implicit representations[C]//Proceedings of the International Conference on 3D Vision (3DV). Washington D. C., USA: IEEE Press, 2021: 1054-1064.
26	ERLER P, GUERRERO P, OHRHALLINGER S, et al. Points2Surf learning implicit surfaces from point clouds[C]// Proceedings of the 16th European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2020: 108-124.
27	LORENSEN W E , CLINE H E . Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics, 1987, 21 (4): 163- 169.
28	CHARLES R Q, HAO S, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2017: 652-660.
29	ÇIÇEK Ö, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]// Proceedings of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). Berlin, Germany: Springer, 2016: 424-432.
30	LIU Z, TANG H, LIN Y, et al. Point-voxel CNN for efficient 3D deep learning[EB/OL]. [2023-12-25]. https://arxiv.org/pdf/1907.03739.
31	CHANG A X, FUNKHOUSER T, GUIBAS L, et al. ShapeNet: an information-rich 3D model repository[EB/OL]. [2023-12-25]. https://arxiv.org/abs/1512.03012v1.
32	KOCH S, MATVEEV A, JIANG Z S, et al. ABC: a big CAD model dataset for geometric deep learning[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2019: 9601-9611.

[1]	沙宇洋, 陆京涛, 杜浩凡, 翟小兵, 孟维宇, 廉旭, 罗刚, 李克峰. 适用于导盲场景的多尺度特征融合轻量化道路图像分割算法[J]. 计算机工程, 2025, 51(7): 314-325.
[2]	欧阳昱中, 韩锐, 刘驰. 边缘侧领域自适应中长尾视觉识别技术研究[J]. 计算机工程, 2025, 51(7): 171-179.
[3]	余鹏, 杨佳琦, 陈欣然, 贺超波. 基于二部图对比学习的特征增强推荐算法[J]. 计算机工程, 2025, 51(7): 100-110.
[4]	周莎, 车生兵, 考友琛, 张旭, 郭甚驿. 基于特征选择和时空特征的网络入侵检测[J]. 计算机工程, 2025, 51(7): 223-231.
[5]	李姜辛, 王鹏, 汪卫. 多机理指导的深度学习工业时序预测框架[J]. 计算机工程, 2025, 51(7): 47-58.
[6]	周哲臣, 胡冀苏, 钱旭升, 郑毅, 戴亚康, 周志勇. 基于查询自适应双层自注意力机制的MRI脑组织分割[J]. 计算机工程, 2025, 51(7): 294-304.
[7]	秦永旺, 张洋, 胡星, 刘胜, 李少青. 基于图注意力网络的门级网表功能识别[J]. 计算机工程, 2025, 51(6): 29-37.
[8]	陈思帆, 杨家志, 黄琳, 吕志玮, 沈露. 融合可变形核和自注意力的点云分类分割边卷积网络[J]. 计算机工程, 2025, 51(6): 146-154.
[9]	廖丁丁, 刘俊峰, 曾君, 邱晓欢. 一种基于块平均正交权重修正的连续学习算法[J]. 计算机工程, 2025, 51(6): 57-64.
[10]	王培吉, 邹承明. 基于向量转换的卷积计算优化方法[J]. 计算机工程, 2025, 51(6): 74-82.
[11]	曹蓓, 赵奎. 基于双重情感和多特征融合的虚假新闻检测[J]. 计算机工程, 2025, 51(6): 193-203.
[12]	庞鑫, 葛凤培, 李艳玲. 声景识音：数字化时代声学场景分类的探索与前沿[J]. 计算机工程, 2025, 51(6): 1-19.
[13]	郝志峰, 黎阳霖, 许柏炎, 蔡瑞初. 面向跨域自然语言生成SQL语句的超图神经网络[J]. 计算机工程, 2025, 51(5): 114-123.
[14]	魏铭康, 李嘉楠, 韩林, 高伟, 赵荣彩, 王洪生. 面向深度学习编译器的多粒度量化框架支持与优化[J]. 计算机工程, 2025, 51(5): 62-72.
[15]	赵瑶谦, 滕奇志, 何小海, 税爱, 陈洪刚. 基于自注意力特征蒸馏的轻量级图像超分辨率重建[J]. 计算机工程, 2025, 51(5): 257-265.

选择文件类型/文献管理软件名称

选择包含的内容