Multi-Branch and Multi-Scale Point Cloud Completion Network

doi:10.19678/j.issn.1000-3428.0068960

Abstract

Abstract:

This paper proposes a point cloud completion network based on multi-branch multi-scale feature fusion because the existing networks cannot extract high-quality global and local features of point clouds simultaneously and lose point cloud detail and coordinate information. The novelty of this network is its hierarchical progressive feature extraction and fusion mechanism. In the encoding stage, the proposed network first uses the Joint Feature Extraction Module(JFEM) to perform multi-scale feature learning using the input point cloud data of three different resolutions and successively extracts global features containing rich semantic information and fine local features to maximize the retention of key information. Subsequently, the Detail-Preserving Pooling (DP-Pool) module is used for reducing the dimensions of the features to avoid the loss of detail caused by traditional pooling operations. The multi-branch encoding structure is combined to achieve efficient fusion of global and local features, ensuring that features of different scales can complement each other. In the decoding stage, the network gradually restores the geometric structure of the point cloud via the Point Cloud Reconstruction (PCR) module, uses the multi-branch decoding structure to finely upsample the features at different levels, and generates a high-fidelity, high-density completed point cloud. Experimental results show that the performance of the proposed network is better than those of the top 10 advanced point cloud completion networks and can further improve the quality of point cloud completion.

Key words: point cloud completion, multi-branch, multi-scale, feature fusion, detail preserving

摘要：

现有点云补全网络无法同时提取高质量的点云全局特征和局部特征，丢失点云细节信息与坐标信息。为此，提出一种基于多分支多尺度特征融合的点云补全网络，该网络的核心创新在于分层渐进式特征提取与融合机制。在编码阶段，该网络首先通过联合特征提取模块(JFEM)，对输入的三种不同分辨率的点云数据进行多尺度特征学习，依次提取包含丰富语义信息的全局特征和精细的局部特征，以最大化保留关键信息，然后利用细节保持池化(DP-Pool)模块对特征进行降维，避免传统池化操作造成的细节损失，并结合多分支编码结构实现全局与局部特征的高效融合，确保不同尺度的特征能够互补增强。在解码阶段，该网络通过点云重构(PCR)模块逐步恢复点云的几何结构，并利用多分支解码结构对不同层次的特征进行精细化上采样，最终生成高保真、高密度的补全点云。实验结果表明，所提网络的性能优于目前先进的10种点云补全网络，能进一步提高点云补全质量。

关键词: 点云补全, 多分支, 多尺度, 特征融合, 细节保持

CHEN Xiaolei, WANG Rong. Multi-Branch and Multi-Scale Point Cloud Completion Network[J]. Computer Engineering, 2025, 51(8): 330-340.

陈晓雷, 王荣. 多分支多尺度点云补全网络[J]. 计算机工程, 2025, 51(8): 330-340.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068960

https://www.ecice06.com/EN/Y2025/V51/I8/330

Figures/Tables 14

Fig.1 Overall network structure

Fig.2 JFEM module structure

Fig.3 Structure of DP-Pool module

Fig.4 PCR module structure

Fig.5 Visualization results of different point cloud completion networks

Fig.6 Enlarged visualization results of part of the point cloud completion networks

Fig.7 Visualization results comparison of point cloud completion results of different branches

Fig.8 Point cloud completion results of different feature extraction methods

References 26

1	沙浩, 刘越, 王涌天, 等. 基于二维图像和三维几何约束神经网络的单目室内深度估计方法. 光学学报, 2022, 42(19): 47- 57.
	SHA H, LIU Y, WANG Y T, et al. Monocular indoor depth estimation method based on neural network with constraints on two-dimensional images and three-dimensional geometrically. Acta Optica Sinica, 2022, 42(19): 47- 57.
2	胡春生, 闫小鹏, 魏红星, 等. 基于立体视觉的目标检测与轨迹预测研究综述. 计算机工程与应用, 2022, 58(3): 50- 65.
	HU C S, YAN X P, WEI H X, et al. Survey of target detection and trajectory prediction based on stereo vision. Computer Engineering and Applications, 2022, 58(3): 50- 65.
3	马建红, 王稀瑶, 陈永霞, 等. 自动驾驶中图像与点云融合方法研究综述. 郑州大学学报(理学版), 2022, 54(6): 24- 33.
	MA J H, WANG X Y, CHEN Y X, et al. A review of research on image and point cloud fusion methods in autonomous driving. Journal of Zhengzhou University (Science Edition), 2022, 54(6): 24- 33.
4	张磊, 徐孝彬, 曹晨飞, 等. 基于动态特征剔除的图像与点云融合的机器人位姿估计方法. 中国激光, 2022, 49(6): 58- 69.
	ZHANG L, XU X B, CAO C F, et al. Robot pose estimation method based on image and point cloud fusion with dynamic feature elimination. China Laser, 2022, 49(6): 58- 69.
5	LIU H B, JIA J Y, GONG N Z. Pointguard: provably robust 3D point cloud classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 6186-6195.
6	兰红, 陈浩, 张蒲芬. 集图卷积和三维方向卷积的点云分类分割模型. 计算机工程与应用, 2023, 59(8): 182- 191.
	LAN H, CHEN H, ZHANG P F. Point cloud classification and segmentation model based on graph convolution and 3D direction convolution. Computer Engineering and Applications, 2023, 59(8): 182- 191.
7	景庄伟, 管海燕, 臧玉府, 等. 基于深度学习的点云语义分割研究综述. 计算机科学与探索, 2021, 15(1): 1- 26.
	JING Z W, GUAN H Y, ZANG Y F, et al. A survey of on semantic segmentation of point cloud based on deep learning. Computer Science and Exploration, 2021, 15(1): 1- 26.
8	QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 652-660.
9	QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[EB/OL]. [2023-11-01]. https://arxiv.org/pdf/1706.02413.
10	YANG Y Q, FENG C, SHEN Y R, et al. FoldingNet: point cloud auto-encoder via deep grid deformation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 206-215.
11	YUAN W T, KHOT T, HELD D, et al. PCN: point completion network[C]//Proceedings of 2018 International Conference on 3D Vision (3DV). Washington D. C., USA: IEEE Press, 2018: 728-737.
12	WANG Y, SUN Y B, LIU Z Q, et al. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 2019, 38(5): 1- 12. URL
13	HUANG Z, YU Y, XU J, et al. PF-Net: point fractal network for 3D point cloud completion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 7662-7670.
14	XIANG P, WEN X, LIU Y S, et al. SnowflakeNet: point cloud completion by snowflake point deconvolution with skip-transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 5499-5509.
15	XIANG P, WEN X, LIU Y S, et al. Snowflake point deconvolution for point cloud completion and generation with skip-transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(5): 6320- 6338. URL
16	WEN X, XIANG P, HAN Z, et al. PMP-Net: point cloud completion by learning multi-step point moving paths[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 7443-7452.
17	WEN X, XIANG P, HAN Z, et al. PMP-Net++: point cloud completion by transformer-enhanced multi-step point moving paths. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(1): 852- 867. URL
18	ZHAO H, JIANG L, JIA J, et al. Point transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 16259-16268.
19	PAN L, CHEN X Y, CAI Z Z, et al. Variational relational point completion network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 8524-8533.
20	PAN L, CHEW C M, LEE G H. PointAtrousGraph: deep hierarchical encoder-decoder with point atrous convolution for unorganized 3D points[EB/OL]. [2023-11-01]. https://arxiv.org/abs/1907.09798v2.
21	TCHAPMI L P, KOSARAJU V, REZATOFIGHI H, et al. TopNet: structural point cloud decoder[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 383-392.
22	WANG X G, ANG JR M H, LEE G H. Cascaded refinement network for point cloud completion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 790-799.
23	PAN L. ECG: edge-aware point cloud completion with graph convolution. IEEE Robotics and Automation Letters, 2020, 5(3): 4392- 4398. URL
24	LIU M H, SHENG L, YANG S, et al. Morphing and sampling network for dense point cloud completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence. [S. l. ]: AAAI Press, 2020: 11596-11603.
25	XIE H Z, YAO H X, ZHOU S C, et al. GRNet: gridding residual network for dense point cloud completion[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 365-381.
26	YU X M, RAO Y M, WANG Z Y, et al. PoinTr: diverse point cloud completion with adaptive geometry-aware transformers[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 1-9.

[1]	MA Manfu, CHEN Jiahao, LI Yong, ZHANG Cong. Multi-Feature Fusion Rumor Detection Model MFLAN Based on Improved Graph Attention Network [J]. Computer Engineering, 2025, 51(8): 181-189.
[2]	YAN Jianhong, LIU Zhiyan, WANG Zhen. Multi-Scale Convolutional Vehicle Trajectory Prediction Integrating Spatiotemporal Attention Mechanism [J]. Computer Engineering, 2025, 51(8): 406-414.
[3]	PENG Juhong, ZHANG Chi, GAO Qian, ZHANG Guangming, TAN Donghua, ZHAO Mingjun. Steel Defect Detection Based on Improved YOLOv8 Algorithm [J]. Computer Engineering, 2025, 51(7): 152-160.
[4]	LIU Chunxia, MENG Jixing, PAN Lihu, GONG Dali. Remote Sensing Small-Target Detection Method with Fusion of RGB and IR Images [J]. Computer Engineering, 2025, 51(7): 326-338.
[5]	WANG Chao, WANG Ting, WANG Shaojun, YANG Wankou. Multi-Scale Image Dehazing Algorithm Based on Improved AOD-Net [J]. Computer Engineering, 2025, 51(7): 305-313.
[6]	LUAN Mengna, ZHENG Qiumei, WANG Fenghua. Real-time Traffic Sign Detection Algorithm Based on DMC-YOLO [J]. Computer Engineering, 2025, 51(7): 90-99.
[7]	ZHOU Sha, CHE Shengbing, KAO Youchen, ZHANG Xu, GUO Shenyi. Network Intrusion Detection Based on Feature Selection and Spatio-Temporal Features [J]. Computer Engineering, 2025, 51(7): 223-231.
[8]	SHA Yuyang, LU Jingtao, DU Haofan, ZHAI Xiaobing, MENG Weiyu, LIAN Xu, LUO Gang, LI Kefeng. Lightweight Road Image Segmentation Algorithm Based on Multi-Scale Feature Fusion for Blind Guiding Scenarios [J]. Computer Engineering, 2025, 51(7): 314-325.
[9]	LI Baiya. CNN-Transformer-Based Lesion and Organ Segmentation Network for Electronic Laryngoscope [J]. Computer Engineering, 2025, 51(6): 327-337.
[10]	ZHENG Cheng, LI Pengfei. Text Classification Based on Feature Fusion of Dual Hypergraph Neural Networks [J]. Computer Engineering, 2025, 51(6): 127-135.
[11]	LIU Kai, REN Hongyi, LI Ying, JI Yi, LIU Chunping. Medical Visual Question Answering Based on Cross-Modal Attention Feature Enhancement [J]. Computer Engineering, 2025, 51(6): 49-56.
[12]	LI Yi, XU Huiying, ZHU Xinzhong, HUANG Xiao, WANG Shumeng, LI Xiyu. Mask-YOLO: Improved Mask Detection Algorithm Based on YOLOv5n [J]. Computer Engineering, 2025, 51(6): 297-310.
[13]	CAO Bei, ZHAO Kui. Dual Emotion and Multi-feature Fusion Based Fake News Detection [J]. Computer Engineering, 2025, 51(6): 193-203.
[14]	ZHANG Yimin, HUANG Xiaoying, HUANG Zhengyang, YANG Chaoxiang, WAN Yongjing, JIANG Cuiling. Application of Bionic Design Algorithms Based on Latent Spatial Multi-Scale Fusion of Cross-Domain Images [J]. Computer Engineering, 2025, 51(5): 266-278.
[15]	DU Chenyang, ZHANG Xueying, HUANG Lixia, LI Juan. Multi-Feature Speech Emotion Recognition Based on Improved Efficient Channel Attention Mechanism [J]. Computer Engineering, 2025, 51(4): 97-106.

Please choose a citation manager

Content to export