Abstract: Multi-view 3D reconstruction aims to reconstruct the 3D shape of a given object from multiple 2D images. However, existing methods usually ignore the learning of both rotation invariance and regional consistency of objects. It is difficult to aggregate features from multiple views accurately, resulting in the loss of fine-grained details in reconstruction results. To address this challenge, a Dual-view Point cloud reconstruction based on Rotation-invariant Regional consistency, called DPR2, is proposed. It takes two RGB images as input and learns the regional consistency across views on the basis of exploring the rotation invariance of object region to promote feature aggregation, reconstructing a refined point cloud. In encoding, a point cloud initialization network is introduced to initialize a rough point cloud for each view. Besides, a region-level rotation-invariant feature extraction network is presented. It captures rotation-invariant features from different regions of the rough point clouds by utilizing the Euclidean distance between points. In decoding, a dual-stage cross-attention mechanism is devised, which learns high-quality region consistency across the point clouds, achieving feature aggregation accurately. Moreover, a point cloud refinement network is developed to refine the rough point cloud as a point cloud with fine-grained details and smooth surfaces using the aggregated features. Extensive experiments on the ShapeNet and Pix3D datasets show that the DPR2 outperforms existing SOTA methods in terms of reconstruction performance. Compared to SOTA methods P2M++ and MVP2M++, the CD metric has improved by 23.62% and 9.06%, respectively.

摘要： 多视图三维重建旨在通过多张二维图像恢复给定对象的三维形状。然而，现有方法忽略学习对象的旋转不变性以及区域一致性，难以准确聚合多视图特征，造成重建结果细节丢失。为了解决该问题，提出了一种基于旋转不变区域一致性的双视图点云重建方法（Dual-view Point cloud reconstruction based on Rotation-invariant Regional consistency），称为DPR2。它以两张RGB图像作为输入，在探索对象区域旋转不变性的基础上，学习跨视图对象的区域一致性促进多视图特征聚合，并重建给定对象的精细点云。编码阶段，首先引入点云初始化网络为每个视图初始化一个粗糙点云。其次，提出区域级旋转不变特征提取网络，通过计算点-点之间的欧氏距离捕捉粗糙点云不同区域的旋转不变特征。解码阶段，设计双阶段交叉注意力机制，它可以构建跨视图点云的高质量区域一致性，从而准确实现多视图特征聚合。另外，开发点云细化网络，利用被聚合的特征，将粗糙点云细化为具有细粒度细节和光滑表面的点云。在ShapeNet和Pix3D数据集上的大量实验表明，所提出的DPR2在重建性能方面优于现有先进方法。与最新方法P2M++和MVP2M++相比，CD指标分别改善了23.62%和9.06%。