Neural Implicit Surface Reconstruction Method Based on Multi-View Mixed Consistency Constraints

doi:10.19678/j.issn.1000-3428.0252071

Abstract

Abstract:

Multi-view 3D reconstruction based on neural implicit surface learning includes inherent ambiguities in representing the geometric shape and appearance of complex objects. Consequently, the fine geometric details of an object are prone to being lost in sparse texture areas, boundaries, and large smooth surfaces, making accurate recovery difficult. To address this issue, this study proposes a novel neural implicit surface reconstruction method based on multi-view mixed consistency constraints. This method uses Multi-View Stereo (MVS), multi-view photometric consistency, feature consistency, and volume rendering techniques to optimize the implicit surface representation, enabling the reconstruction of object models with fine geometric details. First, a dense point generation module based on MVS is proposed to supplement detail information in the sparse texture areas and boundaries of the object surface, achieving multi-view geometric optimization of the object surface. Second, a multi-view mixed consistency constraints module is introduced, which uses the Signed Distance Function (SDF) to locate the zero-level set. It applies multi-view photometric consistency constraints to impose geometric constraints on the smooth regions of the object, supervising the extracted implicit surface. Additionally, multi-view feature consistency constraints are applied to surface points at the zero-crossing of the linearly interpolated SDF, compensating for pixel matching errors in texture-sparse or structurally complex regions, thereby optimizing the object reconstruction model. Finally, volume rendering technology is applied to produce high-quality image renderings from the implicit SDF, enabling precise surface reconstruction of objects. Experimental results show that, compared to methods such as Colmap, the proposed method increases the Peak Signal-to-Noise Ratio (PSNR) by over 40.3% on the DTU dataset and successfully enables accurate surface reconstruction of the objects.

Key words: neural implicit surface reconstruction, Multi-View Stereo (MVS), Signed Distance Function (SDF), multi-view mixed consistency, volume rendering

摘要：

在基于神经隐式表面学习的多视图三维重建过程中, 复杂物体的几何形状和外观表示存在潜在的模糊性。因此, 物体的几何细节信息在纹理稀疏区域、边界区域与较大光滑区域中容易丢失, 难以精确恢复。为解决这个问题, 提出一种基于多视图混合一致性约束的神经隐式表面重建方法。该方法采用多视图立体匹配(MVS)、多视图光度一致性与特征一致性、体渲染技术来优化隐式表面表示, 从而重建具有精细几何细节的复杂物体模型。首先, 提出一个基于MVS的稠密点生成模块, 通过MVS生成稠密点, 来补充物体表面纹理稀疏区域与边界区域的细节信息, 从而实现物体表面的多视图几何优化。其次, 提出多视图混合一致性约束模块, 通过符号距离函数(SDF)定位零水平集, 利用多视图光度一致性约束来对物体光滑区域进行几何约束, 监督所提取的隐式表面, 并对经过线性插值的SDF过零处的表面点应用多视图特征一致性约束, 弥补纹理稀疏区域或结构复杂区域像素匹配的误差, 从而优化物体重建模型。最后, 通过应用体渲染技术, 利用隐式的SDF得出高质量的图像渲染, 以实现复杂物体的精确表面重建。实验结果表明, 在DTU数据集中, 相比于Colmap等方法, 所提方法峰值信噪比(PSNR)提升了40.3%以上, 实现了物体表面的精确重建。

关键词: 神经隐式表面重建, 多视图立体匹配, 符号距离函数, 多视图混合一致性, 体渲染

ZHU Wenqian, SONG Lijuan, GUO Xinru, MA Zirui. Neural Implicit Surface Reconstruction Method Based on Multi-View Mixed Consistency Constraints[J]. Computer Engineering, 2026, 52(4): 239-251.

朱文倩, 宋丽娟, 郭新茹, 马子睿. 基于多视图混合一致性约束的神经隐式表面重建方法[J]. 计算机工程, 2026, 52(4): 239-251.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0252071

https://www.ecice06.com/EN/Y2026/V52/I4/239

Figures/Tables 11

Fig.1 Overview of the proposed network architecture

Fig.2 The acquisition process of dense points

Fig.3 The visibility-based fusion algorithm noise reduction effect

Fig.4 Surface reconstruction comparison on the DTU dataset

Fig.5 Surface reconstruction comparison on the BlendedMVS dataset

Fig.6 Qualitative results of ablation study

References 39

1	KAZHDAN M , HOPPE H . Screened Poisson surface reconstruction. ACM Transactions on Graphics, 2013, 32 (3): 1- 13.
2	XU Q S, TAO W B. Multi-scale geometric consistency guided multi-view stereo[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 5478-5487.
3	CAO C J, REN X L, FU Y W. MVSFormer: learning robust image representations via transformers and temperature-based depth for multi-view stereo[EB/OL]. [2025-01-02]. https://arxiv.org/pdf/2208.02541.
4	GOESELE M, CURLESS B, SEITZ S M. Multi-view stereo revisited[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2006: 2402-2409.
5	FURUKAWA Y , HERNÁNDEZ C . Multi-view stereo: a tutorial. Foundations and Trends^Ⓡ in Computer Graphics and Vision, 2015, 9 (1/2): 1- 148.
6	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 405-421.
7	PARK J J, FLORENCE P, STRAUB J, et al. DeepSDF: learning continuous signed distance functions for shape representation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 165-174.
8	WANG P, LIU L J, LIU Y, et al. NeuS: learning neural implicit surfaces by volume rendering for multi-view reconstruction[EB/OL]. [2025-01-02]. https://arxiv.org/pdf/2106.10689.
9	CHEN D C, ZHANG P, FELDMANN I, et al. Recovering fine details for neural implicit surface reconstruction[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, USA: IEEE Press, 2023: 4319-4328.
10	ZHANG J Y, YAO Y, QUAN L. Learning signed distance field for multi-view surface reconstruction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal, Canada: IEEE Press, 2021: 6505-6514.
11	FU Q , XU Q , ONG Y S , et al. Geo-Neus: geometry-consistent neural implicit surfaces learning for multi-view reconstruction. Advances in Neural Information Processing Systems, 2022, 35, 3403- 3416.
12	蒲建鑫, 宋方伟, 冷齐齐. 基于SFM算法的三维重建关键技术研究. 电子技术, 2021, 50 (6): 36- 37.
	PU J X , SONG F W , LENG Q Q . Study on key technologies of 3D reconstruction based on SFM algorithm. Electronic Technology, 2021, 50 (6): 36- 37.
13	HUANG H , WU Y L , ZHOU J S , et al. NeuSurf: on-surface priors for neural surface reconstruction from sparse input views. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38 (3): 2312- 2320. doi: 10.1609/aaai.v38i3.28005
14	WANG W , GAO F , SHEN Y . Res-NeuS: deep residuals and neural implicit surface learning for multi-view reconstruction. Sensors (Basel), 2024, 24 (3): 881. doi: 10.3390/s24030881
15	SEITZ S M, CURLESS B, DIEBEL J, et al. A comparison and evaluation of multi-view stereo reconstruction algorithms[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE Press, 2006: 519-528.
16	JIANG Y Y , SONG L J . 3D surface reconstruction based on dynamic graph convolutional occupancy network. International Journal of Pattern Recognition and Artificial Intelligence, 2023, 37 (14): 2354022. doi: 10.1142/S0218001423540228
17	JI M Q, GALL J, ZHENG H T, et al. SurfaceNet: an end-to-end 3D neural network for multiview stereopsis[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE Press, 2017: 2326-2334.
18	CAMPBELL N D F, VOGIATZIS G, HERNÁNDEZ C, et al. Using multiple hypotheses to improve depth-maps for multi-view stereo[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2008: 766-779.
19	TOLA E , STRECHA C , FUA P . Efficient large-scale multi-view stereo for ultra high-resolution image sets. Machine Vision and Applications, 2012, 23 (5): 903- 920. doi: 10.1007/s00138-011-0346-8
20	YAO Y, LUO Z X, LI S W, et al. MVSNet: depth inference for unstructured multi-view stereo[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 767-783.
21	WANG F, GALLIANI S, VOGEL C, et al. PatchmatchNet: learned multi-view patchmatch stereo[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE Press, 2021: 14189-14198.
22	XU Q G, XU Z X, PHILIP J, et al. Point-NeRF: point-based neural radiance fields[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 5428-5438.
23	王思成, 江浩, 陈晓. 具有跨尺度Transformer的高效多视图立体网络. 计算机工程, 2024, 50 (11): 266- 275.
	WANG S C , JIANG H , CHEN X . Efficient multi-view stereo network with cross-scale Transformer. Computer Engineering, 2024, 50 (11): 266- 275.
24	BIAN W J, WANG Z R, LI K J, et al. NoPe-NeRF: optimising neural radiance field with No pose prior[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE Press, 2023: 4160-4169.
25	YE Y F, YI R J, GAO Z R, et al. NEF: neural edge fields for 3D parametric curve reconstruction from multi-view images[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE Press, 2023: 8486-8495.
26	WANG X H , YIN Z Y , ZHANG F Q , et al. MP-NeRF: More refined deblurred neural radiance field for 3D reconstruction of blurred images. Knowledge-Based Systems, 2024, 290, 111571. doi: 10.1016/j.knosys.2024.111571
27	肖祎龙, 邓伊琴, 陈志刚. 面向动态三维人体重建的神经辐射场加速方法. 计算机工程, 2025, 51 (8): 95- 106. doi: 10.19678/j.issn.1000-3428.0069317
	XIAO Y L , DENG Y Q , CHEN Z G . Acceleration approach for neural radiance field in dynamic 3D human reconstruction. Computer Engineering, 2025, 51 (8): 95- 106. doi: 10.19678/j.issn.1000-3428.0069317
28	ZHOU J S, WEN X, MA B R, et al. 3D-OAE: occlusion auto-encoders for self-supervised learning on point clouds[EB/OL]. [2025-01-02]. https://arxiv.org/pdf/2203.14084.
29	PENG S, NIEMEYER M, MESCHEDER L, et al. Convolutional occupancy network[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 523-540.
30	费煜哲, 蔡欣, 赵鸣博, 等. 基于隐式表达的服装三维重建. 计算机工程, 2024, 50 (5): 220- 228. doi: 10.19678/j.issn.1000-3428.0067724
	FEI Y Z , CAI X , ZHAO M B , et al. 3D reconstruction of clothing based on implicit expression. Computer Engineering, 2024, 50 (5): 220- 228. doi: 10.19678/j.issn.1000-3428.0067724
31	景维鹏, 王源锋, 李超. 基于锥形追踪和网络分解的NeRF三维重建方法. 计算机工程, 2024, 50 (10): 334- 341. doi: 10.19678/j.issn.1000-3428.0068291
	JING W P , WANG Y F , LI C . NeRF 3D reconstruction method based on cone tracking and network decomposition. Computer Engineering, 2024, 50 (10): 334- 341. doi: 10.19678/j.issn.1000-3428.0068291
32	LORENSEN W E , CLINE H E . Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics, 1987, 21 (4): 163- 169. doi: 10.1145/37402.37422
33	OECHSLE M, PENG S Y, GEIGER A. UNISURF: unifying neural implicit surfaces and radiance fields for multi-view reconstruction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal, Canada: IEEE Press, 2021: 5569-5579.
34	YARIV L , KASTEN Y , MORAN D , et al. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 2020, 33, 2492- 2502.
35	YARIV L , GU J , KASTEN Y , et al. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 2021, 34, 4805- 4815.
36	DARMON F, BASCLE B, DEVAUX J C, et al. Improving neural implicit surfaces geometry with patch warping[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 6250-6259.
37	YU Z , PENG S , NIEMEYER M , et al. MonoSDF: exploring monocular geometric cues for neural implicit surface reconstruction. Advances in Neural Information Processing Systems, 2022, 35, 25018- 25032.
38	JENSEN R, DAHL A, VOGIATZIS G, et al. Large scale multi-view stereopsis evaluation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE Press, 2014: 406-413.
39	YAO Y, LUO Z X, LI S W, et al. BlendedMVS: a large-scale dataset for generalized multi-view stereo networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 1787-1796.

[1]	XIAO Yilong, DENG Yiqin, CHEN Zhigang. Acceleration Approach for Neural Radiance Field in Dynamic 3D Human Reconstruction [J]. Computer Engineering, 2025, 51(8): 95-106.
[2]	Huawei WANG, Ruoyan LIU, Zhiwei AI, Yi CAO. Performance Optimization Technique for Large-Scale Parallel Volume Rendering Based on Multiple Rendering Pipelines [J]. Computer Engineering, 2024, 50(8): 207-215.
[3]	ZHENG Li-Ping, LI Guang-Yao, JIANG Hua. Design and Implementation of Aided Diagnosis System for Oral and Maxillofacial Diseases [J]. Computer Engineering, 2011, 37(21): 279-281,284.
[4]	WEI Ji-zeng; SUN Ji-zhou. Parallel Memory System Based on Ray-casting Algorithm [J]. Computer Engineering, 2009, 35(3): 10-12,1.
[5]	ZHOU Qiao; CHEN Jing-wei; LI Jian-sheng; LAN Chao-zhen; XU Qing. 3D Visualization Technology of Electromagnetic Environment [J]. Computer Engineering, 2008, 34(9): 248-250.
[6]	LI Xin-wu. Frequency Domain Volume Rendering Algorithm Based on Data Difference [J]. Computer Engineering, 2008, 34(11): 41-42,4.
[7]	WEI Jizeng; SUN Jizhou; LI Xiaotu. Parallel Volume Rendering Algorithm Based on Sparse Matrix [J]. Computer Engineering, 2007, 33(10): 187-189,.
[8]	DONG Guiyun; HAN Peiyou. Automatic Segmentation and Volume Transparency Analysis of First Visual Human Body Data [J]. Computer Engineering, 2007, 33(10): 215-217.
[9]	WU Xiaofeng; SUN Jizhou ;WEI Jizeng. Memory Design of Dedicated Architecture for Volume Rendering [J]. Computer Engineering, 2007, 33(06): 275-277.
[10]	XU Jian , YANG Xin , GUO Qiang , SUN Kun. Texture-based 3D Ultrasound Real-time Volume Rendering [J]. Computer Engineering, 2006, 32(3): 231-232，256.

Please choose a citation manager

Content to export