Abstract: Recent breakthroughs in 3D Gaussian Splatting (3DGS) revolutionized novel view synthesis, accelerating its adoption in medical services. However, under sparse-view conditions, 3DGS tends to overfit training views and learn incorrect scene geometry owing to insufficient constraints. To address the above limitation, this paper proposes a novel view synthesis method under sparse inputs (GMMSplat). This method corrects scene representation through prior-guided depth regularization and photometric constraint based on fine-grained local cropping. Specifically, for training views, GMMSplat employs a Gaussian Mixture Model to dynamically adjust confidence thresholds based on monocular depth confidence maps. Depths with confidence below the threshold are discarded, ensuring only high-confidence depth data constrains rendered depth. This effectively mitigates geometric collapse caused by depth errors. Furthermore, to alleviate overfitting, GMMSplat generates warped images from virtual viewpoints derived from training views. A local-crop strategy is applied to these warped images, with higher weights assigned to center-cropped regions based on image quality. This strategically guides scene appearance reconstruction. Evaluations on LLFF、Mip-NeRF360 and ZED2 datasets show GMMSplat surpasses state-of-the-art (SOTA) performance on key metrics, significantly enhancing few-shot novel view synthesis quality. Specifically, on the LLFF dataset (at 1/8 resolution), the PSNR increases by 3.75%, the inference speed improves by 14.52%, and the storage size decreases by 49%.

摘要： 最近，3D Gaussian Splatting(3DGS)技术在新视角合成领域取得了突破性的进展，并广泛应用于医疗等领域。然而，当只有少量视图输入时，由于缺乏有效约束，3DGS易对训练视角过拟合，从而学习到错误的场景几何结构。针对这一挑战，本文提出了一种稀疏条件下基于混合高斯滤波的新视角合成方法(GMMSplat)，该方法通过构建基于先验引导的深度正则化与基于细粒度局部裁剪的光度约束，有效校正了场景表示。首先，在训练视角上，根据单目深度的置信度图，利用混合高斯模型(Gaussian Mixture Model, GMM)动态选择阈值，丢弃置信度低于阈值的深度，确保置信度高的深度后续对渲染深度进行约束，从而减少深度误差导致的场景表示的几何坍塌。此外，为了进一步缓解过拟合问题，由训练视角插值得到虚拟视角下的扭曲图像，对扭曲图像实施局部裁剪策略，并根据扭曲图像的质量对中心裁剪区域分配更高的权重，从而有效引导场景外观重建。本方法在LLFF、Mip-NeRF360、ZED2数据集上的测试结果表明，其在关键评价指标上，超越了现有方法的性能水平，能够提升新视角合成图像的质量。其中，在LLFF(1/8分辨率)数据集上PSNR提升3.75%、推理速度提升14.52%、存储体积减小49%。