Abstract:

Fusion-based RGB-D Salient Object Detection (RGB-D SOD) methods employ a dual-stream structure to balance RGB and depth maps, aiming to achieve superior performance in SOD compared with using RGB maps alone. However, owing to factors such as the scene and shooting equipment, depth maps vary in quality. Low-quality depth maps cannot effectively assist RGB images in salient detection and may even have a negative impact. Existing methods rarely consider the quality differences in depth channels, leading to the unreasonable fusion of interference information from low-quality depth regions, which affects the accuracy of the results. To address these issues, this paper proposes a fine-grained depth-quality-aware RGB-D SOD method, which is divided into two stages: fine-grained depth quality awareness and bilinear routing fusion. In the first stage, the depth channel quality is perceived by comparing the similarity differences between the salient results of the RGB and D branches and Pseudo Ground Truth (PGT) in a fine-grained manner. In the second stage, a new bilinear routing fusion strategy is designed based on the quality perception results for different depth regions. The routing direction is driven by the results of depth channel quality perception, achieving more effective multimodal complementary results than existing fusion schemes. The proposed method is compared to nine State-Of-The-Art (SOTA) methods on the NJU2K, NLPR, SIP, and STERE datasets. The results show that the proposed method outperforms current leading models in all evaluation metrics. Compared with the suboptimal models, the S-measure and F-measure metrics of the proposed method are improved by an average of 0.6% and 0.7%, respectively, and the Mean Absolute Error (MAE) is reduced by an average of 0.4%. The proposed method significantly reduces the impact of low-quality, non-contributing, or negatively impacting depth regions on the results, achieving better multimodal fusion and enhancing the effectiveness of salient detection.

Key words: Salient Object Detection (SOD), depth map quality, fine-grained quality perception, bilinear routing fusion, multi-modal complementarity

摘要：

基于融合的RGB-D显著性物体检测(RGB-D SOD)方法通常采用双流结构在RGB和深度图之间进行权衡, 目的是使得SOD性能优于单独使用RGB图时的性能。然而, 深度图因场景、拍摄设备等因素的影响而存在质量差异, 低质量的深度图无法有效辅助RGB图像进行显著性检测, 甚至可能产生负面影响。现有方法很少考虑深度通道的质量差异, 导致低质量深度区域的干扰信息被不合理地融合, 从而影响结果的准确性。为解决以上问题, 提出一种基于细粒度深度质量感知的RGB-D SOD方法, 该方法主要分为2个阶段, 即细粒度深度质量感知和双线性路由融合。在第一阶段, 通过细粒度地比较RGB和D分支的显著性结果与伪标签(PGT)的相似性差异, 对深度通道进行质量感知; 在第二阶段, 基于不同深度区域的质量感知结果, 设计一种新的双线性路由融合策略, 路由方向由深度通道质量感知结果驱动, 以实现比现有融合方案更有效的多模态互补结果。在NJU2K、NLPR、SIP和STERE数据集上, 将所提方法与9种SOTA方法进行比较, 结果表明, 所提方法在评价指标上都优于当前最先进的模型。相比于性能次优的模型, 该方法的S-measure指标平均提升0.6%, F-measure指标平均提升0.7%, 平均绝对误差(MAE)平均降低0.4%。所提方法显著减少了低质量、无贡献甚至负影响的深度区域对结果的影响, 实现了更优的多模态融合, 提升了显著性检测的效果。

关键词: 显著性物体检测, 深度图质量, 细粒度质量感知, 双线性路由融合, 多模态互补