基于细粒度深度质量感知的RGB-D显著性物体检测

doi:10.19678/j.issn.1000-3428.0070592

摘要/Abstract

摘要： 基于融合的RGB-D显著性物体检测(RGB-D SOD)方法通常采用双流结构在RGB和深度图之间进行权衡,目的是使得SOD性能优于单独使用RGB图时的性能。然而,深度图因场景、拍摄设备等因素的影响而存在质量差异,低质量的深度图无法有效辅助RGB图像进行显著性检测,甚至可能产生负面影响。现有方法很少考虑深度通道的质量差异,导致低质量深度区域的干扰信息被不合理地融合,从而影响结果的准确性。为解决以上问题,提出一种基于细粒度深度质量感知的RGB-D SOD方法,该方法主要分为2个阶段,即细粒度深度质量感知和双线性路由融合。在第一阶段,通过细粒度地比较RGB和D分支的显著性结果与伪标签(PGT)的相似性差异,对深度通道进行质量感知;在第二阶段,基于不同深度区域的质量感知结果,设计一种新的双线性路由融合策略,路由方向由深度通道质量感知结果驱动,以实现比现有融合方案更有效的多模态互补结果。在NJU2K、NLPR、SIP和STERE数据集上,将所提方法与9种SOTA方法进行比较,结果表明,所提方法在评价指标上都优于当前最先进的模型。相比于性能次优的模型,该方法的S-measure指标平均提升0.6%,F-measure指标平均提升0.7%,平均绝对误差(MAE)平均降低0.4%。所提方法显著减少了低质量、无贡献甚至负影响的深度区域对结果的影响,实现了更优的多模态融合,提升了显著性检测的效果。

关键词: 显著性物体检测, 深度图质量, 细粒度质量感知, 双线性路由融合, 多模态互补

Abstract: Fusion-based RGB-D Salient Object Detection (RGB-D SOD) methods employ a dual-stream structure to balance RGB and depth maps, aiming to achieve superior performance in SOD compared with using RGB maps alone. However, owing to factors such as the scene and shooting equipment, depth maps vary in quality. Low-quality depth maps cannot effectively assist RGB images in salient detection and may even have a negative impact. Existing methods rarely consider the quality differences in depth channels, leading to the unreasonable fusion of interference information from low-quality depth regions, which affects the accuracy of the results. To address these issues, this paper proposes a fine-grained depth-quality-aware RGB-D SOD method, which is divided into two stages: fine-grained depth quality awareness and bilinear routing fusion. In the first stage, the depth channel quality is perceived by comparing the similarity differences between the salient results of the RGB and D branches and Pseudo Ground Truth (PGT) in a fine-grained manner. In the second stage, a new bilinear routing fusion strategy is designed based on the quality perception results for different depth regions. The routing direction is driven by the results of depth channel quality perception, achieving more effective multimodal complementary results than existing fusion schemes. The proposed method is compared to nine State-Of-The-Art (SOTA) methods on the NJU2K, NLPR, SIP, and STERE datasets. The results show that the proposed method outperforms current leading models in all evaluation metrics. Compared with the suboptimal models, the S-measure and F-measure metrics of the proposed method are improved by an average of 0.6% and 0.7%, respectively, and the Mean Absolute Error (MAE) is reduced by an average of 0.4%. The proposed method significantly reduces the impact of low-quality, non-contributing, or negatively impacting depth regions on the results, achieving better multimodal fusion and enhancing the effectiveness of salient detection.

Key words: Salient Object Detection (SOD), depth map quality, fine-grained quality perception, bilinear routing fusion, multi-modal complementarity

中图分类号:

TP391

张明月, 崔爽锌, 赵一汎, 卢搏, 王子铭, 陈程立诏. 基于细粒度深度质量感知的RGB-D显著性物体检测[J]. 计算机工程, 2026, 52(7): 233-242.

ZHANG Mingyue, CUI Shuangxin, ZHAO Yifan, LU Bo, WANG Ziming, CHEN Chenglizhao. RGB-D Salient Object Detection Based on Fine-Grained Depth Quality Perception[J]. Computer Engineering, 2026, 52(7): 233-242.

https://www.ecice06.com/CN/Y2026/V52/I7/233

参考文献

[1] SONG M K, SONG W F, YANG G W, et al. Improving RGB-D salient object detection via modality-aware decoder[J]. IEEE Transactions on Image Processing, 2022, 31: 6124-6138.
[2] 李振鲁, 黄威, 孙锴. 复杂环境下的轻量化道路目标识别算法研究[J]. 计算机工程, 2024, 50(4): 219-227. LI Z L, HUANG W, SUN K. Research on lightweight road-target-recognition algorithm in complex environment[J]. Computer Engineering, 2024, 50(4): 219-227. (in Chinese)
[3] CHENG M M, HOU Q B, ZHANG S H, et al. Intelligent visual media processing: when graphics meets vision[J]. Journal of Computer Science and Technology, 2017, 32(1): 110-121.
[4] SONG M K, LI L F, WU D Q, et al. Rethinking object saliency ranking: a novel whole-flow processing paradigm[J]. IEEE Transactions on Image Processing, 2024, 33: 338-353.
[5] LI L M, CHEN C, ZHANG X W. Mask-guided self-distillation for visual tracking[C]//Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). Washington D.C.,USA:IEEE Press,2022: 1-6.
[6] CHEN C, ZHAO H M, YANG H, et al. Full-reference screen content image quality assessment by fusing multilevel structure similarity[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2021, 17(3): 1-21.
[7] 孙福明, 胡锡航, 武景宇, 等. 跨模态交互融合与全局感知的RGB-D显著性目标检测[J]. 软件学报, 2024, 35(4): 1899-1913. SUN F M, HU X H, WU J Y, et al. RGB-D salient object detection based on cross-modal interactive fusion and global awareness[J]. Journal of Software, 2024, 35(4): 1899-1913. (in Chinese)
[8] 于洋洋, 吴敦全, 陈程立诏. 基于稀疏对比自蒸馏的RGB-D显著性物体检测[J]. 计算机工程, 2025, 51(10): 327-335. YU Y Y, WU D Q, CHEN C. RGB-D saliency object detection based on sparse contrastive self-distillation[J]. Computer Engineering, 2025, 51(10): 327-335. (in Chinese)
[9] 吴映霓, 闫河, 姜彬, 等. HDF+: 一种新的RGB-D显著性目标检测网络模型[J]. 小型微型计算机系统, 2025, 46(7): 1645-1651. WU Y N, YAN H, JIANG B, et al. HDF+: a new RGB-D salient object detection network[J]. Journal of Chinese Computer Systems, 2025, 46(7): 1645-1651. (in Chinese)
[10] 宋梦柯, 郑元超, 陈程立诏. 基于深度质量感知和分层特征引导的RGB-D显著性检测[J]. 计算机工程, 2023, 49(5): 255-261, 268. SONG M K, ZHENG Y C, CHEN C. RGB-D saliency detection via depth quality perception and hierarchical feature guidance[J]. Computer Engineering, 2023, 49(5): 255-261, 268. (in Chinese)
[11] LIU Z Y, SHI S, DUAN Q T, et al. Salient object detection for RGB-D image by single stream recurrent convolution neural network[J]. Neurocomputing, 2019, 363: 46-57.
[12] REN J Q, GONG X J, LU Y, et al. Exploiting global priors for RGB-D saliency detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Washington D.C.,USA:IEEE Press,2015: 25-32.
[13] CHEN H, LI Y F. Progressively complementarity-aware fusion network for RGB-D salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press, 2018: 3051-3060.
[14] ZHAO J X, CAO Y, FAN D P, et al. Contrast prior and fluid pyramid integration for RGBD salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C.,USA:IEEE Press,2020: 3922-3931.
[15] DING Y, LIU Z, HUANG M K, et al. Depth-aware saliency detection using convolutional neural networks[J]. Journal of Visual Communication and Image Representation, 2019, 61: 1-9.
[16] SHIGEMATSU R, FENG D, YOU S D, et al. Learning RGB-D salient object detection using background enclosure, depth contrast, and top-down features[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW). Washington D.C.,USA:IEEE Press,2018: 2749-2757.
[17] QU L Q, HE S F, ZHANG J W, et al. RGBD salient object detection via deep fusion[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2274-2285.
[18] CHEN Q, FU K R, LIU Z, et al. EF-Net: a novel enhancement and fusion network for RGB-D saliency detection[J]. Pattern Recognition, 2021, 112: 107740.
[19] JI W, LI J J, YU S, et al. Calibrated RGB-D salient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C.,USA:IEEE Press,2021: 9466-9476.
[20] ZHOU W J, ZHU Y, LEI J S, et al. LSNet: lightweight spatial boosting network for detecting salient objects in RGB-thermal images[J]. IEEE Transactions on Image Processing, 2023, 32: 1329-1340.
[21] CHEN Q, LIU Z, ZHANG Y, et al. RGB-D salient object detection via 3D convolutional neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(2): 1063-1071.
[22] FAN D P, ZHAI Y J, BORJI A, et al. BBS-Net: RGB-D salient object detection with a bifurcated backbone strategy network[EB/OL].[2024-07-05]. https://arxiv.org/abs/2007.02713v1.
[23] CHENG X L, ZHENG X, PEI J L, et al. Depth-induced gap-reducing network for RGB-D salient object detection: an interaction, guidance and refinement approach[J]. IEEE Transactions on Multimedia, 2023, 25: 4253-4266.
[24] KE L, YE M Q, MARTIN D, et al. Segment anything in high quality[EB/OL].[2024-07-05]. https://arxiv.org/abs/2306.01567.
[25] ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282.
[26] WU Y H, LIU Y, ZHANG L, et al. EDN: salient object detection via extremely-downsampled network[J]. IEEE Transactions on Image Processing, 2022, 31: 3125-3136.
[27] MA M C, XIA C Q, XIE C X, et al. Boosting broader receptive fields for salient object detection[J]. IEEE Transactions on Image Processing, 2023, 32: 1026-1038.
[28] CHENG M M, FAN D P. Structure-measure: a new way to evaluate foreground maps[J]. International Journal of Computer Vision, 2021, 129(9): 2622-2638.
[29] PANG Y W, ZHAO X Q, ZHANG L H, et al. CAVER: cross-modal view-mixed transformer for bi-modal salient object detection[J]. IEEE Transactions on Image Processing, 2023, 32: 892-904.
[30] CONG R M, LIU H Y, ZHANG C, et al. Point-aware interaction and CNN-induced refinement network for RGB-D salient object detection[EB/OL].[2024-07-05].https://arxiv.org/abs/2308.08930.
[31] JU R, LIU Y, REN T W, et al. Depth-aware salient object detection using anisotropic center-surround difference[J]. Signal Processing: Image Communication, 2015, 38: 115-126.
[32] PENG H W, LI B, XIONG W H, et al. RGBD salient object detection: a benchmark and algorithms[EB/OL].[2024-07-05]. https://link.springer.com/chapter/10.1007/978-3-319-10578-9_7.
[33] NIU Y Z, GENG Y J, LI X Q, et al. Leveraging stereopsis for saliency analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2012: 454-461.
[34] ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2009: 1597-1604.
[35] PERAZZI F, KRÄHENBVHL P, PRITCH Y, et al. Saliency filters: contrast based filtering for salient region detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2012: 733-740.
[36] LIAO G B, GAO W, LI G, et al. Cross-collaborative fusion-encoder network for robust RGB-thermal salient object detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(11): 7646-7661.
[37] WU Y H, LIU Y, XU J, et al. MobileSal: extremely efficient RGB-D salient object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 10261-10269.
[38] CONG R M, LIN Q W, ZHANG C, et al. CIR-Net: cross-modality interaction and refinement for RGB-D salient object detection[J]. IEEE Transactions on Image Processing, 2022, 31: 6800-6815.

选择文件类型/文献管理软件名称

选择包含的内容