边缘深度挖掘的弱监督显著性目标检测

doi:10.19678/j.issn.1000-3428.0065413

摘要/Abstract

摘要：

基于深度学习的显著性目标检测算法大多依赖于大规模标注数据下的监督学习模式，但是，样本的像素级标签存在获取困难、标注成本高的问题。为此，设计一种边缘深度挖掘的弱监督显著性目标检测算法，仅使用图像级类别标签，从输入图像的显著性目标边缘角度得到能够较准确描述目标轮廓信息的边缘特征图，以生成伪标签对显著性模型进行监督训练。通过粗糙边缘生成模块对显著性目标轮廓特征进行简单标定，用于获取粗糙边缘特征图。在此基础上，利用精细边缘生成模块优化特征以得到精细边缘特征图，能够更准确地反映目标的边界信息，在完整刻画轮廓信息的同时可以更好地抑制背景噪声。伪标签生成模块基于精细边缘特征图生成像素级伪标签。实验结果表明，相比MSW、MFNet、NSAL等算法，该算法能够准确识别显著性区域，获得的预测图具有较完整的细节信息，其中，在ECSSD数据集上S-measure值和E-measure值相较于第2名NSAL算法分别提高1.1和0.6个百分点。

关键词: 弱监督, 显著性目标检测, 深度学习, 图像级类别标签, 伪标签

Abstract:

Salient Object Detection(SOD) algorithms based on deep learning primarily use supervised learning for large-scale labeled data.However, obtaining pixel-level labels for samples is difficult and labeling costs are high.To this end, a weakly supervised SOD algorithm for edge depth mining is designed.It uses image-level category labels to obtain edge feature maps that accurately describe the target's contour information from the perspective of the salient object edge of the input image, producing pseudo labels for supervised training of the saliency model.Through the Coarse Edge Generation(CEG) module, simple calibration of salient target contour features is performed to obtain rough edge feature maps. The Fine Edge Generation(FEG) module is subsequently used to optimize features to obtain a fine edge feature map, which accurately reflects the boundary information of the target and suppresses background noise for complete contour detection. The Pseudo Label Generation(PLG) module generates pixel-level pseudo labels based on fine edge feature maps. Experimental results show that in contrast to algorithms such as MSW, MFNet, and NSAL, the proposed algorithm accurately identifies significant regions and obtains prediction maps with complete detail information.Compared to the second place NSAL algorithm, the S-measure and E-measure values of the proposed algorithm increase by 1.1 and 0.6 percentage points on ECSSD dataset, respectively.

Key words: weakly supervised, Salient Object Detection(SOD), deep learning, image-level category label, pseudo label

李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.

Junxia LI, Xingchi WANG, Zi YIN, Deshuo SHI. Weakly Supervised Salient Object Detection via Edge Depth Mining[J]. Computer Engineering, 2023, 49(7): 169-178.

http://www.ecice06.com/CN/Y2023/V49/I7/169

图/表 12

参考文献 34

1	LI M H, LI S, LI L D, et, al. Spatial feature calibration and temporal fusion for effective one-stage video instance segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 11210-11219.
2	YIN J B, WANG W G, MENG Q H, et al. A unified object motion and affinity model for online multi-object tracking[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 6767-6776.
3	CHEN Y H, CAO Y, HU H, et al. Memory enhanced global-local aggregation for video object detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 10334-10343.
4	WANG L J, LU H C, WANG Y F, et al. Learning to detect salient objects with image-level supervision[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3796-3805.
5	ZENG Y, ZHUGE Y Z, LU H C, et al. Multi-source weak supervision for saliency detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 6067-6076.
6	ZHANG J, YU X, LI A X, et al. Weakly-supervised salient object detection via scribble annotations[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 12543-12552.
7	GAO S Y, ZHANG W, WANG Y, et al. Weakly-supervised salient object detection using point supervision[C]//Proceedings of the AAAI Conference on Artificial Intelligence. [S. l.]: AAAI Press, 2022: 670-678.
8	LI G B, XIE Y A, LIN L A. Weakly supervised salient object detection using image labels[EB/OL]. [2022-06-29]. https://arxiv.org/pdf/1803.06503.pdf.
9	PIAO Y R, WANG J, ZHANG M, et al. MFNet: multi-filter directive network for weakly supervised salient object detection[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2022: 4116-4125.
10	PIAO Y R, WU W, ZHANG M, et al. Noise-sensitive adversarial learning for weakly supervised salient object detection[J/OL]. IEEE Transactions on Multimedia, 2022: 1-10[2022-06-29]. https://ieeexplore.ieee.org/document/9716868.
11	陈威, 李决龙, 邢建春, 等. 融合前景信息的背景模板优化显著性检测算法. 计算机工程, 2019, 45(1): 221-225, 232 URL
	CHEN W, LI J L, XING J C, et al. Background template optimization saliency detection algorithm fusing with foreground information. Computer Engineering, 2019, 45(1): 221-225, 232 URL
12	肖锋, 李茹娜. 语义信息引导下的显著目标检测算法. 计算机工程, 2019, 45(4): 248- 253. URL
	XIAO F, LI R N. Salient object detection algorithm under guidance of semantic information. Computer Engineering, 2019, 45(4): 248- 253. URL
13	PANG Y W, ZHAO X Q, ZHANG L H, et al. Multi-scale interactive network for salient object detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 9410-9419.
14	YANG S, LIN W S, LIN G S, et al. Progressive self-guided loss for salient object detection. IEEE Transactions on Image Processing, 2021, 30, 8426- 8438. doi: 10.1109/TIP.2021.3113794
15	LI J, SU J M, XIA C Q, et al. Salient object detection with purificatory mechanism and structural similarity loss. IEEE Transactions on Image Processing, 2021, 30, 6855- 6868. doi: 10.1109/TIP.2021.3099405
16	KE Y Y, TSUBONO T. Recursive contour-saliency blending network for accurate salient object detection[C]//Proceedings of Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2022: 1360-1370.
17	YU S Y, ZHANG B F, XIAO J M, et al. Structure-consistent weakly supervised salient object detection with local saliency coherence[C]//Proceedings of AAAI Conference on Artificial Intelligence. [S. l.]: AAAI Press, 2021: 3234-3242.
18	ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2921-2929.
19	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[EB/OL]. [2022-06-29]. https://arxiv.org/abs/1512.03385.
20	AHN J, CHO S, KWAK S. Weakly supervised learning of instance segmentation with inter-pixel relations[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 2204-2213.
21	KRAHENBUHL P, KOLTUN V. Efficient inference in fully connected CRFs with Gaussian edge potentials[EB/OL]. [2022-06-29]. https://arxiv.org/pdf/1210.5644.pdf.
22	GAO S H, CHENG M M, ZHAO K, et al. Res2Net: a new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652- 662. doi: 10.1109/TPAMI.2019.2938758
23	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 1-9.
24	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2022-06-29]. https://arxiv.org/pdf/1706.05587.pdf.
25	YAN Q, XU L, SHI J P, et al. Hierarchical saliency detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2013: 1155-1162.
26	LI G B, YU Y Z. Visual saliency based on multiscale deep features[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 5455-5463.
27	YANG C, ZHANG L H, LU H C, et al. Saliency detection via graph-based manifold ranking[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2013: 3166-3173.
28	EVERINGHAM M, GOOL L, WILLIAMS C K I, et al. The PASCAL Visual Object Classes (VOC) challenge. International Journal of Computer Vision, 2010, 88(2): 303- 338. doi: 10.1007/s11263-009-0275-4
29	DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 248-255.
30	FAN D P, CHENG M M, LIU Y, et al. Structure-measure: a new way to evaluate foreground maps[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 4558-4567.
31	FAN D P, GONG C, CAO Y, et al. Enhanced-alignment measure for binary foreground map evaluation[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. New York, USA: ACM Press, 2018: 698-704.
32	ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 1597-1604.
33	PERAZZI F, KRÄHENBÜHL P, PRITCH Y, et al. Saliency filters: contrast based filtering for salient region detection[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2012: 733-740.
34	ZHANG L Q, ZHANG Q, ZHAO R. Progressive dual-attention residual network for salient object detection. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(9): 5902- 5915. doi: 10.1109/TCSVT.2022.3164093

预训练	微调训练	ECSSD		DUTS-TE
预训练	微调训练	F-measure	MAE	F-measure	MAE
√	—	0.824	0.083	0.719	0.077
√	√	0.858	0.078	0.737	0.070

预训练	微调训练	ECSSD		DUTS-TE
预训练	微调训练	F-measure	MAE	F-measure	MAE
√	—	0.824	0.083	0.719	0.077
√	√	0.858	0.078	0.737	0.070

损失函数	ECSSD		DUTS-TE
损失函数	F-measure	MAE	F-measure	MAE
单一损失函数	0.847	0.080	0.711	0.076
多重损失函数	0.858	0.078	0.737	0.070

损失函数	ECSSD		DUTS-TE
损失函数	F-measure	MAE	F-measure	MAE
单一损失函数	0.847	0.080	0.711	0.076
多重损失函数	0.858	0.078	0.737	0.070

CEG	FEG	FD	ECSSD		DUTS-TE
CEG	FEG	FD	F-measure	MAE	F-measure	MAE
√	—	—	0.808	0.087	0.663	0.081
√	—	√	0.822	0.081	0.691	0.074
√	√	—	0.847	0.079	0.714	0.073
√	√	√	0.858	0.078	0.737	0.070

选择文件类型/文献管理软件名称

选择包含的内容