渐进式特征增强的弱监督显著性目标检测

doi:10.19678/j.issn.1000-3428.0068406

摘要/Abstract

摘要：

针对多数弱监督显著性检测方法在复杂场景下容易出现目标结构缺损、边界粗糙等问题, 提出一种渐进式特征增强的弱监督显著性检测算法。首先针对显著目标结构不完整问题, 设计一种渐进式特征增强机制, 主要包括双流语义增强模块和层次化自适应特征聚合模块, 通过复用这种机制可以捕获更丰富的图像特征; 其次为获取清晰完整的目标边缘, 提出边缘引导模块, 可以生成高质量的显著目标边缘图; 最后将得到的边缘对显著区域预测网络进行指导, 以生成结构完整且边界平滑的检测结果。在5个公开数据集上的实验结果表明, 相比经典的WSSA算法, 该算法在PASCAL-S数据集上平均绝对误差(MAE)降低了21.32%, F-measure值提高了6.27%, 优于大多数先进的弱监督显著性目标检测算法。

关键词: 弱监督, 显著性目标检测, 渐进式, 特征聚合, 边缘引导

Abstract:

For the weakly supervised object detection methods, it is easy to have problems such as object structure defect and rough boundary in complex scenarios. To overcome this problem, a progressively feature-enhanced, weakly supervised network for salient object detection has been proposed. First, this model uses a full convolutional neural network (ResNet-50) as the main stem for feature extraction, which helps the model learn the intrinsic features of the salient regions. To address the problem of the incomplete structure of salient objects, a Progressive Feature-Enhanced Mechanism (PFEM) is designed, which mainly includes a dual-stream semantic enhancement module and a hierarchical adaptive feature aggregation module. By reusing the progressive feature-enhancement mechanism, richer image features can be captured. Moreover, an Edge-Guided Module (EGM) is proposed to exploit the salient object edge structure. EGM can effectively aggregate the boundary formation of features and capture high-quality salient edge maps. Finally, salient edge maps are used to generate detection results with a complete structure and clear boundaries. Experimental results on five public datasets show that, compared with the classical WSSA algorithm, the Mean Absolute Error (MAE) on the PASCAL-S dataset is reduced by 21.32%, and the F-measure is increased by 6.27%, which is better than most advanced weakly supervised significance target detection algorithms.

Key words: weakly supervised, Salient Object Detection(SOD), progressive, feature fusion, edge guided

李沼洁, 朱恒亮, 毛国君, 杨鑫. 渐进式特征增强的弱监督显著性目标检测[J]. 计算机工程, 2024, 50(12): 233-244.

LI Zhaojie, ZHU Hengliang, MAO Guojun, YANG Xin. Progressively Feature-Enhanced Weakly Supervised for Salient Object Detection[J]. Computer Engineering, 2024, 50(12): 233-244.

https://www.ecice06.com/CN/Y2024/V50/I12/233

图/表 14

图1 结构不完整和边界粗糙图像

Fig.1 Images of incomplete structure and rough borders

图2 PFENet网络的整体架构

Fig.2 The overall architecture of PFENet network

图3 渐进式特征融合增强效果

Fig.3 The effect of progressive feature fusion enhancement

图4 双流语义增强模块

Fig.4 Dual-stream semantic enhancement module

图5 层次化自适应特征聚合模块

Fig.5 Hierarchical adaptive feature aggregation module

图6 边缘引导模块

Fig.6 Edge-guided module

图7 不同算法在4个SOD数据集上的F-measure曲线

Fig.7 F-measure curves of different algorithms on 4 SOD datasets

图8 本文算法与其他算法的可视化对比

Fig.8 Visual comparison between the proposed algorithm and other algorithm

图9 失败案例的图示

Fig.9 Illustration of the failed examples

参考文献 42

1	SUN F D , LI W H . Saliency guided deep network for weakly-supervised image segmentation. Pattern Recognition Letters, 2019, 120, 62- 68. doi: 10.1016/j.patrec.2019.01.009
2	HE J F, FENG J Y, LIU X L, et al. Mobile product search with bag of Hash bits and boundary reranking[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2012: 3005-3012.
3	HONG S, YOU T, KWAK S, et al. Online tracking by learning discriminative saliency map with convolutional neural network[EB/OL]. [2023-08-10]. https://arxiv.org/pdf/1502.06796.
4	RUTISHAUSER U, WALTHER D, KOCH C, et al. Is bottom-up attention useful for object recognition? [C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2015: 458-467.
5	XIE C X, XIA C Q, MA M C, et al. Pyramid grafting network for one-stage high resolution saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 11707-11716. URL
6	TIAN X, XU K, YANG X, et al. Bi-directional object-context prioritization learning for saliency ranking[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C, USA: IEEE Press, 2022: 5872-5881. URL
7	YU S Y, XIAO J M, ZHANG B F, et al. Democracy does matter: comprehensive feature mining for co-salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 979-988. URL
8	YU S Y , ZHANG B F , XIAO J M , et al. Structure-consistent weakly supervised salient object detection with local saliency coherence. Artificial Intelligence, 2021, 35 (4): 3234- 3242.
9	ZHANG J, YU X, LI A X, et al. Weakly-supervised salient object detection via scribble annotations[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 12543-12552. URL
10	LAI B S, GONG X J. Saliency guided end-to-end learning for weakly supervised object detection[EB/OL]. [2023-08-10]. https://arxiv.org/pdf/1412.26980. URL
11	ZHAO T, WU X Q. Image saliency detection with low-level features enhancement[C]//Proceedings of Chinese Conference on Pattern Recognition and Computer Vision. Berlin, Germany: Springer, 2018: 408-419. URL
12	LI H Y , CHEN J , LU H C , et al. CNN for saliency detection with low-level feature integration. Neurocomputing, 2017, 226, 212- 220. doi: 10.1016/j.neucom.2016.11.056
13	TIAN X. Weakly-supervised salient instance detection (supplementary materials)[EB/OL]. [2023-08-10]. https://arxiv.org/abs/2009.13898.
14	何亚茹, 葛洪伟. 视觉显著区域和主动轮廓结合的图像分割算法. 计算机科学与探索, 2022, 16 (5): 1155- 1168. doi: 10.3778/j.issn.1673-9418.2011043
	HE Y R , GE H W . Image segmentation algorithm combining visual salient regions and active contour. Journal of Frontiers of Computer Science and Technology, 2022, 16 (5): 1155- 1168. doi: 10.3778/j.issn.1673-9418.2011043
15	李军侠, 王星驰, 殷梓, 等. 边缘深度挖掘的弱监督显著性目标检测. 计算机工程, 2022, 48 (3): 1- 12.
	LI J X , WANG X C , YIN Z , et al. Weak supervised salience object detection in deep edge mining. Computer Engineering, 2022, 48 (3): 1- 12.
16	WANG L J, LU H C, WANG Y F, et al. Learning to detect salient objects with image-level supervision[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3796-3805.
17	LI G B , XIE Y , LIN L . Weakly supervised salient object detection using image labels. Artificial Intelligence, 2018, 32 (1): 316- 325.
18	赵世敏, 王鹏杰, 曹乾, 等. 基于图像语义的弱监督显著性物体检测. 计算机辅助设计与图形学学报, 2021, 33 (2): 270- 277. doi: 10.3724/SP.J.1089.2021.18318
	ZHAO S M , WANG P J , CAO Q , et al. Weak supervised salience object detection based on image semantics. Journal of Computer Aided Design and Graphics, 2021, 33 (2): 270- 277. doi: 10.3724/SP.J.1089.2021.18318
19	沈启金, 龙观潮, 陈羽中. 基于图像分类的弱监督RGBD图像显著性检测方法. 小型微型计算机系统, 2022, 43 (1): 61- 68. doi: 10.3969/j.issn.1000-1220.2022.01.011
	SHEN Q J , LONG G C , CHEN Y Z . Weakly-supervised RGBD image saliency detection based on image classification. Journal of Chinese Computer Systems, 2022, 43 (1): 61- 68. doi: 10.3969/j.issn.1000-1220.2022.01.011
20	GONTHIER N, GOUSSEEAU Y, LADJAL S, et al. Weakly super-vised object detection in artworks[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 692-709.
21	AHN J, KWAK S. Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 4981-4990. URL
22	ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 2921-2929.
23	ZHANG J, ZHANG T, DAF Y, et al. Deep unsupervised saliency detection: a multiple noisy labeling perspective[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 9029-9038. URL
24	ZENG Y, ZHUGE Y Z, LU H C, et al. Multi-source weak supervision for saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 6067-6076. URL
25	SIVA P, RUSSELL C, XIANG T, et al. Looking beyond the image: unsupervised learning for object saliency and detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE Press, 2013: 3238-3245. URL
26	PIAO Y R, WANG J, ZHANG M, et al. MFNet: multi-filter directive network for weakly supervised salient object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 4116-4125.
27	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 2011-2023. URL
28	HOU Q B, CHENG M M, HU X W, et al. Deeply supervised salient object detection with short connections[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE Press, 2017: 815-828. URL
29	YAN Q, XU L, SHI J P, et al. Hierarchical saliency detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2013: 1155-1162. URL
30	YANG C, ZHANG L, LU H, et al. Saliency detection via graph-based manifold ranking[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2013: 3166-3173. URL
31	LI G B, YU Y Z. Visual saliency based on multiscale deep features[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE Press, 2015: 5455-5463.
32	LI Y, HOU X D, KOCH C, et al. The secrets of salient object segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 280-287.
33	ZHANG X N, WANG T T, QI J Q, et al. Progressive attention guided recurrent network for salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 714-722.
34	ZHANG L, DAI J, LU H C, et al. A bi-directional message passing model for salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1741-1750.
35	LI X , YANG F , CHENG H , et al. Contour knowledge transfer for salient object detection. Berlin, Germany: Springer, 2018.
36	QIN X B, ZHANG Z C, HUANG C Y, et al. BASNet: boundary-aware salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 7479-7489.
37	PERAZZI F, KRAHENBUHL P, PRITCH Y, et al. Saliency filters: contrast based filtering for salient region detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2012: 733-740.
38	ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 1597-1604.
39	FAN D P, GONG C, CAO Y, et al. Enhanced-alignment measure for binary foreground map evaluation[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2018: 698-704.
40	LIU J J, HOU Q B, CHENG M M, et al. A simple pooling-based design for real-time salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 3917-3926.
41	ZHAO J X, LIU J J, FAN D P, et al. EGNet: edge guidance network for salient object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 8779-8788.
42	LIU N, HAN J W, YANG M H. PiCANet: learning pixel-wise contextual attention for saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 3089-3098.

[1]	周炫余, 吴莲华, 郑勤华, 肖天星, 王紫璇, 张思敏. 联合语义提示和记忆增强的弱监督跳绳视频异常检测方法[J]. 计算机工程, 2024, 50(7): 87-95.
[2]	张慧妍, 梁勇, 兰景宏, 赵强. 基于记忆模块与过滤式生成对抗网络的入侵检测方法[J]. 计算机工程, 2024, 50(6): 197-207.
[3]	崔锦莹, 梁立河, 任雪婷, 强彦, 赵涓涓, 孔晓梅, 尉骁, 张华. 基于噪声鲁棒学习的胸部X射线尘肺鉴别方法[J]. 计算机工程, 2024, 50(11): 350-359.
[4]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[5]	宋鹏鹏, 龚声蓉, 钟珊, 周立凡, 凤黄浩. 基于双注意力擦除和注意力信息聚合的弱监督目标检测[J]. 计算机工程, 2023, 49(3): 113-120,127.
[6]	刘仲任, 彭力. 多尺度视觉感知融合的显著性目标检测[J]. 计算机工程, 2023, 49(12): 186-193.
[7]	李刚, 邵瑞, 周鸣乐, 李敏, 万洪林. 基于注意力的轻量级工业产品缺陷检测网络[J]. 计算机工程, 2023, 49(11): 275-283.
[8]	李惠森, 侯进, 党辉, 周宇航. 基于渐进式训练的多判别器域适应目标检测[J]. 计算机工程, 2023, 49(10): 202-211, 221.
[9]	张欢, 黄涛, 许俊杰, 徐川, 杨威. 多层渐进式特征对齐网络优化的空地影像稳健匹配[J]. 计算机工程, 2023, 49(10): 178-185.
[10]	王安志, 任春洪, 何淋艳, 杨元英, 欧卫华. 基于多模态多级特征聚合网络的光场显著性目标检测[J]. 计算机工程, 2022, 48(7): 227-233,240.
[11]	张洋, 姚登峰, 江铭虎, 李凡姝. 基于EfficientDet网络的细粒度吸烟行为识别[J]. 计算机工程, 2022, 48(3): 302-309,314.
[12]	黄帅, 张毅. 基于梯形跨尺度特征耦合网络的SAR图像舰船检测[J]. 计算机工程, 2022, 48(12): 270-280.
[13]	赵亮, 方伟. 一种快速渐进式卷积神经网络结构搜索算法[J]. 计算机工程, 2022, 48(12): 134-139,149.
[14]	杨兵, 刘晓芳, 张纠. 基于深度特征聚合网络的医学图像分割[J]. 计算机工程, 2021, 47(4): 187-196.
[15]	叶萌, 杨娟, 汪荣贵, 薛丽霞, 李懂. 基于特征聚合网络的小样本学习方法[J]. 计算机工程, 2021, 47(3): 77-82.

选择文件类型/文献管理软件名称

选择包含的内容