多尺度视觉感知融合的显著性目标检测

doi:10.19678/j.issn.1000-3428.0066192

摘要/Abstract

摘要：

显著性目标检测算法大多存在单一特征检测缺陷和多特征融合不充分等问题，从而导致显著图边缘不清晰以及背景抑制效果较差。为此，提出一种多尺度视觉感知融合的显著性目标检测方法，该方法包含多尺度视觉感知模块(MVPM)和多尺度特征融合模块(MFFM)，分别用于处理显著性目标的全局信息和融合多尺度特征。基于U型网络结构，利用空洞卷积模拟视觉皮层中的感受野以构建MVPM，充分发挥空洞卷积在卷积神经网络中的作用，在主干网络中逐级提取显著性目标的全局空间信息，有效增强前景显著性区域，抑制背景噪声区域。设计MFFM，利用特征金字塔和空间注意力机制将高级语义信息与细节信息相融合，在抑制噪声传递的同时有效恢复显著性目标的空间结构信息。在ECSSD、DUTS、SOD等5个具有复杂背景信息的图像数据集上进行实验，结果表明，该方法的平均F-Measure值达到88.4%，比基准网络U-Net提高14.2个百分点，MAE值达到3.5%，比基准网络降低5.4个百分点。

关键词: 卷积神经网络, 显著性目标检测, 多尺度视觉感知, 多尺度特征融合, 感受野

Abstract:

Most salient object detection algorithms have problems such as single-feature detection defects and insufficient fusion of multiple features, resulting in unclear edges of saliency images and poor background suppression effects. To address these problems, a salient object detection method with multi-scale visual perception and fusion is proposed, which includes a Multiscale Visual Perception Module(MVPM) and a Multi-scale Feature Fusion Module(MFFM), for processing global information of saliency objects and fusing multi-scale features. Based on the U-shaped network structure, void convolution is used to simulate the receptive field in the visual cortex to construct MVPM, fully leveraging the role of void convolution in Convolutional Neural Network(CNN). Global spatial information of salient objects in the backbone network is extracted step by step, thus enhancing foreground saliency regions and suppressing background noise regions. The MFFM is designed by utilizing feature pyramids and spatial attention mechanisms to fuse advanced semantic information with detailed information, thereby restoring spatial structure information of saliency objects while suppressing noise transmission. Experiments conducted on five image datasets with complex background information, including the ECSSD, DUTS, and SOD, showed that the average F-Measure value of this method reached 88.4%, which is 14.2 percentage points higher than the benchmark network U-Net, and the Mean Absolute Error(MAE) value reached 3.5%, which is 5.4 percentage points lower than the benchmark network.

Key words: Convolutional Neural Network(CNN), salient object detection, multi-scale visual perception, multi-scale feature fusion, receptive fields

刘仲任, 彭力. 多尺度视觉感知融合的显著性目标检测[J]. 计算机工程, 2023, 49(12): 186-193.

Zhongren LIU, Li PENG. Salient Object Detection with Multi-Scale Visual Perception and Fusion[J]. Computer Engineering, 2023, 49(12): 186-193.

http://www.ecice06.com/CN/Y2023/V49/I12/186

图/表 10

图1 显著性目标检测网络框架

Fig.1 Framework of salient object detection network

图2 多尺度视觉感知模块结构

Fig.2 Multi-scale visual perception module structure

图3 多尺度特征融合模块结构

Fig.3 Multi-scale feature fusion module structure

图4 空间注意力模块结构

Fig.4 Spatial attention module structure

图5 不同模块的显著性检测结果比较

Fig.5 Comparison of salient detection results of different modules

图6 8种算法在5个数据集上的PR曲线比较结果

Fig.6 Comparison results of PR curves of eight algorithms on five datasets

图7 不同算法的显著性目标检测效果对比

Fig.7 Comparison of salient target detection effects of different algorithms

参考文献 31

1	GAO Y, WANG M, TAO D C, et al. 3-D object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing, 2012, 21(9): 4290- 4303. doi: 10.1109/TIP.2012.2199502
2	王安志, 任春洪, 何淋艳, 等. 基于多模态多级特征聚合网络的光场显著性目标检测. 计算机工程, 2022, 48(7): 227-233, 240. URL
	WANG A Z, REN C H, HE L Y, et al. Light field salient object detection based on multi-modal multi-level feature aggregation network. Computer Engineering, 2022, 48(7): 227-233, 240. URL
3	凌艳, 陈莹. 多尺度上下文信息增强的显著目标检测全卷积网络. 计算机辅助设计与图形学学报, 2019, 31(11): 2007- 2016. URL
	LING Y, CHEN Y. Salient object detection with multiscale context enhanced fully convolutional network. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(11): 2007- 2016. URL
4	KIM J, HAN D, TAI Y W, et al. Salient region detection via high-dimensional color transform[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 883-890.
5	JIA Y Q, HAN M. Category-independent object-level saliency detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2014: 1761-1768.
6	FENG M Y, LU H C, DING E R. Attentive feedback network for boundary-aware salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 1623-1632.
7	ZHUGE Y Z, YANG G, ZHANG P P, et al. Boundary-guided feature aggregation network for salient object detection. IEEE Signal Processing Letters, 2018, 25(12): 1800- 1804. doi: 10.1109/LSP.2018.2875586
8	LUO Z M, MISHRA A, ACHKAR A, et al. Non-local deep features for salient object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 6593-6601.
9	ZHANG P P, WANG D, LU H C, et al. Amulet: aggregating multi-level convolutional features for salient object detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 202-211.
10	NOORI M, MOHAMMADI S, MAJELAN S G, et al. DFNet: discriminative feature extraction and integration network for salient object detection. Engineering Applications of Artificial Intelligence, 2020, 89, 103419. doi: 10.1016/j.engappai.2019.103419
11	LI A X, ZHANG J, LÜ Y Q, et al. Uncertainty-aware joint salient object and camouflaged object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 10066-10076.
12	LIU J, HOU Q, CHENG M, et al. A simple pooling-based design for real-time salient object detection[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 3912-3921.
13	HOU Q B, CHENG M M, HU X W, et al. Deeply supervised salient object detection with short connections[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 5300-5309.
14	MEI J, CHENG M M, XU G, et al. SANet: a slice-aware network for pulmonary nodule detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8): 4374- 4387.
15	HE D, WANG Y, FANG F. The critical role of V2 population receptive fields in visual orientation crowding. Current Biology, 2019, 29(13): 2229- 2236. doi: 10.1016/j.cub.2019.05.068
16	窦允冲, 侯进, 曾雷鸣, 等. 基于反馈机制与空洞卷积的道路小目标检测网络. 计算机工程, 2023, 49(1): 287- 294. URL
	DOU Y C, HOU J, ZENG L M, et al. Road small target detection network based on feedback mechanism and dilated convolution. Computer Engineering, 2023, 49(1): 287- 294. URL
17	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
18	ZHANG L H, WU J, WANG T T, et al. A multistage refinement network for salient object detection. IEEE Transactions on Image Processing, 2020, 29, 3534- 3545. doi: 10.1109/TIP.2019.2962688
19	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[EB/OL]. [2022-10-05]. https://arxiv.org/abs/1807.06521.
20	QIN X B, ZHANG Z C, HUANG C Y, et al. BASNet: boundary-aware salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 7471-7481.
21	FAN D P, ZHANG J, XU G, et al. Salient objects in clutter. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 2344- 2366.
22	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2022-10-05]. https://arxiv.org/abs/1412.6980.pdf.
23	MARGOLIN R, ZELNIK-MANOR L, TAL A. How to evaluate foreground maps[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 248-255.
24	ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 1597-1604.
25	FENG M Y, LU H C, YU Y Z. Residual learning for salient object detection. IEEE Transactions on Image Processing, 2020, 29, 4696- 4708. doi: 10.1109/TIP.2020.2975919
26	LIU N, HAN J W, YANG M H. PiCANet: learning pixel-wise contextual attention for saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3089-3098.
27	ZHANG L, DAI J, LU H C, et al. A bi-directional message passing model for salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1741-1750.
28	WANG T T, ZHANG L H, WANG S, et al. Detect globally, refine locally: a novel approach to saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3127-3135.
29	YAO C L, KONG Y Q, FENG L, et al. Contour-aware recurrent cross constraint network for salient object detection. IEEE Access, 2020, 8, 218739- 218751.
30	WU R M, FENG M Y, GUAN W L, et al. A mutual learning method for salient object detection with intertwined multi-supervision[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 8142-8151.
31	WANG L J, LU H C, RUAN X, et al. Deep networks for saliency detection via local estimation and global search[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 3183-3192.

[1]	朱孟栩, 张文豪, 李国洪, 顾行发, 余涛, 郑逢杰, 张丽丽, 吴俣, 邴芳飞, 唐健雄. 基于卷积神经网络的高分六号卫星多光谱图像压缩[J]. 计算机工程, 2023, 49(9): 287-294.
[2]	李现国, 李滨. 基于Transformer和多尺度CNN的图像去模糊[J]. 计算机工程, 2023, 49(9): 226-233, 245.
[3]	杜逸潇, 王红军, 李修和. 基于频谱地图的辐射源指纹定位方法研究[J]. 计算机工程, 2023, 49(9): 183-190, 198.
[4]	胡水. 基于深度强化学习的智能兵棋推演决策方法[J]. 计算机工程, 2023, 49(9): 303-312.
[5]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[6]	李哲铭, 王晋东, 侯建中, 李伟, 张世华, 张恒巍. 基于显著区域优化的对抗样本攻击方法[J]. 计算机工程, 2023, 49(9): 246-255, 264.
[7]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[8]	曹坪, 杨怀志, 薄一军, 尤嘉, 张淳杰, 李丹勇. 面向低质量裂缝图像的多知识蒸馏分类[J]. 计算机工程, 2023, 49(7): 204-213.
[9]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[10]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[11]	代祖华, 刘园园, 狄世龙. 语义增强的图神经网络方面级文本情感分析[J]. 计算机工程, 2023, 49(6): 71-80.
[12]	沈学利, 田桂源, 姜彦吉, 马琳琳. 基于双阶段Conv-Transformer的时频域语音增强算法[J]. 计算机工程, 2023, 49(6): 123-130.
[13]	丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建[J]. 计算机工程, 2023, 49(5): 231-238.
[14]	陈治旭, 靳雁霞, 芦烨, 杨晶, 刘亚变, 史志儒. 基于子图卷积神经网络的多精度服装建模方法[J]. 计算机工程, 2023, 49(4): 174-181.
[15]	曹书鑫, 冯藤藤, 葛凤培, 梁春燕. 基于尺度相关‐双向长短期记忆网络模型的说话人识别[J]. 计算机工程, 2023, 49(4): 289-296.

选择文件类型/文献管理软件名称

选择包含的内容