Salient Object Detection with Multi-Scale Visual Perception and Fusion

doi:10.19678/j.issn.1000-3428.0066192

Abstract

Abstract:

Most salient object detection algorithms have problems such as single-feature detection defects and insufficient fusion of multiple features, resulting in unclear edges of saliency images and poor background suppression effects. To address these problems, a salient object detection method with multi-scale visual perception and fusion is proposed, which includes a Multiscale Visual Perception Module(MVPM) and a Multi-scale Feature Fusion Module(MFFM), for processing global information of saliency objects and fusing multi-scale features. Based on the U-shaped network structure, void convolution is used to simulate the receptive field in the visual cortex to construct MVPM, fully leveraging the role of void convolution in Convolutional Neural Network(CNN). Global spatial information of salient objects in the backbone network is extracted step by step, thus enhancing foreground saliency regions and suppressing background noise regions. The MFFM is designed by utilizing feature pyramids and spatial attention mechanisms to fuse advanced semantic information with detailed information, thereby restoring spatial structure information of saliency objects while suppressing noise transmission. Experiments conducted on five image datasets with complex background information, including the ECSSD, DUTS, and SOD, showed that the average F-Measure value of this method reached 88.4%, which is 14.2 percentage points higher than the benchmark network U-Net, and the Mean Absolute Error(MAE) value reached 3.5%, which is 5.4 percentage points lower than the benchmark network.

Key words: Convolutional Neural Network(CNN), salient object detection, multi-scale visual perception, multi-scale feature fusion, receptive fields

摘要：

显著性目标检测算法大多存在单一特征检测缺陷和多特征融合不充分等问题，从而导致显著图边缘不清晰以及背景抑制效果较差。为此，提出一种多尺度视觉感知融合的显著性目标检测方法，该方法包含多尺度视觉感知模块(MVPM)和多尺度特征融合模块(MFFM)，分别用于处理显著性目标的全局信息和融合多尺度特征。基于U型网络结构，利用空洞卷积模拟视觉皮层中的感受野以构建MVPM，充分发挥空洞卷积在卷积神经网络中的作用，在主干网络中逐级提取显著性目标的全局空间信息，有效增强前景显著性区域，抑制背景噪声区域。设计MFFM，利用特征金字塔和空间注意力机制将高级语义信息与细节信息相融合，在抑制噪声传递的同时有效恢复显著性目标的空间结构信息。在ECSSD、DUTS、SOD等5个具有复杂背景信息的图像数据集上进行实验，结果表明，该方法的平均F-Measure值达到88.4%，比基准网络U-Net提高14.2个百分点，MAE值达到3.5%，比基准网络降低5.4个百分点。

关键词: 卷积神经网络, 显著性目标检测, 多尺度视觉感知, 多尺度特征融合, 感受野

Zhongren LIU, Li PENG. Salient Object Detection with Multi-Scale Visual Perception and Fusion[J]. Computer Engineering, 2023, 49(12): 186-193.

刘仲任, 彭力. 多尺度视觉感知融合的显著性目标检测[J]. 计算机工程, 2023, 49(12): 186-193.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066192

https://www.ecice06.com/EN/Y2023/V49/I12/186

Figures/Tables 10

Fig.1 Framework of salient object detection network

Fig.2 Multi-scale visual perception module structure

Fig.3 Multi-scale feature fusion module structure

Fig.4 Spatial attention module structure

Fig.5 Comparison of salient detection results of different modules

Fig.6 Comparison results of PR curves of eight algorithms on five datasets

Fig.7 Comparison of salient target detection effects of different algorithms

References 31

1	GAO Y, WANG M, TAO D C, et al. 3-D object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing, 2012, 21(9): 4290- 4303. doi: 10.1109/TIP.2012.2199502
2	王安志, 任春洪, 何淋艳, 等. 基于多模态多级特征聚合网络的光场显著性目标检测. 计算机工程, 2022, 48(7): 227-233, 240. URL
	WANG A Z, REN C H, HE L Y, et al. Light field salient object detection based on multi-modal multi-level feature aggregation network. Computer Engineering, 2022, 48(7): 227-233, 240. URL
3	凌艳, 陈莹. 多尺度上下文信息增强的显著目标检测全卷积网络. 计算机辅助设计与图形学学报, 2019, 31(11): 2007- 2016. URL
	LING Y, CHEN Y. Salient object detection with multiscale context enhanced fully convolutional network. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(11): 2007- 2016. URL
4	KIM J, HAN D, TAI Y W, et al. Salient region detection via high-dimensional color transform[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 883-890.
5	JIA Y Q, HAN M. Category-independent object-level saliency detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2014: 1761-1768.
6	FENG M Y, LU H C, DING E R. Attentive feedback network for boundary-aware salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 1623-1632.
7	ZHUGE Y Z, YANG G, ZHANG P P, et al. Boundary-guided feature aggregation network for salient object detection. IEEE Signal Processing Letters, 2018, 25(12): 1800- 1804. doi: 10.1109/LSP.2018.2875586
8	LUO Z M, MISHRA A, ACHKAR A, et al. Non-local deep features for salient object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 6593-6601.
9	ZHANG P P, WANG D, LU H C, et al. Amulet: aggregating multi-level convolutional features for salient object detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 202-211.
10	NOORI M, MOHAMMADI S, MAJELAN S G, et al. DFNet: discriminative feature extraction and integration network for salient object detection. Engineering Applications of Artificial Intelligence, 2020, 89, 103419. doi: 10.1016/j.engappai.2019.103419
11	LI A X, ZHANG J, LÜ Y Q, et al. Uncertainty-aware joint salient object and camouflaged object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 10066-10076.
12	LIU J, HOU Q, CHENG M, et al. A simple pooling-based design for real-time salient object detection[C]//Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 3912-3921.
13	HOU Q B, CHENG M M, HU X W, et al. Deeply supervised salient object detection with short connections[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 5300-5309.
14	MEI J, CHENG M M, XU G, et al. SANet: a slice-aware network for pulmonary nodule detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8): 4374- 4387.
15	HE D, WANG Y, FANG F. The critical role of V2 population receptive fields in visual orientation crowding. Current Biology, 2019, 29(13): 2229- 2236. doi: 10.1016/j.cub.2019.05.068
16	窦允冲, 侯进, 曾雷鸣, 等. 基于反馈机制与空洞卷积的道路小目标检测网络. 计算机工程, 2023, 49(1): 287- 294. URL
	DOU Y C, HOU J, ZENG L M, et al. Road small target detection network based on feedback mechanism and dilated convolution. Computer Engineering, 2023, 49(1): 287- 294. URL
17	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 770-778.
18	ZHANG L H, WU J, WANG T T, et al. A multistage refinement network for salient object detection. IEEE Transactions on Image Processing, 2020, 29, 3534- 3545. doi: 10.1109/TIP.2019.2962688
19	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[EB/OL]. [2022-10-05]. https://arxiv.org/abs/1807.06521.
20	QIN X B, ZHANG Z C, HUANG C Y, et al. BASNet: boundary-aware salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 7471-7481.
21	FAN D P, ZHANG J, XU G, et al. Salient objects in clutter. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 2344- 2366.
22	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2022-10-05]. https://arxiv.org/abs/1412.6980.pdf.
23	MARGOLIN R, ZELNIK-MANOR L, TAL A. How to evaluate foreground maps[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 248-255.
24	ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2009: 1597-1604.
25	FENG M Y, LU H C, YU Y Z. Residual learning for salient object detection. IEEE Transactions on Image Processing, 2020, 29, 4696- 4708. doi: 10.1109/TIP.2020.2975919
26	LIU N, HAN J W, YANG M H. PiCANet: learning pixel-wise contextual attention for saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3089-3098.
27	ZHANG L, DAI J, LU H C, et al. A bi-directional message passing model for salient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1741-1750.
28	WANG T T, ZHANG L H, WANG S, et al. Detect globally, refine locally: a novel approach to saliency detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 3127-3135.
29	YAO C L, KONG Y Q, FENG L, et al. Contour-aware recurrent cross constraint network for salient object detection. IEEE Access, 2020, 8, 218739- 218751.
30	WU R M, FENG M Y, GUAN W L, et al. A mutual learning method for salient object detection with intertwined multi-supervision[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 8142-8151.
31	WANG L J, LU H C, RUAN X, et al. Deep networks for saliency detection via local estimation and global search[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 3183-3192.

[1]	GAO Yubao, WEN Zhicheng. Dual Decoder Image Denoising Method Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(9): 324-332.
[2]	ZHANG Lu, TIAN Chunwei, SONG Huansheng, LIU Shigang. Multi-Level Dual-Tree Complex Wavelet Network for Low-Dose CT Image Denoising [J]. Computer Engineering, 2024, 50(9): 266-275.
[3]	WANG Zhihao, QIAN Yuntao. Super-Resolution Reconstruction of Spatiotemporal Fusion for Dual-Stream Remote Sensing Images Based on Swin Transformer [J]. Computer Engineering, 2024, 50(9): 33-45.
[4]	Lei WANG, Shipeng DANG, Feng PAN. Model for Predicting Concealed Accessory Pathway Based on Convolutional Neural Network [J]. Computer Engineering, 2024, 50(8): 40-49.
[5]	Lili GENG, Baoning NIU. Convolutional Neural Network Pruning Based on Channel Similarity Entropy [J]. Computer Engineering, 2024, 50(7): 133-143.
[6]	Ruiting NUI, Tianfeng YAN, Rui GAO, Yingzhi WANG. Deep Learning TCNN-MobileNet-Based Modulation Recognition Under Low Signal-to-Noise Radio [J]. Computer Engineering, 2024, 50(7): 204-215.
[7]	YANG Shuo, WANG Yiding. Facial Animation Algorithm Based on Improved Thin Plate Spline Motion Model [J]. Computer Engineering, 2024, 50(6): 255-265.
[8]	Lei ZHANG, Guochen SHEN, Dongxiu OU. Filtering Method for Feature Maps from Convolutional Neural Network for Thermal Imaging Data [J]. Computer Engineering, 2024, 50(4): 31-40.
[9]	ZHANG Lei, SHEN Guochen, OU Dongxiu. Filtering Method for Feature Maps from Convolutional Neural Network for Thermal Imaging Data [J]. Computer Engineering, 2024, 50(4): 31-40.
[10]	Bohan WANG, Xiaoyan JIANG, Liuyi FAN. Semantic Segmentation Improvement Method Based on Deep Supervision for the Construction of Latent Space [J]. Computer Engineering, 2024, 50(3): 191-199.
[11]	Xinlin XIE, Dongxu YIN, Taoyuan ZHANG, Gang XIE. Multiscale Fusion Crowd Counting Algorithm Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(3): 290-297.
[12]	Baihao JIANG, Jing LIU, Dawei QIU, Liang JIANG. Review of Deep Learning Applications in Spinal Image Segmentation [J]. Computer Engineering, 2024, 50(3): 1-15.
[13]	Haochen XU, Manhua LIU. Facial Landmark Detection Based on Hierarchical Self-Attention Network [J]. Computer Engineering, 2024, 50(2): 239-246.
[14]	Qian LI, Weibo WEI, Guangyu YANG, Jintao SONG, Lu SUN, Zhenkuan PAN. Poisson Denoising Variational Model Based on Prior-Driven Deep Neural Network [J]. Computer Engineering, 2024, 50(2): 273-280.
[15]	WANG Yaru, TANG Lu, CHEN Aibin, PENG Weixiong, SHEN Ping. Birdsong Recognition Based on LPDMR-NET [J]. Computer Engineering, 2024, 50(10): 174-184.

Please choose a citation manager

Content to export