Weakly Supervised Object Detection Based on Double Attention Erasure and Attention Information Aggregation

doi:10.19678/j.issn.1000-3428.0063613

Abstract

Abstract: Weakly Supervised Object Detection(WSOD) is mainly accomplished by multi-instance detection networks.However, the application of classification feature extraction networks in such methods causes the detection results to readily converge to the most discriminative local object regions, especially those of non-rigid objects.To address this problem, this paper proposes an end-to-end weakly supervised detection framework based on Double Attention Erasure(DAE) and Attentional Information Aggregation(AIA).DAE aims to generate masks and erase attention regions because including both yields the most discriminative local foreground and background regions.As a result, the most discriminative region of an object can be expanded, and the entire object region is better captured.In addition, to accurately locate the regions for different objects and generate erasure masks, this study employs AIA, which aggregates global and local features in a channel and then further improves the detection accuracy by introducing spatial dependencies.DAE and AIA can synergistically achieve improved weakly supervised detection performance.In this study, extensive experiments are conducted using the PASCAL VOC 2007 and VOC 2012 datasets, on which the proposed method achieves detection accuracies of 50.5% and 47.4%, respectively.The experimental results demonstrate that the proposed method improves the detection accuracy on some non-rigid objects by approximately 5%-20% compared to that of a benchmark model.

Key words: Weakly Supervised Object Detection(WSOD), erasure strategy, attention mechanism, non-rigid objects, deep learning

摘要： 现有的弱监督检测方法主要采用多示例检测网络，但在这些方法中应用分类特征提取网络易使目标尤其是非刚性目标的检测结果收敛到目标最显著局部区域。提出一种基于双注意力擦除和注意力信息聚合的端到端的弱监督检测框架DAENet。双注意力擦除模块的目的在于擦除生成的最显著性局部前景区域和部分背景区域，以此来扩展目标显著性区域，使网络能够尽可能地关注目标整体，从而更好地捕获目标整体区域。此外，为准确定位不同目标区域并精确生成注意力擦除掩码，提出注意力信息聚合模块，该模块可提取通道的全局特征和局部特征，并引入空间依赖性进一步提高检测精度。通过将双注意力擦除和注意力信息聚合进行协同工作，从而更好地提高弱监督检测性能。在PASCAL VOC 2007和VOC 2012数据集上的实验结果表明，DAENet框架在两个数据集上的检测精度分别达到50.5%和47.4%，相比基准模型，在部分非刚性目标上的检测精度提高了约5%~20%。

关键词: 弱监督目标检测, 擦除策略, 注意力机制, 非刚性目标, 深度学习

CLC Number:

TP391

SONG Pengpeng, GONG Shengrong, ZHONG Shan, ZHOU Lifan, FENG Huanghao. Weakly Supervised Object Detection Based on Double Attention Erasure and Attention Information Aggregation[J]. Computer Engineering, 2023, 49(3): 113-120,127.

宋鹏鹏, 龚声蓉, 钟珊, 周立凡, 凤黄浩. 基于双注意力擦除和注意力信息聚合的弱监督目标检测[J]. 计算机工程, 2023, 49(3): 113-120,127.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0063613

http://www.ecice06.com/EN/Y2023/V49/I3/113

Figures/Tables 9

References

[1] ZOU Z X, SHI Z W, GUO Y H, et al.Object detection in 20 years:a survey[EB/OL].[2021-11-10].https://arxiv.org/abs/1905.05055.
[2] CARION N, MASSA F, SYNNAEVE G, et al.End-to-end object detection with transformers[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:213-229.
[3] BOCHKOVSKIY A, WANG C Y, LIAO H Y M.YOLOv4:optimal speed and accuracy of object detection[EB/OL].[2021-11-10].https://arxiv.org/abs/2004.10934.
[4] ZHANG D W, HAN J W, CHENG G, et al.Weakly supervised object localization and detection:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9):5866-5885.
[5] SIVA P, XIANG T.Weakly supervised object detector learning with model drift detection[C]//Proceedings of 2011 International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2011:343-350.
[6] WANG C, REN W, HUANG K Q, et al.Weakly supervised object localization with latent category learning[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2014:431-445.
[7] BILEN H, PEDERSOLI M, TUYTELAARS T.Weakly supervised object detection with convex clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:1081-1089.
[8] BILEN H, VEDALDI A.Weakly supervised deep detection networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:2846-2854.
[9] TANG P, WANG X, BAI X, et al.Multiple instance detection network with online instance classifier refinement[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:2843-2851.
[10] TANG P, WANG X G, BAI S, et al.PCL:proposal cluster learning for weakly supervised object detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(1):176-191.
[11] KOSUGI S, YAMASAKI T, AIZAWA K.Object-aware instance labeling for weakly supervised object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:6064-6072.
[12] LIN C H, WANG S W, XU D Q, et al.Object instance mining for weakly supervised object detection[C]//Proceedings of AAAI Conference on Artificial Intelligence.[S.1.]:AAAI Press, 2020:11482-11489.
[13] ZENI L F, JUNG C R.Distilling knowledge from refinement in multiple instance detection networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2020:3324-3333.
[14] WAN F, WEI P, JIAO J, et al.Min-entropy latent model for weakly supervised object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:2395-2409.
[15] TANG P, WANG X, WANG A, et al.Weakly supervised region proposal network and object detection[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:352-368.
[16] SHEN Y, JI R, CHEN Z, et al.UWSOD:toward fully-supervised-level capacity weakly supervised object detection[C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge, USA:MIT Press, 2020:268-279.
[17] REN Z, YU Z, YANG X, et al.Instance-aware, context-focused, and memory-efficient weakly supervised object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:10598-10607.
[18] ZHONG Y, WANG J, PENG J, et al.Boosting weakly supervised object detection with progressive knowledge transfer[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:615-631.
[19] SINGH K K, LEE Y J.You reap what you sow:using videos to generate high precision object proposals for weakly-supervised object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:9414-9422.
[20] CHENG G, YANG J Y, GAO D C, et al.High-quality proposals for weakly supervised object detection[J].IEEE Transactions on Image Processing, 2020, 29(1):5794-5804.
[21] GAO M, LI A, YU R, et al.C-WSL:count-guided weakly supervised localization[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:152-168.
[22] KIM D, LEE G, JEONG J, et al.Tell me what they're holding:weakly-supervised object detection with transferable knowledge from human-object interaction[C]//Proceedings of AAAI Conference on Artificial Intelligence.[S.1.]:AAAI Press, 2020:11246-11253.
[23] CHEN L Y, YANG T, ZHANG X Y, et al.Points as queries:weakly semi-supervised object detection by points[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2021:8819-8828.
[24] WANG X, GIRSHICK R, GUPTA A, et al.Non-local neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7794-7803.
[25] HU J, SHEN L, SUN G.Squeeze-and-excitation networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7132-7141.
[26] LI X, WANG W, HU X, et al.Selective kernel networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:510-519.
[27] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[28] GAO Z, XIE J, WANG Q, et al.Global second-order pooling convolutional networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:3019-3028.
[29] WANG Q, WU B, ZHU P, et al.ECA-Net:efficient channel attention for deep convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:11531-11539.
[30] DAI Y, GIESEKE F, OEHMCKE S, et al.Attentional feature fusion[C]//Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision.Washington D.C., USA:IEEE Press, 2021:3559-3568.
[31] ZHANG X, WEI Y, FENG J, et al.Adversarial complementary learning for weakly supervised object localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:1325-1334.
[32] CHOE J, SHIM H.Attention-based dropout layer for weakly supervised object localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:2219-2228.
[33] MAI J J, YANG M, LUO W F.Erasing integrated learning:a simple yet effective approach for weakly supervised object localization[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:8763-8772.
[34] WEI Y, FENG J, LIANG X, et al.Object region mining with adversarial erasing:a simple classification to semantic segmentation approach[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:1568-1576.
[35] ZHANG X, WEI Y, KANG G, et al.Self-produced guidance for weakly-supervised object localization[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:597-613.
[36] EVERINGHAM M, ZISSERMAN A, WILLIAMS C K I, et al.The PASCAL visual object classes challenge 2006 results[EB/OL].[2021-11-10].http://host.robots.ox.ac.uk/pascal/VOC/voc2007/guidelines.html.
[37] EVERINGHAM M, ESLAMI S M A, GOOL L, et al.The pascal visual object classes challenge:a retrospective[J].International Journal of Computer Vision, 2015, 111(1):98-136.
[38] UIJLINGS J R R, SANDE K, GEVERS T, et al.Selective search for object recognition[J].International Journal of Computer Vision, 2013, 104(2):154-171.
[39] LI X, KAN M, SHAN S, et al.Weakly supervised object detection with segmentation collaboration[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:9735-9744.
[40] WAN F, LIU C, KE W, et al.C-MIL:continuation multiple instance learning for weakly supervised object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:2199-2208.
[41] JI R Y, LIU Z Y, ZHANG L B, et al.Multi-peak graph-based multi-instance learning for weakly supervised object detection[J].ACM Transactions on Multimedia Computing, Communications, and Applications, 2021, 17(2):1-21.

Please choose a citation manager

Content to export