多区域注意力的细粒度图像分类网络

doi:10.19678/j.issn.1000-3428.0066426

摘要/Abstract

摘要：

目前细粒度图像分类的难点在于如何精准定位图像中高度可辨的局部区域以及其他辅助判别特征。提出一种多区域注意力的细粒度图像分类网络来解决这个问题。首先使用Inception-V3对图像特征进行提取，通过重复使用注意力擦除的方法使模型关注次要特征；然后通过背景去除以及上采样的方法获取图像更精准的局部图像，对提取到的局部特征进行位置统计，并以矩形框的方式获取图像整体，减少细节信息丢失；最后对局部与整体图像进行更加细致的学习。此外，设计联合损失函数，通过动态平衡难易样本和缩小类内差距的方法改善模型的识别效果。实验结果表明，该方法在公开的细粒度图像数据集CUB-200-2011、Stanford-Cars和FGVC-Aircraft上的准确率分别达到89.2%、94.8%、94.0%，相较于对比方法性能更优。

关键词: 多区域注意力, 细粒度图像分类, 擦除策略, 联合损失, 深度学习, 卷积神经网络

Abstract:

The current challenge in fine-grained image classification lies in accurately identifying highly recognizable local areas and auxiliary distinguishing features within an image. To address this issue, a multi-region attention network for fine-grained image classification is proposed. The process begins with the use of Inception-V3 for feature extraction. The model is then directed to focus on secondary features via repeated application of attention erasure. Subsequently, more precise local images are generated by removing the background and employing up-sampling techniques. This is followed by the analysis of the position statistics of the extracted local features. The entire image is then represented as a rectangular box, minimizing the loss of detailed information. Further detailed learning is conducted on local and overall images. Additionally, a joint loss function is designed to enhance the model's recognition capabilities. This is realized by dynamically balancing between difficult and easy samples and reducing intra-class variance. Experimental results on the public fine-grained image datasets CUB-200-2011, Stanford-Cars, and FGVC-Aircraft demonstrate that this method can realize accuracies of 89.2%, 94.8%, and 94.0%, respectively. These figures surpass those achieved by other methods.

Key words: multi-region attention, fine-grained image classification, erasure strategy, joint loss, deep learning, Convolutional Neural Network(CNN)

白尚旺, 王梦瑶, 胡静, 陈志泊. 多区域注意力的细粒度图像分类网络[J]. 计算机工程, 2024, 50(1): 271-278.

Shangwang BAI, Mengyao WANG, Jing HU, Zhibo CHEN. Multi-Region Attention Network for Fine-Grained Image Classification[J]. Computer Engineering, 2024, 50(1): 271-278.

http://www.ecice06.com/CN/Y2024/V50/I1/271

图/表 7

图1 MRA-CNN模型框架

Fig.1 MRA-CNN model framework

图2 多区域特征提取模块架构

Fig.2 Multi-region feature extraction module architecture

图3 裁剪模板

Fig.3 Crop template

图4 保留局部与删除局部的IoU对比

Fig.4 Comparison of IoU when part retained and part deleted

图5 MRA-CNN在3个不同数据集上的准确率变化曲线

Fig.5 Accuracy curves of MRA-CNN on three different datasets

参考文献 30

1	ZHANG N, DONAHUE J, GIRSHICK R, et al. Part-based R-CNNs for fine-grained category detection[C]//Proceedings of ECCV 2014. Berlin, Germany: Springer, 2014: 834-849.
2	BRANSON S, VAN HORN G, PERONA P, et al. Improved bird species recognition using pose normalized deep convolutional nets[C]//Proceedings of 2014 British Machine Vision Conference. Nottingham, UK: The British Machine Vision Association, 2014: 1-14.
3	LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear CNN models for fine-grained visual recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 1449-1457.
4	闫子旭, 侯志强, 熊磊, 等. YOLOv3和双线性特征融合的细粒度图像分类. 中国图象图形学报, 2021, 26(4): 847- 856.
	YAN Z X, HOU Z Q, XIONG L, et al. Fine-grained classification based on bilinear feature fusion and YOLOv3. Journal of Image and Graphics, 2021, 26(4): 847- 856.
5	FU J L, ZHENG H L, MEI T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 4438-4446.
6	刘洋, 金忠. 一种结合非局部和多区域注意力机制的细粒度图像识别方法. 计算机科学, 2021, 48(1): 197- 203.
	LIU Y, JIN Z. Fine-grained image recognition method combining with non-local and multi-region attention mechanism. Computer Science, 2021, 48(1): 197- 203.
7	赵毅力, 徐丹. 联合语义部件的鸟类图像细粒度识别. 计算机辅助设计与图形学学报, 2018, 30(8): 1522- 1529.
	ZHAO Y L, XU D. Joint semantic parts for fine-grained bird images recognition. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(8): 1522- 1529.
8	李新叶, 王光陛. 基于卷积神经网络语义检测的细粒度鸟类识别. 科学技术与工程, 2018, 18(10): 240- 244. doi: 10.3969/j.issn.1671-1815.2018.10.041
	LI X Y, WANG G B. Fine-grained bird recognition based on convolution neural network semantic detection. Science Technology and Engineering, 2018, 18(10): 240- 244. doi: 10.3969/j.issn.1671-1815.2018.10.041
9	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2818-2826.
10	ZHENG H L, FU J L, MEI T, et al. Learning multi-attention convolutional neural network for fine-grained image recognition[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 5209-5217.
11	SUN M, YUAN Y C, ZHOU F, et al. Multi-attention multi-class constraint for fine-grained image recognition[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 834-850.
12	DONAHUE J, JIA Y Q, VINYALS O, et al. DeCAF: a deep convolutional activation feature for generic visual recognition[EB/OL]. [2022-11-10]. https://arxiv.org/abs/1310.1531.pdf.
13	WANG Y M, MORARIU V I, DAVIS L S. Learning a discriminative filter bank within a CNN for fine-grained recognition[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1-10.
14	GE W F, LIN X R, YU Y Z. Weakly supervised complementary parts models for fine-grained image classification from the bottom up[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 3029-3038.
15	CHEN Y E, BAI Y L, ZHANG W, et al. Destruction and construction learning for fine-grained image recognition[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5157-5166.
16	LUO W, ZHANG H M, LI J, et al. Learning semantically enhanced feature for fine-grained image classification. IEEE Signal Processing Letters, 2020, 27, 1545- 1549. doi: 10.1109/LSP.2020.3020227
17	KONG S, FOWLKES C. Low-rank bilinear pooling for fine-grained classification[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 365-374.
18	GAO Y, BEIJBOM O, ZHANG N, et al. Compact bilinear pooling[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 317-326.
19	KIM J H, ON K W, LIM W, et al. Hadamard product for low-rank bilinear pooling[EB/OL]. [2022-11-10]. https://arxiv.org/abs/1610.04325.pdf.
20	WANG J Z, LI N Y, LUO Z M, et al. High-order-interaction for weakly supervised fine-grained visual categorization. Neurocomputing, 2021, 464, 27- 36. doi: 10.1016/j.neucom.2021.08.108
21	CUI Y, ZHOU F, WANG J A, et al. Kernel pooling for convolutional neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3049-3058.
22	ZHENG H L, FU J L, ZHA Z J, et al. Looking for the devil in the details: learning trilinear attention sampling network for fine-grained image recognition[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5007-5016.
23	陆鑫伟, 余鹏飞, 李海燕, 等. 基于注意力自身线性融合的弱监督细粒度图像分类算法. 计算机应用, 2021, 41(5): 1319- 1325.
	LU X W, YU P F, LI H Y, et al. Weakly supervised fine-grained image classification algorithm based on attention-attention bilinear pooling. Journal of Computer Applications, 2021, 41(5): 1319- 1325.
24	胡志伟, 杨华, 黄济民, 等. 基于注意力残差机制的细粒度番茄病害识别. 华南农业大学学报, 2019, 40(6): 124- 132.
	HU Z W, YANG H, HUANG J M, et al. Fine-grained tomato disease recognition based on attention residual mechanism. Journal of South China Agricultural University, 2019, 40(6): 124- 132.
25	ZHANG L B, HUANG S L, LIU W, et al. Learning a mixture of granularity-specific experts for fine-grained categorization[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 8330-8339.
26	JI J S, JIANG L F, LEI C X, et al. Learning two-level features for fine-grained image classification[C]//Proceedings of the 14th IEEE International Conference on Signal Processing. Washington D. C., USA: IEEE Press, 2018: 544-549.
27	DING Y, ZHOU Y Z, ZHU Y, et al. Selective sparse sampling for fine-grained image recognition[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6599-6608.
28	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2999-3007.
29	毛昊, 李新利, 王孝伟, 等. 基于多类别Focal Loss损失函数的变电站场景图像语义分割研究. 华北电力大学学报(自然科学版), 2022, 49(5): 84- 92.
	MAO H, LI X L, WANG X W, et al. Research on semantic segmentation of substation scene image based on multi-class Focal Loss function. Journal of North China Electric Power University(Natural Science Edition), 2022, 49(5): 84- 92.
30	WEN Y D, ZHANG K P, LI Z F, et al. A discriminative feature learning approach for deep face recognition[C]//Proceedings of 2016 ECCV European Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2016: 499-515.

[1]	祝冰艳, 陈志华, 盛斌. 基于感知增强Swin Transformer的遥感图像检测[J]. 计算机工程, 2024, 50(1): 216-223.
[2]	蒋心璐, 陈天恩, 王聪, 赵春江. 大田环境下的农业害虫图像小目标检测算法[J]. 计算机工程, 2024, 50(1): 232-241.
[3]	圣文顺, 余熊峰, 林佳燕, 陈欣. 融合注意力与特征金字塔的小尺度目标检测算法[J]. 计算机工程, 2024, 50(1): 242-250.
[4]	曹广硕, 黄瑞章, 陈艳平, 秦永彬. 基于多模态学习的乳腺癌生存预测研究[J]. 计算机工程, 2024, 50(1): 296-305.
[5]	池亚平, 岳梓岩, 林雨衡. 基于Transformer的SM4算法工作模式识别[J]. 计算机工程, 2023, 49(9): 109-117.
[6]	杜逸潇, 王红军, 李修和. 基于频谱地图的辐射源指纹定位方法研究[J]. 计算机工程, 2023, 49(9): 183-190, 198.
[7]	李现国, 李滨. 基于Transformer和多尺度CNN的图像去模糊[J]. 计算机工程, 2023, 49(9): 226-233, 245.
[8]	林中霖, 时金桥, 王美琪, 王学宾, 王雨燕. 基于应用行为划分的Android恶意应用检测技术[J]. 计算机工程, 2023, 49(9): 125-136.
[9]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[10]	李哲铭, 王晋东, 侯建中, 李伟, 张世华, 张恒巍. 基于显著区域优化的对抗样本攻击方法[J]. 计算机工程, 2023, 49(9): 246-255, 264.
[11]	朱孟栩, 张文豪, 李国洪, 顾行发, 余涛, 郑逢杰, 张丽丽, 吴俣, 邴芳飞, 唐健雄. 基于卷积神经网络的高分六号卫星多光谱图像压缩[J]. 计算机工程, 2023, 49(9): 287-294.
[12]	胡水. 基于深度强化学习的智能兵棋推演决策方法[J]. 计算机工程, 2023, 49(9): 303-312.
[13]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[14]	李泽水, 冀俊忠, 杨翠翠. 基于边权重信息深度网络嵌入的PPIN功能模块检测[J]. 计算机工程, 2023, 49(8): 69-76.
[15]	王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.

选择文件类型/文献管理软件名称

选择包含的内容