多区域注意力的细粒度图像分类网络

doi:10.19678/j.issn.1000-3428.0066426

摘要/Abstract

摘要：

目前细粒度图像分类的难点在于如何精准定位图像中高度可辨的局部区域以及其他辅助判别特征。提出一种多区域注意力的细粒度图像分类网络来解决这个问题。首先使用Inception-V3对图像特征进行提取，通过重复使用注意力擦除的方法使模型关注次要特征；然后通过背景去除以及上采样的方法获取图像更精准的局部图像，对提取到的局部特征进行位置统计，并以矩形框的方式获取图像整体，减少细节信息丢失；最后对局部与整体图像进行更加细致的学习。此外，设计联合损失函数，通过动态平衡难易样本和缩小类内差距的方法改善模型的识别效果。实验结果表明，该方法在公开的细粒度图像数据集CUB-200-2011、Stanford-Cars和FGVC-Aircraft上的准确率分别达到89.2%、94.8%、94.0%，相较于对比方法性能更优。

关键词: 多区域注意力, 细粒度图像分类, 擦除策略, 联合损失, 深度学习, 卷积神经网络

Abstract:

The current challenge in fine-grained image classification lies in accurately identifying highly recognizable local areas and auxiliary distinguishing features within an image. To address this issue, a multi-region attention network for fine-grained image classification is proposed. The process begins with the use of Inception-V3 for feature extraction. The model is then directed to focus on secondary features via repeated application of attention erasure. Subsequently, more precise local images are generated by removing the background and employing up-sampling techniques. This is followed by the analysis of the position statistics of the extracted local features. The entire image is then represented as a rectangular box, minimizing the loss of detailed information. Further detailed learning is conducted on local and overall images. Additionally, a joint loss function is designed to enhance the model's recognition capabilities. This is realized by dynamically balancing between difficult and easy samples and reducing intra-class variance. Experimental results on the public fine-grained image datasets CUB-200-2011, Stanford-Cars, and FGVC-Aircraft demonstrate that this method can realize accuracies of 89.2%, 94.8%, and 94.0%, respectively. These figures surpass those achieved by other methods.

Key words: multi-region attention, fine-grained image classification, erasure strategy, joint loss, deep learning, Convolutional Neural Network(CNN)

白尚旺, 王梦瑶, 胡静, 陈志泊. 多区域注意力的细粒度图像分类网络[J]. 计算机工程, 2024, 50(1): 271-278.

Shangwang BAI, Mengyao WANG, Jing HU, Zhibo CHEN. Multi-Region Attention Network for Fine-Grained Image Classification[J]. Computer Engineering, 2024, 50(1): 271-278.

https://www.ecice06.com/CN/Y2024/V50/I1/271

图/表 7

图1 MRA-CNN模型框架

Fig.1 MRA-CNN model framework

图2 多区域特征提取模块架构

Fig.2 Multi-region feature extraction module architecture

图3 裁剪模板

Fig.3 Crop template

图4 保留局部与删除局部的IoU对比

Fig.4 Comparison of IoU when part retained and part deleted

图5 MRA-CNN在3个不同数据集上的准确率变化曲线

Fig.5 Accuracy curves of MRA-CNN on three different datasets

参考文献 30

1	ZHANG N, DONAHUE J, GIRSHICK R, et al. Part-based R-CNNs for fine-grained category detection[C]//Proceedings of ECCV 2014. Berlin, Germany: Springer, 2014: 834-849.
2	BRANSON S, VAN HORN G, PERONA P, et al. Improved bird species recognition using pose normalized deep convolutional nets[C]//Proceedings of 2014 British Machine Vision Conference. Nottingham, UK: The British Machine Vision Association, 2014: 1-14.
3	LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear CNN models for fine-grained visual recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 1449-1457.
4	闫子旭, 侯志强, 熊磊, 等. YOLOv3和双线性特征融合的细粒度图像分类. 中国图象图形学报, 2021, 26(4): 847- 856.
	YAN Z X, HOU Z Q, XIONG L, et al. Fine-grained classification based on bilinear feature fusion and YOLOv3. Journal of Image and Graphics, 2021, 26(4): 847- 856.
5	FU J L, ZHENG H L, MEI T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 4438-4446.
6	刘洋, 金忠. 一种结合非局部和多区域注意力机制的细粒度图像识别方法. 计算机科学, 2021, 48(1): 197- 203.
	LIU Y, JIN Z. Fine-grained image recognition method combining with non-local and multi-region attention mechanism. Computer Science, 2021, 48(1): 197- 203.
7	赵毅力, 徐丹. 联合语义部件的鸟类图像细粒度识别. 计算机辅助设计与图形学学报, 2018, 30(8): 1522- 1529.
	ZHAO Y L, XU D. Joint semantic parts for fine-grained bird images recognition. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(8): 1522- 1529.
8	李新叶, 王光陛. 基于卷积神经网络语义检测的细粒度鸟类识别. 科学技术与工程, 2018, 18(10): 240- 244. doi: 10.3969/j.issn.1671-1815.2018.10.041
	LI X Y, WANG G B. Fine-grained bird recognition based on convolution neural network semantic detection. Science Technology and Engineering, 2018, 18(10): 240- 244. doi: 10.3969/j.issn.1671-1815.2018.10.041
9	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2818-2826.
10	ZHENG H L, FU J L, MEI T, et al. Learning multi-attention convolutional neural network for fine-grained image recognition[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 5209-5217.
11	SUN M, YUAN Y C, ZHOU F, et al. Multi-attention multi-class constraint for fine-grained image recognition[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 834-850.
12	DONAHUE J, JIA Y Q, VINYALS O, et al. DeCAF: a deep convolutional activation feature for generic visual recognition[EB/OL]. [2022-11-10]. https://arxiv.org/abs/1310.1531.pdf.
13	WANG Y M, MORARIU V I, DAVIS L S. Learning a discriminative filter bank within a CNN for fine-grained recognition[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1-10.
14	GE W F, LIN X R, YU Y Z. Weakly supervised complementary parts models for fine-grained image classification from the bottom up[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 3029-3038.
15	CHEN Y E, BAI Y L, ZHANG W, et al. Destruction and construction learning for fine-grained image recognition[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5157-5166.
16	LUO W, ZHANG H M, LI J, et al. Learning semantically enhanced feature for fine-grained image classification. IEEE Signal Processing Letters, 2020, 27, 1545- 1549. doi: 10.1109/LSP.2020.3020227
17	KONG S, FOWLKES C. Low-rank bilinear pooling for fine-grained classification[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 365-374.
18	GAO Y, BEIJBOM O, ZHANG N, et al. Compact bilinear pooling[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 317-326.
19	KIM J H, ON K W, LIM W, et al. Hadamard product for low-rank bilinear pooling[EB/OL]. [2022-11-10]. https://arxiv.org/abs/1610.04325.pdf.
20	WANG J Z, LI N Y, LUO Z M, et al. High-order-interaction for weakly supervised fine-grained visual categorization. Neurocomputing, 2021, 464, 27- 36. doi: 10.1016/j.neucom.2021.08.108
21	CUI Y, ZHOU F, WANG J A, et al. Kernel pooling for convolutional neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3049-3058.
22	ZHENG H L, FU J L, ZHA Z J, et al. Looking for the devil in the details: learning trilinear attention sampling network for fine-grained image recognition[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5007-5016.
23	陆鑫伟, 余鹏飞, 李海燕, 等. 基于注意力自身线性融合的弱监督细粒度图像分类算法. 计算机应用, 2021, 41(5): 1319- 1325.
	LU X W, YU P F, LI H Y, et al. Weakly supervised fine-grained image classification algorithm based on attention-attention bilinear pooling. Journal of Computer Applications, 2021, 41(5): 1319- 1325.
24	胡志伟, 杨华, 黄济民, 等. 基于注意力残差机制的细粒度番茄病害识别. 华南农业大学学报, 2019, 40(6): 124- 132.
	HU Z W, YANG H, HUANG J M, et al. Fine-grained tomato disease recognition based on attention residual mechanism. Journal of South China Agricultural University, 2019, 40(6): 124- 132.
25	ZHANG L B, HUANG S L, LIU W, et al. Learning a mixture of granularity-specific experts for fine-grained categorization[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 8330-8339.
26	JI J S, JIANG L F, LEI C X, et al. Learning two-level features for fine-grained image classification[C]//Proceedings of the 14th IEEE International Conference on Signal Processing. Washington D. C., USA: IEEE Press, 2018: 544-549.
27	DING Y, ZHOU Y Z, ZHU Y, et al. Selective sparse sampling for fine-grained image recognition[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6599-6608.
28	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2999-3007.
29	毛昊, 李新利, 王孝伟, 等. 基于多类别Focal Loss损失函数的变电站场景图像语义分割研究. 华北电力大学学报(自然科学版), 2022, 49(5): 84- 92.
	MAO H, LI X L, WANG X W, et al. Research on semantic segmentation of substation scene image based on multi-class Focal Loss function. Journal of North China Electric Power University(Natural Science Edition), 2022, 49(5): 84- 92.
30	WEN Y D, ZHANG K P, LI Z F, et al. A discriminative feature learning approach for deep face recognition[C]//Proceedings of 2016 ECCV European Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2016: 499-515.

[1]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[2]	高煜宝, 文志诚. 基于注意力机制的双路解码器图像去噪方法[J]. 计算机工程, 2024, 50(9): 324-332.
[3]	魏嵬, 丁香香, 郭梦星, 杨钊, 刘辉. 文本相似度计算方法综述[J]. 计算机工程, 2024, 50(9): 18-32.
[4]	王志浩, 钱沄涛. 基于Swin Transformer的双流遥感图像时空融合超分辨率重建[J]. 计算机工程, 2024, 50(9): 33-45.
[5]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[6]	张鲁, 田春伟, 宋焕生, 刘侍刚. 用于低剂量CT图像去噪的多级双树复小波网络[J]. 计算机工程, 2024, 50(9): 266-275.
[7]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[8]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[9]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.
[10]	张亚洲, 和玉, 戎璐, 王祥凯. 基于上下文知识增强型Transformer网络的抑郁检测[J]. 计算机工程, 2024, 50(8): 75-85.
[11]	高爽, 史轶伦, 徐巧枝, 于磊. 基于对比学习的非对称编解码结构的心脏MRI分割研究[J]. 计算机工程, 2024, 50(8): 290-300.
[12]	高伟, 李帅龙, 茆琳, 王磊, 李颖颖, 韩林. 一种基于TVM的算子生成加速策略[J]. 计算机工程, 2024, 50(8): 353-362.
[13]	王宇, 祁琦, 王纯, 许才. 储能变流器信号高精度故障诊断方法[J]. 计算机工程, 2024, 50(8): 389-396.
[14]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[15]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.

选择文件类型/文献管理软件名称

选择包含的内容