结合双语义数据增强与目标定位的细粒度图像分类

doi:10.19678/j.issn.1000-3428.0060111

摘要/Abstract

摘要： 细粒度图像分类旨在对属于同一基础类别的图像进行更细致的子类划分，其较大的类内差异和较小的类间差异使得提取局部关键特征成为关键所在。提出一种结合双语义数据增强与目标定位的细粒度图像分类算法。为充分提取具有区分度的局部关键特征，在训练阶段基于双线性注意力池化和卷积块注意模块构建注意力学习模块和信息增益模块，分别获取目标局部细节信息和目标重要轮廓这2类不同语义层次的数据，以双语义数据增强的方式提高模型准确率。同时，在测试阶段构建目标定位模块，使模型聚焦于分类目标整体，从而进一步提高分类准确率。实验结果表明，该算法在CUB-200-2011、FGVC Aircraft和Stanford Cars数据集中分别达到89.5%、93.6%和94.7%的分类准确率，较基准网络Inception-V3、双线性注意力池化特征聚合方式以及B-CNN、RA-CNN、MA-CNN等算法具有更好的分类性能。

关键词: 细粒度图像分类, 数据增强, 双线性网络, 注意力学习, 目标定位

Abstract: Fine-grained image classification aims to classify images of the same basic category into more specific subcategories.These images are characterized by large intra-class differences and minor inter-class differences, so the extraction of local key features is crucial to fine-grained image classification.A fine-grained image classification algorithm combining dual semantic data augmentation and target location is proposed.To extract discriminative local key features, two modules are constructed in the training phase to obtain two types of data at different semantic levels.The attention learning module is constructed based on Bilinear Attention Pooling(BAP) to obtain local detail information of the target, and the information gain module is constructed based on Convolutional Block Attention Module(CBAM) to obtain the important contour of the target.Then the accuracy of the model can be improved in the way of dual semantic data augmentation.At the same time, a target location module is built in the testing phase to make the model focus on the overall classification target and further improve the classification accuracy.The experimental results show that the proposed model displays a classification accuracy of 89.5% on CUB-200-2011 dataset, 93.6% on FGVC Aircraft dataset and 94.7% on Stanford Cars dataset, delivering higher performance than benchmark network Inception-V3, Bilinear Attention Pooling(BAP) feature aggregation method, B-CNN, RA-CNN, MA-CNN and other algorithms.

Key words: fine-grained image classification, data augmentation, bilinear network, attention learning, target location

中图分类号:

TP391.41

谭润, 叶武剑, 刘怡俊. 结合双语义数据增强与目标定位的细粒度图像分类[J]. 计算机工程, 2022, 48(2): 237-242,249.

TAN Run, YE Wujian, LIU Yijun. Fine-Grained Image Classification Combining Dual Semantic Data Augmentation and Target Location[J]. Computer Engineering, 2022, 48(2): 237-242,249.

http://www.ecice06.com/CN/Y2022/V48/I2/237

图/表 11

20220301181516

20220301181520

20220301181523

20220301181527

20220301181530

20220301181533

20220301181536

20220301181540

20220301181542

20220301181546

20220301181550

参考文献

[1] 罗建豪, 吴建鑫.基于深度卷积特征的细粒度图像分类研究综述[J].自动化学报, 2017, 43(8):1306-1318. LUO J H, WU J X.Review of fine-grained image classification based on deep convolution feature[J].Acta Automatica Sinica, 2017, 43(8):1306-1318.(in Chinese)
[2] HU T, XU J, HUANG C, et al.Weakly supervised bilinear attention network for fine-grained visual classification[EB/OL].(2018-08-06)[2020-10-21].http://arxiv.org/pdf/1808.02152.pdf.
[3] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].(2015-04-10)[2020-10-21].https://arxiv.org/pdf/1409.1556.pdf.
[4] HE K M, ZHANG X, REN S, et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[5] SZEGEDY C, VANHOUCKE V, IOFFE S, et al.Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:2818-2826.
[6] SZEGEDY C, IOFFE S, VANHOUCKE V, et al.Inception-v4, Inception-ResNet and the impact of residual connections on learning[EB/OL].(2016-08-23)[2020-10-21].https://arxiv.org/pdf/1602.07261.pdf.
[7] ZHANG N, DONAHUE J, GIRSHICK R, et al.Part-based R-CNNs for fine-grained category detection[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2014:834-849.
[8] GIRSHICK R, DONAHUE J, DARRELL T, et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2014:580-587.
[9] BRANSON S, HORN G, BELONGIE S, et al.Bird species categorization using pose normalized deep convolutional nets[EB/OL].(2014-06-11)[2020-10-21].http://de.arxiv.org/pdf/1406.2952.
[10] LIN D, SHEN X, LU C, et al.Deep LAC:deep localization, alignment and classification for fine-grained recognition[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:1666-1674.
[11] JADERBERG M, SIMONYAN K, ZISSERMAN A.Spatial transformer networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2015:2017-2025.
[12] FU J, ZHENG H, MEI T.Look closer to see better:recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:4438-4446.
[13] ZHENG H, FU J, MEI T, et al.Learning multi-attention convolutional neural network for fine-grained image recognition[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:5209-5217.
[14] LIN T Y, ROYCHOWDHURY A, MAJI S.Bilinear CNN models for fine-grained visual recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:1449-1457.
[15] LIN T Y, MAJI S.Improved bilinear pooling with CNNs[EB/OL].(2017-07-21)[2020-10-21].https://arxiv.org/pdf/1707.06772.pdf.
[16] LI P, XIE J, WANG Q, et al.Towards faster training of global covariance pooling networks by iterative matrix square root normalization[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:947-955.
[17] HU T, QI H, HUANG Q, et al.See better before looking closer:weakly supervised data augmentation network for fine-grained visual classification[EB/OL].(2019-03-23)[2020-10-21].https://arxiv.org/pdf/1901.09891.pdf.
[18] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[19] WAH C, BRANSON S, WELINDER P, et al.The caltech-ucsd birds-200-2011 dataset[EB/OL].[2020-10-21].http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=374669091E8C13903183C647B249A20B?doi=10.1.1.372.852&rep=rep1&type=pdf.
[20] MAJI S, RAHTU E, KANNALA J, et al.Fine-grained visual classification of aircraft[EB/OL].(2013-06-21)[2020-10-21].https://arxiv.org/pdf/1306.5151.pdf.
[21] KRAUSE J, STARK M, DENG J, et al.3d object representations for fine-grained categorization[C]//Proceedings of 2013 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2013:554-561.
[22] WANG Y, MORARIU V I, DAVIS L S.Learning a discriminative filter bank within a CNN for fine-grained recognition[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:4148-4157.
[23] YANG Z, LUO T, WANG D, et al.Learning to navigate for fine-grained classification[C]//Proceedings of 2018 European Conference on Computer Vision.Berlin, Germany:Springer, 2018:420-435.

选择文件类型/文献管理软件名称

选择包含的内容