Few-shot Image Classification Method Based on Salient Position Interaction Transformer

doi:10.19678/j.issn.1000-3428.0070134

Abstract

Abstract:

Image classification, a fundamental task in computer vision, has achieved remarkable results on large-scale datasets. However, traditional deep learning methods tend to overfit under low sample size conditions, thereby affecting the model's generalization ability. To address this issue, this study presents a novel small sample image classification method to improve classification performance when sample data are scarce. This method is based on the significant position interaction Transformer and the target classifier, specifically leveraging the structure and advantages of the Vision Transformer (ViT) model. The Interaction Multi-Head Self Attention (HI-MHSA) module with significant position selection is introduced, increasing the interaction between each attention head in the multi-head self-attention module, strengthening the model's attention to significant regions in the input image, saving computational resources, and further improving the learning efficiency and accuracy of the model through the supervision and guidance of the target classifier. Experimental results show that on the miniImageNet, tieredImageNet, and CUB datasets, the proposed method achieves classification accuracies of approximately 67.09%, 72.07%, and 79.82% in a 5-way 1-shot task and approximately 83.54%, 85.62%, and 90.35%, in a 5-way 5-shot task, respectively. Therefore, the proposed method can perform well and is highly practical for small sample image classification tasks.

Key words: few-shot image classification, Transformer, attention mechanism, few-shot classifier, salient position selection

摘要：

图像分类作为计算机视觉的基础任务, 目前在大规模数据集上的研究已取得显著成效。然而, 在低样本量数据条件下, 传统的深度学习方法受制于过拟合问题, 影响模型的泛化能力。为此, 设计一种新颖的小样本图像分类方法, 用于提升模型在样本数据稀缺时的分类性能。该方法基于显著位置相互作用Transformer与目标分类器, 借鉴ViT(Vision Transformer)模型的结构和优势, 引入具有显著位置选择的相互作用多头自注意力(HI-MHSA)模块, 同时增加对多头自注意力模块中各个注意力头之间的交互, 强化模型对输入图像中显著区域的关注, 节省计算资源, 并通过目标分类器的监督指导, 进一步提升模型的学习效率和准确性。实验结果表明, 在miniImageNet、tieredImageNet以及CUB数据集上, 该方法在5-way 1-shot任务中分类准确率分别约为67.09%、72.07%和79.82%, 在5-way 5-shot任务中分类准确率分别约为83.54%、85.62%和90.35%。实验结果显示, 该方法在小样本图像分类任务中具有优秀的性能和高度的实用性。

关键词: 小样本图像分类, Transformer, 注意力机制, 小样本分类器, 显著位置选择

SONG Chaoqi, LIU Ying, HE Jinglu, LI Daxiang. Few-shot Image Classification Method Based on Salient Position Interaction Transformer[J]. Computer Engineering, 2026, 52(2): 167-176.

宋朝琦, 刘颖, 何敬鲁, 李大湘. 基于显著位置交互Transformer的小样本图像分类方法[J]. 计算机工程, 2026, 52(2): 167-176.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0070134

https://www.ecice06.com/EN/Y2026/V52/I2/167

Figures/Tables 12

Fig.1 Overall framework of the model

Fig.2 Structure of image block embedding module

Fig.3 Overall framework of the SPI

Fig.4 The process of generating head token

Fig.5 Output of salient position selection algorithm

Fig.6 Few-shot learner training and target classifier construction

Fig.7 Visualization results of attention maps

Fig.8 t-SNE visualization results

References 50

1	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2012, 60, 84- 90.
2	WANG Y Q, YAO Q M, KWOK J T, et al. Generalizing from a few examples: a survey on few-shot learning. ACM Computing Surveys, 2021, 53(3): 1- 34.
3	张河萍, 方志军, 卢俊鑫, 等. 基于知识增强自适应原型网络的小样本关系分类. 计算机工程, 2025, 51(4): 129- 136. doi: 10.19678/j.issn.1000-3428.0068855
	ZHANG H P, FANG Z J, LU J X, et al. Classification of small sample relationships based on knowledge-enhanced adaptive prototype network. Computer Engineering, 2025, 51(4): 129- 136. doi: 10.19678/j.issn.1000-3428.0068855
4	LI F F, FERGUS R, PERONA P. One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(4): 594- 611. doi: 10.1109/TPAMI.2006.79
5	ZHENG K P, ZHANG H S, HUANG W R. DiffKendall: a novel approach for few-shot learning with differentiable Kendall's rank correlation[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2307.15317.
6	KANG S, HWANG D, EO M, et al. Meta-learning with a geometry-adaptive preconditioner[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2023: 16080-16090.
7	LU Y N, WEN L J, LIU J Z, et al. Self-supervision can be a good few-shot learner[EB/OL]. [2024-05-05]. https://arxiv.org/pdf/2207.09176.
8	HILLER M, HARANDI M, DRUMMOND T. On enforcing better conditioned meta-learning for rapid few-shot adaptation[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2022: 4059-4071.
9	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1706.03762.
10	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2010.11929.
11	SUN C, SHRIVASTAVA A, SINGH S, et al. Revisiting unreasonable effectiveness of data in deep learning era[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2017: 843-852.
12	李清格, 杨小冈, 卢瑞涛, 等. 计算机视觉中的Transformer发展综述. 小型微型计算机系统, 2023, 44(4): 850- 861.
	LI Q G, YANG X G, LU R T, et al. Transformer in computer vision: a survey. Journal of Chinese Computer Systems, 2023, 44(4): 850- 861.
13	LU Z Y, XIE H T, LIU C B, et al. Bridging the gap between vision transformers and convolutional neural networks on small datasets[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2022: 14663-14677.
14	FANG S, LI K Y, LI Z. Salient positions based attention network for image classification[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2106.04996.
15	YE H J, MING L, ZHAN D C, et al. Few-shot learning with a strong teacher. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(3): 1425- 1440. doi: 10.1109/TPAMI.2022.3160362
16	VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2016: 3637-3645.
17	REN M Y, TRIANTAFILLOU E, RAVI S, et al. Meta-learning for semi-supervised few-shot classification[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1803.00676.
18	HE X T, PENG Y X. Fine-grained visual-textual representation learning. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(2): 520- 531. doi: 10.1109/TCSVT.2019.2892802
19	HENDRYCKS D, GIMPEL K. Gaussian Error Linear Units (GELUs)[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1606.08415.
20	MCCULLOCH W S, PITTS W. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biology, 1990, 52(1/2): 99- 115.
21	BRIDLE J S. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition[EB/OL]. [2024-05-05]. https://link.springer.com/chapter/10.1007/978-3-642-76153-9_28.
22	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1711.05101.
23	SUNG F, YANG Y X, ZHANG L, et al. Learning to compare: relation network for few-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 1199-1208.
24	SNELL J, SWERSKY K, ZEMEL R S. Prototypical networks for few-shot learning[C]. //Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 4077-4087.
25	FINN C, ABBEEL P, LEVINE S. Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning. New York, USA: ACM Press, 2017: 1126-1135.
26	CHEN Y B, LIU Z, XU H J, et al. Meta-baseline: exploring simple meta-learning for few-shot learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2022: 9042-9051.
27	YOON S W, SEO J, MOON J. TapNet: neural network augmented with task-adaptive projection for few-shot learning[EB/OL]. [2024-05-05]. https://arxiv.org/pdf/1905.06549.
28	FLENNERHAG S, RUSU A A, PASCANU R, et al. Meta-learning with warped gradient descent[EB/OL]. [2024-05-05]. https://arxiv.org/abs/1909.00025.
29	RAJASEGARAN J, KHAN S, HAYAT M, et al. Meta-learning the learning trends shared across tasks[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2010.09291.
30	FAN C, RAM P, LIU S J. Sign-MAML: efficient model-agnostic meta-learning by SignSGD[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2109.07497.
31	PENG D N, PAN S J. Clustered task-aware meta-learning by learning from learning paths. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 9426- 9438. doi: 10.1109/TPAMI.2023.3250323
32	ABBAS M, XIAO Q W, CHEN L S, et al. Sharp-MAML: sharpness-aware model-agnostic meta learning[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2206.03996.
33	李小雨, 罗娜. 基于迁移类内变化增强数据的小样本学习方法. 计算机工程, 2025, 51(9): 242- 251. doi: 10.19678/j.issn.1000-3428.0069754
	LI X Y, LUO N. Few-shot learning method with augmentation data based on transferring intra-class variations. Computer Engineering, 2025, 51(9): 242- 251. doi: 10.19678/j.issn.1000-3428.0069754
34	PRZEWIEŹLIKOWSKI M, PRZYBYSZ P, TABOR J, et al. HyperMAML: few-shot adaptation of deep models with hypernetworks[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2205.15745.
35	LI G, REN B Y, WANG H Z. EEML: ensemble embedded meta-learning[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2206.09195.
36	CHEN Z Y, GE J X, ZHAN H S, et al. Pareto self-supervised training for few-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2021: 13658-13667.
37	CHEN T, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2002.05709.
38	CHEN X L, HE K M. Exploring simple Siamese representation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2021: 15745-15753.
39	AFHAM M, KHAN S, KHAN M H, et al. Rich semantics improve few-shot learning[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2104.12709.
40	ZHOU Z Q, QIU X, XIE J T, et al. Binocular mutual learning for improving few-shot classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2022: 8382-8391.
41	YE H J, HU H X, ZHAN D C, et al. Few-shot learning via embedding adaptation with set-to-set functions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 8805-8814.
42	HUANG S Y, MA J W, HAN G X, et al. Task-adaptive negative envision for few-shot open-set recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2022: 7161-7170.
43	KANG D, KWON H, MIN J H, et al. Relational embedding for few-shot classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2022: 8802-8813.
44	ATANBORI J, ROSE S. MergedNET: a simple approach for one-shot learning in Siamese networks based on similarity layers. Neurocomputing, 2022, 509, 1- 10.
45	LIU B, CAO Y, LIN Y T, et al. Negative margin matters: understanding margin in few-shot classification[EB/OL]. [2024-05-05]. https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123490426.pdf.
46	ZHANG C, CAI Y J, LIN G S, et al. DeepEMD: few-shot image classification with differentiable earth mover's distance and structured classifiers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 12200-12210.
47	CHEN C F, YANG X S, XU C S, et al. ECKPN: explicit class knowledge propagation network for transductive few-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2021: 6592-6601.
48	KIM M, HOSPEDALES T. Gaussian process meta few-shot classifier learning via linear discriminant Laplace approximation[EB/OL]. [2024-05-05]. https://arxiv.org/abs/2111.05392.
49	SENDERA M, PRZEWIEŹLIKOWSKI M, KARANOWSKI K, et al. HyperShot: few-shot learning by kernel hypernetworks[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Washington D.C., USA: IEEE Press, 2023: 2468-2477.
50	VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(11): 2579- 2605.

[1]	LI Jianlang, WU Xindian, CHEN Ling, YANG Bo, TANG Wensheng. 3D Object Detection Algorithm Based on 4D Millimeter-Wave Radar and Vision Fusion [J]. Computer Engineering, 2026, 52(2): 299-310.
[2]	ZHANG Xinjia, WANG Fang. UAV Image Small Object Detection Algorithm Based on Multi-layer Feature Fusion and Attention Mechanism [J]. Computer Engineering, 2026, 52(2): 148-157.
[3]	WEN Lang, GOU Guanglei, BAI Ruifeng, MIAO Wanyu. Few-shot Fine-grained Image Classification Based on Neighborhood Fusion and Feature Enhancement [J]. Computer Engineering, 2026, 52(2): 158-166.
[4]	WANG Qingrong, HAO Fule, ZHU Changfeng, WANG Junjie. Research on Vehicle Trajectory Prediction Based on Multifeature Fusion [J]. Computer Engineering, 2026, 52(2): 331-341.
[5]	SONG Quanzhen, CHEN Zuojun, QIN Pinle, ZENG Jianchao. Superpixel Guide for Transformer Low-Light Image Denoising Method [J]. Computer Engineering, 2026, 52(2): 186-196.
[6]	LIU Chang, LIANG Bingxue, TIAN Rongkun, QIN Yuhua. Medical and Health Question Classification Based on Multi-feature Fusion and Hybrid Neural Network [J]. Computer Engineering, 2026, 52(2): 342-355.
[7]	XU Xiaoyang, WEI Wei, GAO Chongyang. Infrared Ship Target Detection Based on Improved YOLOv7-tiny [J]. Computer Engineering, 2026, 52(2): 209-220.
[8]	XUE Yang, QIN Yao, ZHANG Shuxiang. Collaborative Recommendation Based on Graph Neural Network Clustering Subgraphs Using Dual Graph Attention Network [J]. Computer Engineering, 2026, 52(2): 89-100.
[9]	DAN Chonghong, WEI Honglei, HE Zhou, WU Guanfeng. SRMpose: Multi-Scale Feature Extraction Keypoint Detection Algorithm [J]. Computer Engineering, 2026, 52(2): 136-147.
[10]	WANG Jun, LI Kunlun, ZHANG Yifei, ZHU Qizhen, LIU Lei, WANG Shuai. Location and Semantic Separation Attention Based Lightweight Visual Object Tracking Algorithm [J]. Computer Engineering, 2026, 52(1): 228-241.
[11]	LI Bingye, BAO Yu, CAO Wei, LI Mingze, XU Fangzheng. Cross-layer Channel Pruning Method Based on Variable Sequences [J]. Computer Engineering, 2026, 52(1): 381-389.
[12]	CHEN Dongji, LAI Huicheng, GAO Guxue, MA Jun, LI Junkai, QUAN Hutuo. Knowledge Distillation-based Transformer for Human-Object Interaction Detection [J]. Computer Engineering, 2026, 52(1): 206-216.
[13]	LI Dongfeng, CHEN Yuren, YU Bo. Pavement Crack Detection Method Based on Multi-Level Feature Fusion [J]. Computer Engineering, 2026, 52(1): 154-165.
[14]	HUANG Jingui, LIU Peng, TANG Wensheng. MMD-YOLOv7: Vehicle Detection Method Under Dark Conditions [J]. Computer Engineering, 2025, 51(9): 340-349.
[15]	FU Jiacheng, TIAN Jin, ZHANG Yujin, FANG Zhijun. Knowledge Graph Recommendation Based on Previous Triple Set [J]. Computer Engineering, 2025, 51(9): 101-109.

Please choose a citation manager

Content to export