Transformer Object Tracking Method Based on Real-Time Dynamic Template Update

doi:10.19678/j.issn.1000-3428.0068892

Abstract

Abstract:

Transformer-based object tracking methods are widely used in the field of computer vision and have achieved excellent results. However, object transformations, object occlusion, illumination changes, and rapid object motion can change object information during actual tracking tasks, and consequently, the underutilization of object template change information in existing methods prevents the tracking performance from improving. To solve this problem, this paper presents a Transformer object tracking method, TransTRDT, based on real-time dynamic template update. A dynamic template updating branch is attached to reflect the latest appearance and motion state of an object. The branch determines whether the template is updated through the template quality scoring header; when it identifies the possibility of an update, it passes the initial template, the dynamic template of the previous frame, and the latest prediction after cropping into the dynamic template updating network to update the dynamic template. As a result, the object can be tracked more accurately by obtaining a more reliable template. The tracking performance of TransTRDT on GOT-10k, LsSOT, and TrackingNet is superior to algorithms such as SwinTrack and StarK. It outperforms to achieve a tracking success rate of 71.9% on the OTB100 dataset, with a tracking speed of 36.82 frames per second, reaching the current leading level in the industry.

Key words: object tracking, attention mechanism, dynamic template update, quality scoring header, Transformer object tracking

摘要：

基于Transformer的目标跟踪方法广泛应用在计算机视觉领域, 并取得了优异的效果。但是, 由于在实际跟踪任务中受目标变换、目标遮挡、光照变化以及目标快速运动等因素的影响, 导致目标信息发生变化, 现有方法对目标模板变化信息利用不足, 限制了跟踪性能的提高。为此, 通过附加一条动态模板更新分支反映目标最新的外观和运动状态, 提出一种基于实时动态模板更新的Transformer目标跟踪方法TransTRDT。该分支通过模板质量评分头对模板是否更新进行判断, 当判定可以进行更新时, 随后将初始模板、前一帧动态模板以及裁剪后的最新预测结果传入动态模板更新网络中更新动态模板, 通过获取更可靠的模板从而实现更准确的目标跟踪。在公共数据集上的实验结果表明, TransTRDT在GOT-10k、LaSOT以及TrackingNet上的跟踪性能优于SwinTrack和StarK等算法, 在OTB100中的跟踪成功率为71.9%, 跟踪速度为36.82帧/s, 达到目前行业的领先水平。

关键词: 目标跟踪, 注意力机制, 动态模板更新, 质量评分头, Transformer目标跟踪

SUN Ziwen, QIAN Lizhi, YUAN Guanglin, YANG Chuandong, LING Chong. Transformer Object Tracking Method Based on Real-Time Dynamic Template Update[J]. Computer Engineering, 2025, 51(4): 158-168.

孙子文, 钱立志, 袁广林, 杨传栋, 凌冲. 基于实时动态模板更新的Transformer目标跟踪方法[J]. 计算机工程, 2025, 51(4): 158-168.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068892

https://www.ecice06.com/EN/Y2025/V51/I4/158

Figures/Tables 7

References 42

1	DOE J . Advancements in precision guided munitions. Journal of Military Technology, 2015, 25 (2): 45- 55.
2	HOSSAIN S , LEE D J . Deep learning-based real-time multiple-object detection and tracking from aerial imagery via a flying robot with GPU-based embedded devices. Sensors, 2019, 19 (15): 3371. doi: 10.3390/s19153371
3	黄宏程, 李净, 胡敏, 等. 基于强化学习的机器人认知情感交互模型. 电子与信息学报, 2021, 43 (6): 1781- 1788.
	HUANG H C , LI J , HU M , et al. Cognitive emotional interaction model of robot based on reinforcement learning. Journal of Electronics and Information Technology, 2021, 43 (6): 1781- 1788.
4	邵振峰, 蔡家骏, 王中元, 等. 面向智能监控摄像头的监控视频大数据分析处理. 电子与信息学报, 2017, 39 (5): 1116- 1122.
	SHAO Z F , CAI J J , WANG Z Y , et al. Analytical processing method of big surveillance video data based on smart monitoring cameras. Journal of Electronics and Information Technology, 2017, 39 (5): 1116- 1122.
5	黄宏程, 廖强, 胡敏, 等. 基于知识图谱波纹网络的人机交互模型. 电子与信息学报, 2022, 44 (1): 221- 229.
	HUANG H C , LIAO Q , HU M , et al. Human-computer interaction model based on knowledge graph ripple network. Journal of Electronics and Information Technology, 2022, 44 (1): 221- 229.
6	DAVID S B, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2010: 2544-2550.
7	CHOI J W, CHANG H J, FISCHER T, et al. Context-aware deep feature compression for high-speed visual tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 479-488.
8	DANELLJAN M, HÄGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 4310-4318.
9	HENRIQUES J F , CASEIRO R , MARTINS P , et al. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596. doi: 10.1109/TPAMI.2014.2345390
10	GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking[C]//Proceedings of the International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 1144-1152.
11	LUCA B, JACK V, JOAO F H, et al. Fully-convolutional siamese networks for object tracking[EB/OL]. [2023-10-18]. https://link.springer.com/content/pdf/10.1007/978-3-319-48881-3_56.pdf.
12	LI B, YAN J J, WU W, et al. High performance visual tracking with siamese region proposal network[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 4821-4830.
13	WANG Q, TENG Z, XING J L, et al. Learning attentions: residual attentional siamese network for high performance online visual tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 4854-4863.
14	ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 103-119.
15	DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 472-488.
16	DANELLJAN M, HÄGER G, KHAN F S, et al. Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 1430-1438.
17	DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 6931-6939.
18	PENG J L, JIANG Z K, GU Y Y, et al. SiamRCR: reciprocal classification and regression for visual object tracking[EB/OL]. [2023-10-18]. https://arxiv.org/abs/2105.11237?context=cs.CV.
19	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceeding of the Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 5998-6008.
20	YU B, TANG M, ZHENG L Y, et al. High-performance discriminative tracking with Transformers[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 9836-9845.
21	WANG N, ZHOU W G, WANG J, et al. Transformer meets tracker: exploiting temporal context for robust visual tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 1571-1580.
22	ZHAO M J, OKADA K, INABA M. TrTr: visual tracking with transformer[EB/OL]. [2023-10-18]. https://arxiv.org/abs/2105.03817v1.
23	CHEN X , YAN B , ZHU J W , et al. High-performance Transformer tracking. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2023, 45 (7): 8507- 8523.
24	CHEN X, YAN B, ZHU J W, et al. Transformer tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 8126-8135.
25	YAN B, PENG H W, FU J L. Learning spatio-temporal Transformer for visual tracking[C]//[JP3] Proceedings of the IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 10428-10437.
26	XIE F, CHU L, LI J H, et al. VideoTrack: learning to track objects via video Transformer[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2023: 22826-22835.
27	HE J K, CAN L Z, XIE S, et al. Target-aware tracking with long-term context attention[C]//Proceedings of the Conference on Artificial Intelligence AAAI. [S. l. ]: AAAI Press, 2023: 773-780.
28	GAO A Y, ZHOU C L, ZHANG J. Generalized relation modeling for Transformer tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2023: 18686-18695.
29	WU Q Q, YANG T Y, LIU Z Q, et al. DropMAE: masked autoencoders with spatial-attention dropout for tracking tasks[C]//Proceedings of the Conference on Computer Vision and Pattern Recogntion. Washington D. C., USA: IEEE Press, 2023: 14561-14571.
30	LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[C]//Proceedings of the International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2021: 9992-10002.
31	FAN H, LIN L, YANG F, et al. LaSOT: a high quality benchmark for large scale single object tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 5374-5383.
32	MULLER M, BIBI A, GIANCOLA S, et al. TrackingNet: a large scale dataset and benchmark for object tracking in the wild[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 300-317.
33	HUANG L H , ZHAO X , HUANG K Q . GOT-10k: a large high diversity benchmark for generic object tracking in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (5): 1562- 1577. doi: 10.1109/TPAMI.2019.2957464
34	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[EB/OL]. [2023-10-18]. https://arxiv.org/pdf/1711.05101.
35	MAYER C, DANELLJIA M, BHAT G, et al. Transforming model prediction for tracking[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 8721-8730.
36	LIN L T, FAN H, ZHANG Z P, et al. SwinTrack: a simple and strong baseline for Transformer tracking[EB/OL]. [2023-10-18]. https://arxiv.org/abs/2112.00995.
37	CHEN B Y, LI P X, BAI L, et al. Backbone is all your need: a simplified architecture for visual object tracking[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2021: 375-392.
38	WU Y , LIM J , YANG M H . Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1834- 1848. doi: 10.1109/TPAMI.2014.2388226
39	TANG C M , WANG X , BAI Y C , et al. Learning spatial-frequency transformer for visual object tracking. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33 (9): 5102- 5116. doi: 10.1109/TCSVT.2023.3249468
40	BLATTER P, KANAKIS M, DANELLJAN M, et al. Efficient visual tracking with exemplar Transformers[C]//Proceedings of the Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2023: 1571-1581.
41	CUI Y T , JIANG C , WU G S , et al. MixFormer: end-to-end tracking with iterative mixed attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46 (6): 4129- 4146. doi: 10.1109/TPAMI.2024.3349519
42	GAO S Y, ZHOU C L, MA C. AiATrack: attention in attention for transformer visual tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 146-164.

[1]	ZHOU Hanqi, FANG Dongxu, ZHANG Ningbo, SUN Wensheng. Multi-UAV Multi-Object Tracking Based on Deep Learning [J]. Computer Engineering, 2025, 51(4): 57-65.
[2]	DU Chenyang, ZHANG Xueying, HUANG Lixia, LI Juan. Multi-Feature Speech Emotion Recognition Based on Improved Efficient Channel Attention Mechanism [J]. Computer Engineering, 2025, 51(4): 97-106.
[3]	DONG Hongliang, NIU Yan, SUN Yang, LI Jun. Speech Emotion Recognition Based on Memory Capsules and Attention [J]. Computer Engineering, 2025, 51(4): 169-177.
[4]	XIE Qing, ZHANG Lingfeng, MA Yanchun, LIU Yongjian. Single Image Reflection Removal Model Based on Reflection Classifier and Gradient Restorer [J]. Computer Engineering, 2025, 51(4): 227-238.
[5]	YANG Ping, ZHANG Xi. Improved DeepLabv3+ Road Surface Crack Detection Method [J]. Computer Engineering, 2025, 51(4): 261-270.
[6]	XU Yonggang, SUN Qixuan, LI Fanjia, CHENG Jianwei, DAI Jiajun. Skeleton Behavior Recognition Based on Extended Temporal and Spatiotemporal Feature Fusion Graph Convolutional Network [J]. Computer Engineering, 2025, 51(4): 281-292.
[7]	GENG Xia, WANG Yao. Cloth-Changing Person Re-Identification Method Based on CLIP Enhanced Fine-Grained Features [J]. Computer Engineering, 2025, 51(4): 293-302.
[8]	LIU Yunxiang, LIANG Zhichao. A Highly Efficient Traffic Prediction Model for Continuous Time-series Graph Attention Networks [J]. Computer Engineering, 2025, 51(4): 350-359.
[9]	HU Shulin, ZHANG Huajun, DENG Xiaotao, WANG Zhenghua. Similarity Calculation for Chinese Text Based on Dependency Graph Convolution [J]. Computer Engineering, 2025, 51(3): 76-85.
[10]	LU Peng, ZHONG Chuang. Improved CycleGAN Algorithm for Semi-Supervised Building Extraction [J]. Computer Engineering, 2025, 51(3): 241-251.
[11]	WANG Xinliang, WANG Luying. Safety Helmet Detection Algorithm with Feature Enhancement in Low Light Blasting Scenes [J]. Computer Engineering, 2025, 51(3): 252-260.
[12]	CUI Jinrong, YE Weihao, ZHENG Hong, LIU Tonglai, QI Long, XU Yong. Rice Seedling Counting in Complex Environments Based on Domain-Adaptive NWD-YOLOv5 [J]. Computer Engineering, 2025, 51(3): 320-333.
[13]	SUN Ting, YANG Jie, LI Jiaxuan, WANG Yaozong. Optimization of YOLOv7 Road Sign Detection Algorithm for Low-Light Traffic Scenes [J]. Computer Engineering, 2025, 51(3): 342-351.
[14]	LUAN Fangjun, GONG Qi, YUAN Shuai. Crowd Counting Network Based on Attention Mechanism and Multiscale Fusion [J]. Computer Engineering, 2025, 51(3): 352-361.
[15]	ZHANG Huan, WANG Chen, SHAN Jingdong, QIU Runhe. Elevator Safety Risk Prediction Based on Domain Adaptation and Attention Mechanism [J]. Computer Engineering, 2025, 51(2): 86-93.

Please choose a citation manager

Content to export