基于位置和语义分离注意力机制的轻量视频目标跟踪算法

doi:10.19678/j.issn.1000-3428.0069368

计算机工程 ›› 2026, Vol. 52 ›› Issue (1): 228-241. doi: 10.19678/j.issn.1000-3428.0069368

• 计算机视觉与图形图像处理 • 上一篇下一篇

基于位置和语义分离注意力机制的轻量视频目标跟踪算法

王珺¹, 李昆仑¹^,*(), 张伊菲¹, 朱其振¹, 刘磊², 王帅²

1. 河北大学电子信息工程学院, 河北保定 071000
2. 淮北师范大学计算机科学与技术学院, 安徽淮北 235000

收稿日期:2024-02-06 修回日期:2024-06-26 出版日期:2026-01-15 发布日期:2024-10-14
通讯作者: 李昆仑
作者简介:
王珺(CCF专业会员), 男, 副教授、博士, 主研方向为人工智能、计算机视觉、视频目标跟踪
李昆仑(通信作者), 教授
张伊菲, 学士
朱其振, 硕士研究生
刘磊, 讲师
王帅, 副教授
基金资助:
河北省自然科学基金(F2022201013); 河北省自然科学基金(F2022201055); 河北大学人才引进启动金项目(521100221003); 安徽省高校自然科学基金(KJ2021A0528); 安徽省高校自然科学基金(KJ2020A1202)

Location and Semantic Separation Attention Based Lightweight Visual Object Tracking Algorithm

WANG Jun¹, LI Kunlun¹^,*(), ZHANG Yifei¹, ZHU Qizhen¹, LIU Lei², WANG Shuai²

1. College of Electronic and Information Engineering, Hebei University, Baoding 071000, Hebei, China
2. School of Computer Science and Technology, Huaibei Normal University, Huaibei 235000, Anhui, China

Received:2024-02-06 Revised:2024-06-26 Online:2026-01-15 Published:2024-10-14
Contact: LI Kunlun

摘要/Abstract

摘要：

随着深度大模型技术的不断发展, 基于孪生网络的视频目标跟踪算法主干网络也不断深化, 参数量不断增多, 导致模型训练时间和成本的成倍增长, 对模型在边缘设备上的部署造成了困难。因此, 针对提升轻量级小模型对目标位置和语义信息提取能力的问题, 提出基于位置和语义分离注意力机制的轻量视频目标跟踪算法。首先对归一化注意力机制进行改进并结合水平和竖直方向卷积构建位置注意力, 嵌入到主干网络的浅层特征, 实现对目标位置信息的提取。然后联合通道方向归一化注意力与压缩-激励网络(SENet)注意力, 并将其与主干网络的深层特征进行融合实现对目标语义信息的提取。与之前的注意力机制不同, 分别利用网络中浅层特征有利于空间信息的提取和深层特征有利于语义特征提取的性质将位置注意力和语义注意力分离, 在不明显增加网络参数量的情况下, 提升算法对目标位置和语义信息的提取能力。在通用视频目标跟踪数据集上的实验结果表明, 所提算法能够提升基于轻量级孪生网络跟踪算法的精度和成功率。

关键词: 视频目标跟踪, 孪生网络, 注意力机制, 语义注意力, 位置注意力

Abstract:

With the continuous development of the large deep model, the backbone of Siamese-based visual object tracking is strengthening and the number of parameters is increasing. This has led to doubling of the model training time and cost, making deployment of the model on edge devices challenging. This paper focuses on improving the ability of lightweight models to extract target location and semantic information and proposes a lightweight visual object tracking algorithm based on the location and semantic separation attention mechanism. First, the normalized attention mechanism is improved by combining horizontal and vertical convolutions to construct the position attention and embedding it into the shallow features of the backbone network to extract the target position in the formation. Subsequently, the squeeze-and-excitation network and channel direction normalized attention are fused with the deep features of the backbone network to extract semantic information. In contrast to the previous studies on attention mechanism, this study uses the properties of shallow features that are conducive to spatial information extraction and deep features that are conducive to semantic feature extraction in the network to separate location attention and semantic attention and improves the algorithm's ability to extract the target location and semantic information without significantly increasing the number of parameters. Experimental results on a general tracking dataset demonstrate that the proposed algorithm can improve the precision and success rate of a tracking algorithm based on a lightweight Siamese network.

Key words: visual object tracking, Siamese network, attention mechanism, semantic attention, location attention

王珺, 李昆仑, 张伊菲, 朱其振, 刘磊, 王帅. 基于位置和语义分离注意力机制的轻量视频目标跟踪算法[J]. 计算机工程, 2026, 52(1): 228-241.

WANG Jun, LI Kunlun, ZHANG Yifei, ZHU Qizhen, LIU Lei, WANG Shuai. Location and Semantic Separation Attention Based Lightweight Visual Object Tracking Algorithm[J]. Computer Engineering, 2026, 52(1): 228-241.

https://www.ecice06.com/CN/Y2026/V52/I1/228

图/表 14

图1 提出算法整体结构

Fig.1 The overall structure of the proposed algorithm

图2 NAM-C模块

Fig.2 The NAM-C module

图3 SCNet模块

Fig.3 The SCNet module

图4 SENet模块

Fig.4 The SENet module

图5 NAM-C注意力的有效性验证

Fig.5 Verification of the effectiveness of NAM-C attention

图6 SCNet模块的有效性验证

Fig.6 Verification of the effectiveness of SCNet module

图7 语义注意力模块的有效性验证

Fig.7 Verification of the effectiveness of semantic attention module

图8 OTB100数据集上的跟踪精度和成功率曲线

Fig.8 The tracking precision and success rate curves on OTB100 dataset

图9 4种跟踪挑战下的精度和成功率

Fig.9 The precision and success rates under four tracking challenges

图10 VOT2016数据集中的准确性、鲁棒性和期望平均重叠率

Fig.10 Accuracy, robustness and expected average overlap rate in the VOT2016 dataset

图11 VOT2018数据集中的准确性、鲁棒性和期望平均重叠率

Fig.11 The accuracy, robustness and expected average overlap rate in the VOT2018 dataset

图12 定性分析实验结果

Fig.12 The experimental results of qualitative analysis

参考文献 56

1	单玉刚, 胡卫国. 尺度方向自适应视觉目标跟踪方法综述. 计算机工程与应用, 2020, 56 (9): 13- 23.
	SHAN Y G , HU W G . Review of visual object tracking algorithms of adaptive direction and scale. Computer Engineering and Applications, 2020, 56 (9): 13- 23.
2	张海涛, 秦鹏程. 基于GMS与FPME的视频目标跟踪方法. 计算机工程, 2021, 47 (7): 226- 231. doi: 10.19678/j.issn.1000-3428.0058428
	ZHANG H T , QIN P C . Video target tracking method based on GMS and FPME. Computer Engineering, 2021, 47 (7): 226- 231. doi: 10.19678/j.issn.1000-3428.0058428
3	YANG J C , XU R , CUI J , et al. Robust visual tracking using adaptive local appearance model for smart transportation. Multimedia Tools and Applications, 2016, 75 (24): 17487- 17500. doi: 10.1007/s11042-016-3285-6
4	LI S Y , YEUNG D Y . Visual object tracking for unmanned aerial vehicles: a benchmark and new motion models. Artificial Intelligence, 2017, 31 (1): 358- 366.
5	李雪松, 张锲石, 宋呈群, 等. 自动驾驶场景下的轨迹预测技术综述. 计算机工程, 2023, 49 (5): 1- 11. doi: 10.19678/j.issn.1000-3428.0065627
	LI X S , ZHANG Q S , SONG C Q , et al. Review of trajectory prediction technology in autonomous driving scenes. Computer Engineering, 2023, 49 (5): 1- 11. doi: 10.19678/j.issn.1000-3428.0065627
6	MAKHMUTOVA A, ANIKIN I V, DAGAEVA M. Object tracking method for videomonitoring in intelligent transport systems[C]//Proceedings of the International Russian Automation Conference. Washington D. C., USA: IEEE Press, 2020: 535-540.
7	SONG Y B, MA C, GONG L J, et al. CREST: convolutional residual learning for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2555-2564.
8	BOLME D, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE Press, 2010: 2544-2550.
9	BELHUMEUR P N , HESPANHA J P , KRIEGMAN D J . Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19 (7): 711- 720. doi: 10.1109/34.598228
10	厍向阳, 罗佳琪, 任海青, 等. 基于稀疏表示的相关滤波目标跟踪算法. 计算机工程与应用, 2023, 59 (11): 71- 79.
	SHE X Y , LUO J Q , REN H Q , et al. Correlation filter for object tracking method based on spare representation. Computer Engineering and Applications, 2023, 59 (11): 71- 79.
11	谢余庆, 黄旭东, 胡丽莹. 基于正则化正交非负矩阵分解的旋转目标检测方法. 福建师范大学学报(自然科学版), 2024, 40 (1): 106- 115.
	XIE Y Q , HUANG X D , HU L Y . Rotating target detection method based on regularized orthogonal nonnegative matrix factorization. Journal of Fujian Normal University (Natural Science Edition), 2024, 40 (1): 106- 115.
12	HENRIQUES J F , CASEIRO R , MARTINS P , et al. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596. doi: 10.1109/TPAMI.2014.2345390
13	KOCH G, ZEMEL R, SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition[C]// Proceedings of the ICML Deep Learning Workshop. Washington D. C., USA: IEEE Press, 2015: 211-223.
14	梁浩, 刘克俭, 刘康, 等. 引入再检测机制的孪生神经网络目标跟踪. 光学精密工程, 2019, 27 (7): 1621.
	LIANG H , LIU K J , LIU K , et al. Siamese network tracking with redetection mechanism. Optics and Precision Engineering, 2019, 27 (7): 1621.
15	NAM H, HAN B. Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 4293-4302.
16	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 850-865.
17	LI B, YAN J J, WU W, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 8971-8980.
18	LI B, WU W, WANG Q, et al. SiamRPN++: evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 4282-4291.
19	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 770-778.
20	WEI B B , CHEN H Y , DING Q H , et al. SiamOAN: Siamese object-aware network for real-time target tracking. Neurocomputing, 2022, 471, 161- 174. doi: 10.1016/j.neucom.2021.10.112
21	WEI B B , CHEN H Y , DING Q H , et al. SiamAGN: Siamese attention-guided network for visual tracking. Neurocomputing, 2022, 324
22	BAO J H , CHEN K Q , SUN X , et al. SiamTHN: Siamese target highlight network for visual tracking. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 35 (7): 7061- 7074.
23	LI Y F, WANG B, WU X Y, et al. Lightweight full-convolutional Siamese tracker[EB/OL]. [2024-01-03]. https://arxiv.org//pdf/2310.05393.
24	SUN P, CAO J, JIANG Y, et al. Transtrack: multiple object tracking with transformer[EB/OL]. [2024-01-03]. https://arxiv.org//pdf/2012.15460.
25	CHEN H , WU C , DU B , et al. Change detection in multisource VHR images via deep Siamese convolutional multiple-layers recurrent neural network. IEEE Transactions on Geoscience and Remote Sensing, 2019, 58 (4): 2848- 2864.
26	WU Z H , PAN S R , CHEN F W , et al. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32 (1): 4- 24.
27	CHEN X, YAN B, ZHU J, et al. Transformer tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 8126-8135.
28	GAO S Y, ZHOU C L, MA C, et al. AiATrack: attention in attention for transformer visual tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 146-164.
29	BORSUK V, VEI R, KUPYN O, et al. FEAR: fast, efficient, accurate and robust visual tracker[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 644-663.
30	BLATTER P, KANAKIS M, DANELLJAN M, et al. Efficient visual tracking with exemplar transformers[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Washington D. C., USA: IEEE Press, 2023: 1571-1581.
31	WEI Q, ZENG B, LIU J, et al. LiteTrack: layer pruning with asynchronous feature extraction for lightweight and efficient visual tracking[EB/OL]. [2024-01-03]. https://arxiv.org//pdf/2309.09249.
32	HE Q B , SUN X , YAN Z Y , et al. Multi-object tracking in satellite videos with graph-based multitask modeling. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, 1423- 1431.
33	PARMIGGIANI A , LIU D , PSOTA E , et al. Don't get lost in the crowd: graph convolutional network for online animal tracking in dense groups. Computers and Electronics in Agriculture, 2023, 212, 108038. doi: 10.1016/j.compag.2023.108038
34	LIN J , LI S Y , YANG X , et al. CS-ViG-UNet: infrared small and dim target detection based on cycle shift vision graph convolution network. Expert Systems with Applications, 2024, 254, 124385. doi: 10.1016/j.eswa.2024.124385
35	DU Y H , YAN Y , CHEN S , et al. Object-adaptive LSTM network for real-time visual tracking with adversarial data augmentation. Neurocomputing, 2020, 384, 67- 83. doi: 10.1016/j.neucom.2019.12.022
36	JAIN M , SUBRAMANYAM A V , DENMAN S , et al. LSTM guided ensemble correlation filter tracking with appearance model pool. Computer Vision and Image Understanding, 2020, 195, 102935. doi: 10.1016/j.cviu.2020.102935
37	LIU Y, SHAO Z, TENG Y, et al. NAM: normalization-based attention module[EB/OL]. [2024-01-03]. https://arxiv.org//pdf/2111.12419.
38	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 13713-13722.
39	HUANG L H , ZHAO X , HUANG K Q . GOT-10k: a large high-diversity benchmark for generic object tracking in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43 (5): 1562- 1577.
40	WU Y , LIM J , YANG M H . Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1834- 1848. doi: 10.1109/TPAMI.2014.2388226
41	KRISTAN M, LEONARDIS A, MATAS J, et al. The visual object tracking VOT2016 challenge results[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 777-823.
42	KRISTAN M, LEONARDIS A, MATAS J, et al. The sixth visual object tracking VOT2018 challenge results[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2019: 44-53.
43	LI Y, ZHU J K. A scale adaptive kernel correlation filter tracker with feature integration[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2015: 254-265.
44	DANELLJAN M, HAGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 4310-4318.
45	ZHANG J M, MA S G, SCLAROFF S. MEEM: robust tracking via multiple experts using entropy minimization[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 188-203.
46	DANELLJAN M, HÄGER G, SHAHBAZ KHAN F, et al. Accurate scale estimation for robust visual tracking[C]//Proceedings of the British Machine Vision Conference. Washington D. C., USA: IEEE Press, 2014: 1-5.
47	BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple: complementary learners for real-time tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 1401-1409.
48	DONG X P, SHEN J B. Triplet loss in siamese network for object tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 472-488.
49	SHEN Z L, DAI Y C, RAO Z B. CFNet: cascade and fused cost volume for robust stereo matching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE Press, 2021: 13906-13915.
50	VOJIR T , NOSKOVA J , MATAS J . Robust scale-adaptive mean-shift for tracking. Pattern Recognition Letters, 2014, 49, 250- 258. doi: 10.1016/j.patrec.2014.03.025
51	DANELLJAN M , HAGER G , KHAN F S , et al. Discriminative scale space tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 39 (8): 1561- 1575.
52	MA C, HUANG J B, YANG X K, et al. Hierarchical convolutional features for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 3074-3082.
53	DANELLJAN M, ROBINSON A, SHAHBAZ KHAN F, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 472-488.
54	CUI Y T, JIANG C, WANG L M, et al. MixFormer: end-to-end tracking with iterative mixed attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 13608-13618.
55	CAI Y D, LIU J, TANG J, et al. Robust object modeling for visual tracking[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2023: 9589-9600.
56	SHI L, ZHONG B, LIANG Q, et al. Explicit visual prompts for visual object tracking[EB/OL]. [2024-01-03]. https://arxiv.org//pdf/03142.

[1]	黎东丰, 陈雨人, 余博. 基于多层次特征融合的路面裂缝检测方法[J]. 计算机工程, 2026, 52(1): 154-165.
[2]	翟志鹏, 曹阳, 沈琴琴, 施佺. 基于多时空图融合与动态注意力的交通流预测[J]. 计算机工程, 2025, 51(9): 139-148.
[3]	马淦, 谷雨, 彭冬亮. 结合改进YOLOv5s和动态数据增强的海面舰船检测[J]. 计算机工程, 2025, 51(9): 294-305.
[4]	陈彦如, 刘珂良, 冉茂亮. 基于深度强化学习的外卖即时配送实时优化[J]. 计算机工程, 2025, 51(9): 328-339.
[5]	黄金贵, 刘朋, 唐文胜. MMD-YOLOv7:黑暗条件下车辆检测方法[J]. 计算机工程, 2025, 51(9): 340-349.
[6]	符家成, 田瑾, 张玉金, 方志军. 结合前置三元组集的知识图谱推荐[J]. 计算机工程, 2025, 51(9): 101-109.
[7]	倪源松, 韩军, 邹小燕, 胡广怡, 王文帅. 两阶段自适应分块输电线路螺栓缺陷检测方法[J]. 计算机工程, 2025, 51(8): 281-291.
[8]	郝宏达, 罗健旭. 基于多尺度区域特征融合的多器官语义分割模型[J]. 计算机工程, 2025, 51(8): 270-280.
[9]	张昭理, 李家豪, 刘海, 石佛波, 何嘉文. 基于个性化遗忘建模的知识追踪方法[J]. 计算机工程, 2025, 51(8): 120-130.
[10]	闫建红, 刘芝妍, 王震. 融合时空注意力机制的多尺度卷积车辆轨迹预测[J]. 计算机工程, 2025, 51(8): 406-414.
[11]	宋杰, 徐慧英, 朱信忠, 黄晓, 陈晨, 王泽宇. 基于YOLOv8改进的跌倒检测算法: OEF-YOLO[J]. 计算机工程, 2025, 51(7): 127-139.
[12]	彭菊红, 张弛, 高谦, 张光明, 谈栋华, 赵明俊. 基于改进的YOLOv8算法的钢材缺陷检测[J]. 计算机工程, 2025, 51(7): 152-160.
[13]	刘春霞, 孟吉星, 潘理虎, 龚大立. 融合RGB与IR图像的遥感小目标检测方法[J]. 计算机工程, 2025, 51(7): 326-338.
[14]	栾孟娜, 郑秋梅, 王风华. 基于DMC-YOLO的交通标志实时检测算法[J]. 计算机工程, 2025, 51(7): 90-99.
[15]	华家宝, 张京瑞, 朱福民, 陈璐. 基于路侧相机的自适应空间变换车辆检测方法[J]. 计算机工程, 2025, 51(6): 349-359.

选择文件类型/文献管理软件名称

选择包含的内容

基于位置和语义分离注意力机制的轻量视频目标跟踪算法

Location and Semantic Separation Attention Based Lightweight Visual Object Tracking Algorithm

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 56

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于位置和语义分离注意力机制的轻量视频目标跟踪算法

Location and Semantic Separation Attention Based Lightweight Visual Object Tracking Algorithm

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 56

相关文章 15

编辑推荐

Metrics

本文评价