基于特征融合与双模板嵌套更新的孪生网络跟踪算法

doi:10.19678/j.issn.1000-3428.0058437

计算机工程 ›› 2021, Vol. 47 ›› Issue (7): 239-248. doi: 10.19678/j.issn.1000-3428.0058437

基于特征融合与双模板嵌套更新的孪生网络跟踪算法

任立成^1,2, 杨嘉棋^1,2, 魏宇星¹, 张建林¹

1. 中国科学院光电技术研究所, 成都 610209;
2. 中国科学院大学计算机科学与技术学院, 北京 100049

收稿日期:2020-05-26 修回日期:2020-06-30 发布日期:2020-07-10
作者简介:任立成(1994-),男,硕士研究生,主研方向深度学习、机器学习、目标跟踪;杨嘉棋,硕士研究生;魏宇星(通信作者),副研究员;张建林,研究员、博士生导师。
基金资助:
国家重点研发计划（G158207）。

Tracking Algorithm Using Siamese Network Based on Feature Fusion and Dual-Template Nested Update

REN Licheng^1,2, YANG Jiaqi^1,2, WEI Yuxing¹, ZHANG Jianlin¹

1. Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China;
2. School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China

Received:2020-05-26 Revised:2020-06-30 Published:2020-07-10

摘要/Abstract

摘要： 为提高全卷积孪生网络SiamFC在复杂场景下的识别和定位能力，提出一种基于多响应图融合与双模板嵌套更新的实时目标跟踪算法。使用深度ResNet-22替换AlexNet作为骨干网络以提升网络特征提取性能，建立强识别能力的骨干语义分支。在ResNet-22的浅层使用高分辨率特征，构造强定位能力的浅层位置分支，计算并融合两个分支响应。通过高置信度的双模板嵌套更新机制对两个分支的模板进行更新，以适应目标的外观和位置变化。在OTB2015和VOT2016数据集上的实验结果表明，与基于SiamFC、SiamDW等的目标跟踪算法相比，该算法在目标快速移动、遮挡等复杂场景下跟踪效果更稳定，并且运行速度达到34 frame/s，满足实时性要求。

关键词: 孪生网络, 目标跟踪, ResNet-22结构, 语义分支, 位置分支, 双模板嵌套更新

Abstract: In order to improve the recognition and positioning performance of the fully convolutional Siamese network(SiamFC) in complex scenarios,a real-time visual tracking algorithm with fused multiple response graphs and dual-template nested update mechanism is proposed.The algorithm employs the deep network,ResNet-22,to replace AlexNet as the backbone network for stronger feature extraction ability,and the semantic branch of backbone with enhanced recognition ability is built.The high-resolution feature is used in the shallow layer of ResNet-22 to construct the shallow position branch with strong positioning ability.Then the responses of the two branches are calculated and fused.In addition,the templates of the two branches are updated by using a high-confidence dual-template nested update mechanism to adapt to the changes in the appearance and position of the target.Experimental results on the datasets of OTB2015 and VOT2016 show that the algorithm is more stable than tracking algorithms based on SiamFC,SiamDW and other networks in the scenarios with Fast Motion(FM) and Occlusion(OCC).At the same time,the algorithm runs at the speed of 34 frame/s,providing required real-time performance.

Key words: Siamese network, target tracking, ResNet-22 structure, semantic branch, location branch, dual-template nested update

中图分类号:

TP751

任立成, 杨嘉棋, 魏宇星, 张建林. 基于特征融合与双模板嵌套更新的孪生网络跟踪算法[J]. 计算机工程, 2021, 47(7): 239-248.

REN Licheng, YANG Jiaqi, WEI Yuxing, ZHANG Jianlin. Tracking Algorithm Using Siamese Network Based on Feature Fusion and Dual-Template Nested Update[J]. Computer Engineering, 2021, 47(7): 239-248.

https://www.ecice06.com/CN/Y2021/V47/I7/239

图/表 11

20210721091419

20210721091423

20210721091426

20210721091430

20210721091433

20210721091436

20210721091439

20210721091442

20210721091445

20210721091448

20210721091452

参考文献

[1] 李珑,刘凯,李玲.基于目标检测的时空上下文跟踪算法[J].计算机工程,2018,44(9):263-268,273. LI L,LIU K,LI L.Space-time context tracking algorithm based on target detection[J].Computer Engineering,2018,44(9):263-268,273.(in Chinese)
[2] LI D Q,YU Y,CHEN X L.Object tracking framework with Siamese network and re-detection mechanism[J].EURASIP Journal on Wireless Communications and Networking,2019(1):1-14.
[3] HENRIQUES J F,CASEIRO R,MARTINS P,et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(3):583-596.
[4] 徐宁,王娟娟,郭晓雨,等.判别式相关滤波器的目标跟踪综述[J].小型微型计算机系统,2020,41(12):2484-2493. XU N,WANG J J,GUO X Y,et al.Survey of visual tracking based on discriminative correlation filters[J].Journal of Chinese Computer Systems,2020,41(12):2484-2493.(in Chinese)
[5] DANELLJAN M,BHAT G,KHAN F S,et al.ECO:efficient convolution operators for tracking[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:6638-6646.
[6] DANELLJAN M,HAGER G,KHAN F S,et al.Convolutional features for correlation filter based visual tracking[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Washington D.C.,USA:IEEE Press,2015:58-66.
[7] 罗元,肖航,欧俊雄.基于深度学习的目标跟踪技术的研究综述[J].半导体光电,2020,41(6):757-767. LUO Y,XIAO H,OU J X.Research on target tracking technology based on deep learning[J].Semiconductor Optoelectronics,2020,41(6):757-767.(in Chinese)
[8] WANG N,YEUNG D Y.Learning a deep compact image representation for visual tracking[C]//Proceedings of NIPS'13.Washington D.C.,USA:IEEE Press,2013:809-817.
[9] BERTINETTO L,VALMADRE J,HENRIQUES J F,et al.Fully-convolutional Siamese networks for object tracking[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2016:850-865.
[10] LI B,YAN J,WU W,et al.High performance visual tracking with Siamese region proposal network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:8971-8980.
[11] LI B,WU W,WANG Q,et al.Evolution of Siamese visual tracking with very deep networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:16-20.
[12] ZHANG Z,PENG H.Deeper and wider Siamese networks for real-time visual tracking[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:4591-4600.
[13] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of Advances in Neural Information Processing Systems.Washington D.C.,USA:IEEE Press,2012:1097-1105.
[14] 张腾飞,周书仁,彭建.基于双孪生网络的自适应选择跟踪系统[J].计算机工程,2020,46(6):103-107. ZHANG T F,ZHOU S R,PENG J.Adaptive selective tracking system based on twofold Siamese network[J].Computer Engineering,2020,46(6):103-107.(in Chinese)
[15] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[16] RUSSAKOVSKY O,DENG J,SU H,et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[17] WANG M,LIU Y,HUANG Z.Large margin object tracking with circulant feature maps[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4021-4029.
[18] WU Y,LIM J,YANG M H.Object tracking benchmark[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1834-1848.
[19] KRISTAN M,LEONARDIS A,MATAS J,et al.The visual object tracking VOT2016 challenge results[C]//Proceedings of the 14th European Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2016:777-823.
[20] DANELLJAN M,HAGER G,KHAN F S,et al.Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2015:4310-4318.
[21] DANELLJAN M,HAGER G,KHAN F S,et al.Discriminative scale space tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,39(8):1561-1575.
[22] LI P,CHEN B,OUYANG W,et al.GradNet:gradient-guided network for visual object tracking[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2019:6162-6171.
[23] BERTINETTO L,VALMADRE J,GOLODETZ S,et al.Staple:complementary learners for real-time tracking[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1401-1409.

选择文件类型/文献管理软件名称

选择包含的内容

基于特征融合与双模板嵌套更新的孪生网络跟踪算法

Tracking Algorithm Using Siamese Network Based on Feature Fusion and Dual-Template Nested Update

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	林畅, 郭伟, 任哲聪, 金海波. 基于Transformer的目标跟踪与分割统一算法[J]. 计算机工程, 2024, 50(9): 130-141.
[2]	安玉, 葛海波, 何文昊, 马赛, 程梦洋. 基于补偿注意力机制的Siamese网络跟踪算法[J]. 计算机工程, 2024, 50(4): 187-196.
[3]	曹发鑫, 孙媛媛, 王治政, 潘丁豪, 林鸿飞. 面向借贷案件的相似案例匹配模型[J]. 计算机工程, 2024, 50(1): 306-312.
[4]	王春雷, 张建林, 李美惠, 徐智勇, 魏宇星. 结合卷积Transformer的目标跟踪算法[J]. 计算机工程, 2023, 49(4): 281-288,296.
[5]	周海赟, 项学智, 王馨遥, 任文凯. 多特征融合的端到端链式行人多目标跟踪网络[J]. 计算机工程, 2022, 48(9): 305-313.
[6]	杨帅东, 谌海云, 徐钒诚, 赵书朵, 袁杰敏. 基于孪生区域建议网络的无人机目标跟踪算法[J]. 计算机工程, 2022, 48(1): 288-295,304.
[7]	张海涛, 秦鹏程. 基于GMS与FPME的视频目标跟踪方法[J]. 计算机工程, 2021, 47(7): 226-231.
[8]	苏超群, 朱正为, 郭玉英. 基于高效卷积算子的异常抑制目标跟踪算法[J]. 计算机工程, 2021, 47(7): 266-272,288.
[9]	王涛, 刘超辉, 郑青青, 黄嘉曦. 基于单向Transformer和孪生网络的多轮任务型对话技术[J]. 计算机工程, 2021, 47(7): 55-58,66.
[10]	李志鹏, 张睿. 一种基于对抗学习的实时跟踪模型设计[J]. 计算机工程, 2021, 47(6): 262-270.
[11]	冯磊, 蒋磊, 许华, 苟泽中. 基于深度级联孪生网络的小样本调制识别算法[J]. 计算机工程, 2021, 47(4): 108-114.
[12]	尚桠朝, 孟令军. 基于多特征和尺度估计的KCF_MTSA算法[J]. 计算机工程, 2021, 47(3): 102-108,116.
[13]	罗朔, 侯进, 谭光鸿, 韩雁鹏. 一种低参数的孪生卷积网络实时目标跟踪算法[J]. 计算机工程, 2021, 47(2): 84-89.
[14]	张腾飞, 周书仁, 彭建. 基于双孪生网络的自适应选择跟踪系统[J]. 计算机工程, 2020, 46(6): 103-107.
[15]	沈泽君, 丁飞飞, 杨文元. 多粒度相关滤波视频跟踪方法[J]. 计算机工程, 2020, 46(5): 274-281.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于特征融合与双模板嵌套更新的孪生网络跟踪算法

Tracking Algorithm Using Siamese Network Based on Feature Fusion and Dual-Template Nested Update

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价