多特征融合的端到端链式行人多目标跟踪网络

doi:10.19678/j.issn.1000-3428.0062296

摘要/Abstract

摘要： 目标检测、特征提取与数据关联作为多目标跟踪网络中重要的组件，独立或部分联合地发挥作用，这种组件分离的方法虽取得了良好的跟踪效果，但增加了跟踪网络的复杂性，影响了跟踪速度。为提升行人多目标跟踪速度及维持跟踪精度，提出一种端到端链式行人多目标跟踪网络。将目标检测、特征提取与数据关联集成到一个统一的框架中，将连续2帧图片组成一个节点作为输入，直接回归出节点之间相同目标的成对边界框，利用相邻节点之间公共帧的强相似性，仅使用交并比匹配进行数据关联，以提高跟踪速度。使用多特征融合的双向特征金字塔，并在金字塔网络中引用改进可变形卷积，提高模型对目标形变的适应性。为解决正负样本不平衡及梯度贡献的差异，将focal loss与BalancedL1 Loss组成多任务学习损失函数以促进网络的均衡学习。在MOT17数据集上的实验结果表明，与DeepSORT、TubeTK、CenterTrack等网络相比，该网络可有效实现跟踪速度与精度的平衡，多目标跟踪精度为69.6，跟踪速度保持为21.6 frame/s。

关键词: 多目标跟踪, 链式跟踪, 多特征融合, 特征金字塔, 多任务损失函数

Abstract: Object detection, feature extraction, and data association as important components in multi-target tracking network, work independently or partially jointly.Despite the improved tracking performance, separated components increase the tracking network complexity and decrease the tracking speed.An end-to-end chained network with multifeature fusion is proposed to increase the speed of pedestrian multi-object tracking while maintaining tracking accuracy.The network integrates object detection, feature extraction, and data association into a framework.Two adjacent frames form a node as the input.The network regresses the bounding box pairs of the same target in the node.The common frames across nodes have a strong correlation such that using Intersection over Union (IoU) matching for data association improves the tracking speed.In addition, the multi-feature fusion pyramid is adopted to fully integrate the high-level semantic information and low-level position information.The pyramid adopts deformable convolution v2, which increases adaptability to the deformation of objects.Focal loss and balanced L1 loss form multitask learning loss for promoting the balanced learning to improve the tracking performance, owing to the imbalance in the positive and negative samples and the differences in the gradient contributions.The experimental results for the MOT17 dataset show that compared with DeepSORT, TubeTK, CenterTrack, and other networks, this network can effectively achieve the trade-off between the tracking speed and accuracy.The tracking accuracy Mota value is 69.6, and the tracking speed is maintained at 21.6 frame/s.

Key words: multi-object tracking, chained-tracker, multi-feature fusion, feature pyramid, multi-task loss function

中图分类号:

TP391

周海赟, 项学智, 王馨遥, 任文凯. 多特征融合的端到端链式行人多目标跟踪网络[J]. 计算机工程, 2022, 48(9): 305-313.

ZHOU Haiyun, XIANG Xuezhi, WANG Xinyao, REN Wenkai. Chained End-to-End Pedestrian Multi-Object Tracking Network with Multi-Feature Fusion[J]. Computer Engineering, 2022, 48(9): 305-313.

http://www.ecice06.com/CN/Y2022/V48/I9/305

图/表 8

20221017111615

20221017111619

20221017111622

20221017111626

20221017111630

20221017111633

20221017111636

20221017111640

参考文献

[1] LUO W H, XING J L, MILAN A, et al.Multiple object tracking:a literature review[EB/OL].[2021-07-07].https://arxiv.org/abs/1409.7618.
[2] BERCLAZ J, FLEURET F, TURETKEN E, et al.Multiple object tracking using K-shortest paths optimization[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(9):1806-1819.
[3] PIRSIAVASH H, RAMANAN D, FOWLKES C C.Globally-optimal greedy algorithms for tracking a variable number of objects[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2011:1201-1208.
[4] KALMAN R E.A new approach to linear filtering and prediction problems[J].Journal of Basic Engineering, 1960, 82(1):35-45.
[5] HENRIQUES J F, CASEIRO R, MARTINS P, et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3):583-596.
[6] BEWLEY A, GE Z Y, OTT L, et al.Simple online and realtime tracking[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2016:3464-3468.
[7] WOJKE N, BEWLEY A, PAULUS D.Simple online and realtime tracking with a deep association metric[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C., USA:IEEE Press, 2017:3645-3649.
[8] BAE S H, YOON K J.Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3):595-610.
[9] CHEN L, AI H Z, ZHUANG Z J, et al.Real-time multiple people tracking with deeply learned candidate selection and person re-identification[C]//Proceedings of IEEE International Conference on Multimedia and Expo.Washington D.C., USA:IEEE Press, 2018:1-6.
[10] BERGMANN P, MEINHARDT T, LEAL-TAIXÉ L.Tracking without bells and whistles[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:941-951.
[11] ZHOU X Y, KOLTUN V, KRÄHENBÜHL P.Tracking objects as points[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:474-490.
[12] WANG Z D, ZHENG L, LIU Y X, et al.Towards real-time multi-object tracking[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:107-122.
[13] ZHANG Y F, WANG C Y, WANG X G, et al.FairMOT:on the fairness of detection and re-identification in multiple object tracking[EB/OL].[2021-07-07]. https://arxiv.org/abs/2004.01888.
[14] PENG J L, WANG C G, WAN F B, et al.Chained-tracker:chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2020:145-161.
[15] REN S Q, HE K M, GIRSHICK R, et al.Faster R-CNN:towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[16] LIU W, ANGUELOV D, ERHAN D, et al.SSD:single shot multibox detector[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:21-37.
[17] LIN T Y, DOLLÁR P, GIRSHICK R, et al.Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:936-944.
[18] LIU S, QI L, QIN H F, et al.Path aggregation network for instance segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:8759-8768.
[19] DAI J F, QI H Z, XIONG Y W, et al.Deformable convolutional networks[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:764-773.
[20] ZHU X Z, HU H, LIN S, et al.Deformable ConvNets V2:more deformable, better results[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:9300-9308.
[21] LIN T Y, GOYAL P, GIRSHICK R, et al.Focal loss for dense object detection[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2999-3007.
[22] PANG J M, CHEN K, SHI J P, et al.Libra R-CNN:towards balanced learning for object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:821-830.
[23] BERNARDIN K, STIEFELHAGEN R.Evaluating multiple object tracking performance:the CLEAR MOT metrics[J].Journal on Image and Video Processing, 2008, 28:1-11.
[24] RISTANI E, SOLERA F, ZOU R, et al.Performance measures and a data set for multi-target, multi-camera tracking[C]//Proceedings of the 2016 European Conference on Computer Vision.Berlin, Germany:Springer, 2016:17-35.
[25] SUN S J, AKHTAR N, SONG H S, et al.Deep affinity network for multiple object tracking[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(1):104-119.
[26] PANG B, LI Y Z, ZHANG Y F, et al.TubeTK:adopting tubes to track multi-object in a one-step training model[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:6307-6317.
[27] WANG Y X, KITANI K, WENG X S.Joint object detection and multi-object tracking with graph neural networks[EB/OL].[2021-07-07].https://arxiv.org/abs/2006.13164.
[28] PANG J M, QIU L L, LI X, et al.Quasi-dense similarity learning for multiple object tracking[EB/OL].[2021-07-07].https://arxiv.org/abs/2006.06664.
[29] XU Y H, BAN Y T, DELORME G, et al.TransCenter:transformers with dense representations for multiple-object tracking[EB/OL].[2021-07-07].https://arxiv.org/abs/2103.15145.

选择文件类型/文献管理软件名称

选择包含的内容