Review of Trajectory Prediction Technology in Autonomous Driving Scenes

doi:10.19678/j.issn.1000-3428.0065627

Abstract

Abstract: Trajectory prediction is a key technology in the fields of autonomous driving and intelligent transportation. The accurate prediction of trajectories for vehicles and moving pedestrians can improve the perception of environmental changes in autonomous driving systems，thereby ensuring overall safety.The data-driven trajectory prediction method accurately captures interaction characteristics between agents，analyzes the historical motion and static environment information of all agents within a scene，and predicts the agents' future trajectories.The mathematical models of trajectory prediction are introduced and categorized as traditional and data-driven trajectory prediction methods.The four main challenges faced by mainstream data-driven trajectory prediction methods include intelligent agent interaction modeling，motion behavior intention prediction，trajectory diversity prediction，and static environmental information fusion within a scene.Herein，starting from the use of trajectory prediction datasets，the performance evaluation indicators，model characteristics，and other aspects of typical data-driven trajectory prediction methods are analyzed and compared. On this basis，the solutions and application scenarios of the said methods to address the abovementioned challenges are summarized，and future development directions of trajectory prediction technology in autonomous driving are proposed.

Key words: trajectory prediction, deep learning, autonomous driving, data-driven, multi-agent interaction

摘要： 轨迹预测是自动驾驶和智能交通领域的关键技术，对于车辆和移动行人轨迹的准确预测可提升自动驾驶系统对周围环境变化的感知能力，保障自动驾驶系统的安全性。数据驱动轨迹预测方法可捕捉智能体之间的交互特征，对场景内智能体历史运动和静态环境信息进行分析，准确预测智能体的未来轨迹。介绍轨迹预测的数学模型并将其分为传统轨迹预测方法和数据驱动轨迹预测方法2类，阐述主流数据驱动轨迹预测方法所面临的智能体交互建模、运动行为意图预测、轨迹多样性预测、场景内静态环境信息融合等4个主要挑战，从轨迹预测数据集使用、性能评价指标、模型特点等方面出发对典型数据驱动轨迹预测方法进行分析与对比，总结归纳这些典型数据驱动轨迹预测方法针对上述挑战的解决思路和应用场景，并对自动驾驶场景下轨迹预测技术的未来发展方向进行展望。

关键词: 轨迹预测, 深度学习, 自动驾驶, 数据驱动, 多智能体交互

CLC Number:

TP319

LI Xuesong, ZHANG Qieshi, SONG Chengqun, KANG Yuhang, CHENG Jun. Review of Trajectory Prediction Technology in Autonomous Driving Scenes[J]. Computer Engineering, 2023, 49(5): 1-11.

李雪松, 张锲石, 宋呈群, 康宇航, 程俊. 自动驾驶场景下的轨迹预测技术综述[J]. 计算机工程, 2023, 49(5): 1-11.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0065627

http://www.ecice06.com/EN/Y2023/V49/I5/1

Figures/Tables 6

References

[1] LEON F,GAVRILESCU M.A review of tracking and trajectory prediction methods for autonomous driving[J].Mathematics,2021,9(6):660-696.
[2] LEFÈVRE S,VASQUEZ D,LAUGIER C.A survey on motion prediction and risk assessment for intelligent vehicles[J].ROBOMECH Journal,2014,1(1):1-14.
[3] 乔少杰,韩楠,丁治明,等.多模式移动对象不确定性轨迹预测模型[J].自动化学报,2018,44(4):608-618. QIAO S J,HAN N,DING Z M,et al.A multiple-motion-pattern trajectory prediction model for uncertain moving objects[J].Acta Automatica Sinica,2018,44(4):608-618.(in Chinese)
[4] 张显炀,朱晓宇,林浩申,等.基于高斯混合-变分自编码器的轨迹预测算法[J].计算机工程,2020,46(7):50-57. ZHANG X Y,ZHU X Y,LIN H S,et al.Trajectory prediction algorithm based on Gaussian mixture-variational autoencoder[J].Computer Engineering,2020,46(7):50-57.(in Chinese)
[5] 乔少杰,金琨,韩楠,等.一种基于高斯混合模型的轨迹预测算法[J].软件学报,2015,26(5):1048-1063. QIAO S J,JIN K,HAN N,et al.Trajectory prediction algorithm based on Gaussian mixture model[J].Journal of Software,2015,26(5):1048-1063.(in Chinese)
[6] WIEST J,HÖFFKEN M,KREßEL U,et al.Probabilistic trajectory prediction with Gaussian mixture models[C]//Proceedings of IEEE Intelligent Vehicles Symposium.Washington D.C.,USA:IEEE Press,2012:141-146.
[7] 高建,毛莺池,李志涛.基于高斯混合-时间序列模型的轨迹预测[J].计算机应用,2019,39(8):2261-2270. GAO J,MAO Y C,LI Z T.Trajectory prediction based on Gauss mixture time series model[J].Journal of Computer Applications,2019,39(8):2261-2270.(in Chinese)
[8] XU W D,PAN J,WEI J Q,et al.Motion planning under uncertainty for on-road autonomous driving[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C.,USA:IEEE Press,2014:2507-2512.
[9] XIE G T,GAO H B,QIAN L J,et al.Vehicle trajectory prediction by integrating physics- and maneuver-based approaches using interactive multiple models[J].IEEE Transactions on Industrial Electronics,2017,65(7):5999-6008.
[10] LI J X,DAI B,LI X H,et al.A real-time and predictive trajectory-generation motion planner for autonomous ground vehicles[C]//Proceedings of the 9th International Conference on Intelligent Human-Machine Systems and Cybernetics.Washington D.C.,USA:IEEE Press,2017:108-113.
[11] 宗长富,王畅,何磊,等.基于双层隐式马尔科夫模型的驾驶意图辨识[J].汽车工程,2011,33(8):701-706. ZONG C F,WANG C,HE L,et al.Driving intention recognition based on double-layer HMM[J].Automotive Engineering,2011,33(8):701-706.(in Chinese)
[12] XIE G T,GAO H B,HUANG B,et al.A driving behavior awareness model based on a dynamic Bayesian network and distributed genetic algorithm[J].International Journal of Computational Intelligence Systems,2018,11(1):469-482.
[13] HU M J,LIAO Y,WANG W J,et al.Decision tree-based maneuver prediction for driver rear-end risk-avoidance behaviors in cut-in scenarios[EB/OL].[2022-07-11].https://www.hindawi.com/journals/jat/2017/7170358/.
[14] SCHLECHTRIEMEN J,WIRTHMUELLER F,WEDEL A,et al.When will it change the lane?A probabilistic regression approach for rarely occurring events[C]//Proceedings of IEEE Intelligent Vehicles Symposium.Washington D.C.,USA:IEEE Press,2015:1373-1379.
[15] DEO N,RANGESH A,TRIVEDI M M.How would surround vehicles move?A unified framework for maneuver classification and motion prediction[J].IEEE Transactions on Intelligent Vehicles,2018,3(2):129-140.
[16] SCHLECHTRIEMEN J,WEDEL A,HILLENBRAND J,et al.A lane change detection approach using feature ranking with maximized predictive power[C]//Proceedings of IEEE Intelligent Vehicles Symposium.Washington D.C.,USA:IEEE Press,2014:108-114.
[17] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York,USA:ACM Press,2017:6000-6010.
[18] ZHANG P,OUYANG W L,ZHANG P F,et al.SR-LSTM:state refinement for LSTM towards pedestrian trajectory prediction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:12077-12086.
[19] ZHU Y L,QIAN D H,REN D C,et al.StarNet:pedestrian trajectory prediction using deep neural network in star topology[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C.,USA:IEEE Press,2019:8075-8080.
[20] HUANG Y F,BI H K,LI Z X,et al.STGAT:modeling spatial-temporal interactions for human trajectory prediction[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2019:6271-6280.
[21] MEDSKER L,JAIN L C.Recurrent neural networks:design and applications[M].[S.l.]:CRC Press,2001.
[22] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[23] ALAHI A,GOEL K,RAMANATHAN V,et al.Social LSTM:human trajectory prediction in crowded spaces[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:961-971.
[24] CHANDRA R,BHATTACHARYA U,BERA A,et al.TraPHic:trajectory prediction in dense and heterogeneous traffic using weighted interactions[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:8475-8484.
[25] LECUN Y,BENGIO Y.Convolutional networks for images,speech and time-series[M].Cambridge,USA:MIT Press,1995.
[26] QIN Y,SONG D,CHEN H,et al.A dual-stage attention-based recurrent neural network for time series prediction[EB/OL].[2022-07-11].https://arxiv.org/abs/1704.02971.
[27] DAI S Z,LI L,LI Z H.Modeling vehicle interactions via modified LSTM models for trajectory prediction[J].IEEE Access,2019,7:38287-38296.
[28] LI X,YING X W,CHUAH M C.GRIP:graph-based interaction-aware trajectory prediction[C]//Proceedings of IEEE Intelligent Transportation Systems Conference.Washington D.C.,USA:IEEE Press,2019:3960-3966.
[29] DAN X.Spatial-temporal block and LSTM network for pedestrian trajectories prediction[EB/OL].[2022-07-11].https://arxiv.org/abs/2009.10468.
[30] KIPF T N,WELLING M.Semi-supervised classification with graph convolutional networks[EB/OL].[2022-07-11].https://arxiv.org/abs/1609.02907.
[31] LI K M,EIFFERT S,SHAN M,et al.Attentional-GCNN:adaptive pedestrian trajectory prediction towards generic autonomous vehicle use cases[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C.,USA:IEEE Press,2021:14241-14247.
[32] CASAS S,LUO W,URTASUN R.IntentNet:learning to predict intention from raw sensor data[EB/OL].[2022-07-11].https://arxiv.org/abs/2101.07907.
[33] XIN L,WANG P,CHAN C Y,et al.Intention-aware long horizon trajectory prediction of surrounding vehicles using dual LSTM networks[C]//Proceedings of the 21st International Conference on Intelligent Transportation Systems.Washington D.C.,USA:IEEE Press,2018:1441-1446.
[34] CHANDRA R,GUAN T R,PANUGANTI S,et al.Forecasting trajectory and behavior of road-agents using spectral clustering in graph-LSTMs[J].IEEE Robotics and Automation Letters,2020,5(3):4882-4890.
[35] 季学武,费聪,何祥坤,等.基于LSTM网络的驾驶意图识别及车辆轨迹预测[J].中国公路学报,2019,32(6):34-42. JI X W,FEI C,HE X K,et al.Intention recognition and trajectory prediction for vehicles using LSTM network[J].China Journal of Highway and Transport,2019,32(6):34-42.(in Chinese)
[36] AMIRIAN J,HAYET J B,PETTRÉ J.Social ways:learning multi-modal distributions of pedestrian trajectories with GANs[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2019:2964-2972.
[37] FANG L J,JIANG Q H,SHI J P,et al.TPNet:trajectory proposal network for motion prediction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2020:6796-6805.
[38] NARAYANAN S,MOSLEMI R,PITTALUGA F,et al.Divide-and-conquer for lane-aware diverse trajectory prediction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2021:15794-15803.
[39] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[40] GUPTA A,JOHNSON J,LI F F,et al.Social GAN:socially acceptable trajectories with generative adversarial networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2255-2264.
[41] 孙亚圣,姜奇,胡洁,等.基于注意力机制的行人轨迹预测生成模型[J].计算机应用,2019,39(3):668-674. SUN Y S,JIANG Q,HU J,et al.Attention mechanism based pedestrian trajectory prediction generation model[J].Journal of Computer Applications,2019,39(3):668-674.(in Chinese)
[42] BRITO B,ZHU H,PAN W,et al.Social-VRNN:one-shot multi-modal trajectory prediction for interacting pedestrians[EB/OL].[2022-07-11].https://arxiv.org/abs/2010.09056.
[43] DEO N,TRIVEDI M M.Convolutional social pooling for vehicle trajectory prediction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2018:15491-15498.
[44] GILLES T,SABATINI S,TSISHKOU D,et al.HOME:heatmap output for future motion estimation[C]//Proceedings of IEEE International Intelligent Transportation Systems Conference.Washington D.C.,USA:IEEE Press,2021:500-507.
[45] GU J R,SUN C,ZHAO H.DenseTNT:end-to-end trajectory prediction from dense goal sets[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2021:15283-15292.
[46] SHAFIEE N,PADIR T,ELHAMIFAR E.Introvert:human trajectory prediction via conditional 3D attention[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2021:16810-16820.
[47] HUYNH M,ALAGHBAND G.Trajectory prediction by coupling Scene-LSTM with human movement LSTM[M].Berlin,Germany:Springer,2019.
[48] SADEGHIAN A,KOSARAJU V,SADEGHIAN A,et al.SoPhie:an attentive GAN for predicting paths compliant to social and physical constraints[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:1349-1358.
[49] LEE N,CHOI W,VERNAZA P,et al.DESIRE:distant future prediction in dynamic scenes with interacting agents[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:2165-2174.
[50] RELLA E M,ZAECH J N,LINIGER A,et al.Decoder fusion RNN:context and interaction aware decoders for trajectory prediction[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C.,USA:IEEE Press,2021:5937-5943.
[51] GAO J Y,SUN C,ZHAO H,et al.VectorNet:encoding HD maps and agent dynamics from vectorized representation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2020:11522-11530.
[52] ZHAO H,GAO J,LAN T,et al.TNT:target-driven trajectory prediction[EB/OL].[2022-07-11].https://arxiv.org/abs/2008.08294.
[53] ZHAO T Y,XU Y F,MONFORT M,et al.Multi-agent tensor fusion for contextual trajectory prediction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:12118-12126.
[54] PFEIFFER M,PAOLO G,SOMMER H,et al.A data-driven model for interaction-aware pedestrian motion prediction in object cluttered environments[C]//Proceedings of IEEE International Conference on Robotics and Automation.Washington D.C.,USA:IEEE Press,2018:5921-5928.
[55] XUE H,HUYNH D Q,REYNOLDS M.SS-LSTM:a hierarchical LSTM model for pedestrian trajectory prediction[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision.Washington D.C.,USA:IEEE Press,2018:1186-1194.
[56] CAI Y F,WANG Z H,WANG H,et al.Environment-attention network for vehicle trajectory prediction[J].IEEE Transactions on Vehicular Technology,2021,70(11):11216-11227.
[57] VELIČKOVIĆ P,CUCURULL G,CASANOVA A,et al.Graph attention networks[EB/OL].[2022-07-11].https://arxiv.org/abs/1710.10903.
[58] SHI L S,WANG L,LONG C J,et al.SGCN:sparse graph convolution network for pedestrian trajectory prediction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2021:8990-8999.
[59] Federal Highway Administration.NGSIM(Next Generation Simulation)[EB/OL].[2022-07-11].https://ops.fhwa.dot.gov/trafficanalysistools/ngsim.htm.
[60] LERNER A,CHRYSANTHOU Y,LISCHINSKI D.Crowds by example[J].Computer Graphics Forum,2007,26(3):655-664.
[61] PELLEGRINI S,ESS A,SCHINDLER K,et al.You'll never walk alone:modeling social behavior for multi-target tracking[C]//Proceedings of the 12th International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2009:261-268.
[62] GEIGER A,LENZ P,URTASUN R.Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2012:3354-3361.
[63] ROBICQUET A,SADEGHIAN A,ALAHI A,et al.Learning social etiquette:human trajectory understanding in crowded scenes[M].Berlin,Germany:Springer,2016.
[64] KRAJEWSKI R,BOCK J,KLOEKER L,et al.The HighD dataset:a drone dataset of naturalistic vehicle trajectories on German highways for validation of highly automated driving systems[C]//Proceedings of the 21st International Conference on Intelligent Transportation Systems.Washington D.C.,USA:IEEE Press,2018:2118-2125.
[65] ZHAN W,SUN L,WANG D,et al.Interaction dataset:an international,adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps[EB/OL].[2022-07-11].https://arxiv.org/abs/1910.03088.
[66] CHANG M F,LAMBERT J,SANGKLOY P,et al.Argoverse:3D tracking and forecasting with rich maps[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:8740-8749.
[67] HUANG X Y,WANG P,CHENG X J,et al.The Apolloscape open dataset for autonomous driving and its application[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(10):2702-2719.
[68] CAESAR H,BANKITI V,LANG A H,et al.NuScenes:a multimodal dataset for autonomous driving[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2020:11618-11628.
[69] ZENG W Y,LIANG M,LIAO R J,et al.LaneRCNN:distributed representations for graph-centric motion forecasting[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington D.C.,USA:IEEE Press,2021:532-539.
[70] SCARSELLI F,GORI M,TSOI A C,et al.The graph neural network model[J].IEEE Transactions on Neural Networks,2009,20(1):61-80.
[71] CHANDRA R,BHATTACHARYA U,RONCAL C,et al.RobustTP:end-to-end trajectory prediction for heterogeneous road-agents in dense traffic with noisy sensor inputs[C]//Proceedings of the 3rd ACM Computer Science in Cars Symposium.New York,USA:ACM Press,2019:1-9.

Please choose a citation manager

Content to export