Knowledge-infused deep learning algorithm for vehicle trajectory prediction

doi:10.7523/j.ucas.2024.045

Abstract

Abstract:

In the field of autonomous driving， accurate vehicle trajectory prediction plays a crucial role. While current deep learning-based algorithms have significantly improved the accuracy of vehicle trajectory prediction， they lack interpretability regarding the decision-making process of the algorithm. To address this issue， we incorporate prior knowledge into the deep learning-based algorithm and propose a trajectory prediction algorithm based on attention mechanisms. Diverging from traditional methods that add constraints for knowledge integration， we employ a tailored model architecture that embeds insights from the social force model to replicate the decision-making processes of drivers in complex traffic scenarios， thereby enhancing the interpretability of the predictions. Knowledge-infused trajectory prediction algorithm（KIT） leverages an attention mechanism to imitate drivers’ perception of their environment and uses a multilayer perceptron network for predicting accelerations influenced by the driver’s intentions， nearby traffic， and surrounding roads. The proposed method is validated on the Argoverse dataset， and the results indicate that KIT demonstrates superior predictive performance compared to current advanced trajectory prediction methods.

Key words: autonomous driving, vehicle trajectory prediction, prior knowledge, deep learning

CLC Number:

U461.91

Cui JIANG, Jianbin JIAO. Knowledge-infused deep learning algorithm for vehicle trajectory prediction[J]. Journal of University of Chinese Academy of Sciences, 2026, 43(2): 209-217.

Figures/Tables 8

Fig.1 Influences on pedestrians and vehicles

Fig.2 Framework of KIT

Fig.3 Detailed structure of various modules of KIT

Fig.4 Structure of decoder module

Fig.5 Examples of KIT predictions in Argoverse scenes

Fig.6 Accelerations of vehicles outputted by KIT in Argoverse scenes， induced by the environment， surrounding vehicles， and self-propulsion

Table 1 Results on Argoverse motion forecasting benchmark

模型	$K$ =1			$K$ =6
模型	ADE	FDE	MR/%	minADE	minFDE	MR/%
MultiPath++^[12]	1.624	3.614	56.5	0.790	1.214	13.2
GANet^[13]	1.592	3.455	55.0	0.806	1.161	11.8
HiVT^[14]	1.598	3.533	54.7	0.774	1.169	12.7
Wayformer^[15]	1.636	3.656	57.2	0.767	1.163	11.9
KIT	1.305	2.975	52.6	0.738	1.186	12.5

Table 1 Results on Argoverse motion forecasting benchmark

模型	$K$ =1			$K$ =6
模型	ADE	FDE	MR/%	minADE	minFDE	MR/%
MultiPath++^[12]	1.624	3.614	56.5	0.790	1.214	13.2
GANet^[13]	1.592	3.455	55.0	0.806	1.161	11.8
HiVT^[14]	1.598	3.533	54.7	0.774	1.169	12.7
Wayformer^[15]	1.636	3.656	57.2	0.767	1.163	11.9
KIT	1.305	2.975	52.6	0.738	1.186	12.5

Table 2 Results of the ablation experiment

设置	$K$ =6
设置	minADE	minFDE	MR/%
位置	0.836	1.327	15.5
轨迹	0.825	1.295	13.8
KIT	0.738	1.186	12.5

Table 2 Results of the ablation experiment

设置	$K$ =6
设置	minADE	minFDE	MR/%
位置	0.836	1.327	15.5
轨迹	0.825	1.295	13.8
KIT	0.738	1.186	12.5

References 24

[1]	Xie G， Shangguan A Q， Fei R， et al. Motion trajectory prediction based on a cnn-lstm sequential model［J］. Science China Information Sciences， 2020， 63（11）： 212207. DOI： 10.1007/s11432-019-2761-y .
[2]	Varadarajan B， Hefny A， Srivastava A， et al. Multipath++： efficient information fusion and trajectory aggregation for behavior prediction［C］//2022 International Conference on Robotics and Automation （ICRA）. May 23-27， 2022， Philadelphia， PA， USA. IEEE， 2022： 7814-7821. DOI： 10.1109/ICRA46639.2022.9812107 .
[3]	Nayakanti N， Al-Rfou R， Zhou A， et al. Wayformer： motion forecasting via simple & efficient attention networks［C］//2023 IEEE International Conference on Robotics and Automation （ICRA）. May 29-June 02， 2023， London， United Kingdom. IEEE， 2023： 2980-2987. DOI： 10.1109/ICRA48891.2023.10160609 .
[4]	Polychronopoulos A， Tsogas M， Amditis A J， et al. Sensor fusion for predicting vehicles' path for collision avoidance systems［J］. IEEE Transactions on Intelligent Transportation Systems， 2007， 8（3）： 549-562. DOI： 10.1109/TITS.2007.903439 .
[5]	Barth A， Franke U. Where will the oncoming vehicle be the next second？［C］//2008 IEEE Intelligent Vehicles Symposium. June 04-06， 2008， Eindhoven， Netherlands. IEEE， 2008： 1068-1073. DOI： 10.1109/IVS.2008.4621210 .
[6]	Jiang R， Wu Q S. Study on the complex dynamic properties of traffic flow from the micro and macro modelling［J］. Journal of Graduate School of Chinese Academy of Sciences， 2006， 23（6）： 848-854. DOI： 10.7523/j.issn.2095-6134.2006.6.021 .
[7]	Greer R， Deo N， Trivedi M. Trajectory prediction in autonomous driving with a lane heading auxiliary loss［J］. IEEE Robotics and Automation Letters， 2021， 6（3）： 4907-4914. DOI： 10.1109/LRA.2021.3068919 .
[8]	Li X， Rosman G， Gilitschenski I， et al. Differentiable logic layer for rule guided trajectory prediction［C］//Conference on Robot Learning. November 16-18， 2020， Cambridge， MA， USA. PMLR， 2021： 2178-2194.
[9]	Bahari M， Nejjar I， Alahi A. Injecting knowledge in data-driven vehicle trajectory predictors［J］. Transportation Research Part C： Emerging Technologies， 2021， 128： 103010. DOI： 10.1016/j.trc.2021.103010 .
[10]	Schlauch C， Wirth C， Klein N. Informed priors for knowledge integration in trajectory prediction［C］//Joint European Conference on Machine Learning and Knowledge Discovery in Databases. September 18-22， 2023， Turin， Italy. Cham： Springer， 2023： 392-407. DOI： 10.1007/978-3-031-43424-2_24 .
[11]	Helbing D， Molnár P. Social force model for pedestrian dynamics［J］. Physical Review. E， Statistical Physics， Plasmas， Fluids， and Related Interdisciplinary Topics， 1995， 51（5）： 4282-4286. DOI： 10.1103/physreve.51.4282 .
[12]	Yang D F， Özgüner Ü， Redmill K. Social force based microscopic modeling of vehicle-crowd interaction［C］//2018 IEEE Intelligent Vehicles Symposium （IV）. June 26-30， 2018， Changshu， China. IEEE， 2018： 1537-1542. DOI： 10.1109/IVS.2018.8500499 .
[13]	Rinke N， Schiermeyer C， Pascucci F， et al. A multi-layer social force approach to model interactions in shared spaces using collision prediction［J］. Transportation Research Procedia， 2017， 25： 1249-1267. DOI： 10.1016/j.trpro.2017.05.144 .
[14]	Vaswani A， Shazeer N， Parmar N， et al. Attention is all you need［C］//Advances In Neural Information Processing Systems 30： Annual Conference On Neural Information Processing Systems 2017. December 4-9， 2017， Long Beach， CA， USA. 2017： 5998-6008. DOI： 10.48550/arXiv.1706.03762 .
[15]	Wang M K， Zhu X G， Yu C Q， et al. Ganet： goal area network for motion forecasting［C］//2023 IEEE International Conference on Robotics and Automation （ICRA）. May 29-June 02， 2023， London， United Kingdom. IEEE， 2023： 1609-1615. DOI： 10.1109/ICRA48891.2023.10160468 .
[16]	Zhou Z K， Ye L Y， Wang J P， et al. Hivt： hierarchical vector transformer for multi-agent motion prediction［C］//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 18-24， 2022， New Orleans， LA， USA. IEEE， 2022： 8813-8823. DOI： 10.1109/CVPR52688.2022.00862 .
[17]	Huang Y J， Du J T， Yang Z R， et al. A survey on trajectory-prediction methods for autonomous driving［J］. IEEE Transactions on Intelligent Vehicles， 2022， 7（3）： 652-674. DOI： 10.1109/TIV.2022.3167103 .
[18]	Gupta A， Johnson J， Li F F， et al. Social GAN： socially acceptable trajectories with generative adversarial networks［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 18-23， 2018， Salt Lake City， UT， USA. IEEE， 2018： 2255-2264. DOI： 10.1109/CVPR.2018.00240 .
[19]	Gu J R， Sun C， Zhao H. Densetnt： end-to-end trajectory prediction from dense goal sets［C］//2021 IEEE/CVF International Conference on Computer Vision（ICCV）. October 10-17， 2021， Montreal， QC， Canada. IEEE， 2021： 15283-15292. DOI： 10.1109/ICCV48922.2021.01502 .
[20]	Park D， Ryu H， Yang Y， et al. Leveraging future relationship reasoning for vehicle trajectory prediction［EB/OL］. arXiv 2023：2305.14715.（2023-05-24）［2024-04-30］. .
[21]	Ye M S， Xu J M， Xu X N， et al. Bootstrap motion forecasting with self-consistent constraints［C］//2023 IEEE/CVF International Conference on Computer Vision（ICCV）. June 18-22， 2023， Paris， France. IEEE， 2023： 8470-8480. DOI： 10.1109/ICCV51070.2023.00781 .
[22]	Deo N， Wolff E M， Beijbom O. Multimodal trajectory prediction conditioned on lane-graph traversals［C］//Proceedings of the 5th Conference on Robot Learning（CoRL）. November 8-11， 2021， London， UK. PMLR， 2022： 203-212. DOI： 10.48550/arXiv.2106.15004 .
[23]	Wang Y T， Zhou H N， Zhang Z G， et al. Tenet： transformer encoding network for effective temporal flow on motion prediction［EB/OL］. arXiv 2022：2207.00170.（2022-06-30）［2024-04-30］. .
[24]	Chang M F， Lambert J， Sangkloy P， et al. Argoverse： 3d tracking and forecasting with rich maps［C］//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 27-30， 2019， Long Beach， CA， USA. IEEE， 2019： 8740-8749. DOI：10.1109/CVPR.2019.00895 .