Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving

Zhihua Hua   Junli Wang   Pengfei Li   Qihao Jin  
Kehua Sheng   Yilun Chen   Zhongxue Gan   WenChao Ding  


Fudan University   |   Didi Chuxing   |   AIR, Tsinghua University   |   CASIA

Abstract

Navigation information serves as a critical component in end-to-end autonomous driving systems, providing essential decision-making references for planner. However, our experimental results reveal that many existing end-to-end autonomous driving systems may not adequately comprehend navigation information, consequently failing to execute appropriate planning based on navigation information. To overcome this limitation, we propose a Sequential Navigation Guidance (SNG) framework, which is designed based on real-world navigation patterns. The SNG incorporates both a navigation path to constrain long-term trajectories and Turn-by-Turn (TBT) information for real-time decision logic. We also introduce an efficient and streamlined model that achieves state-of-the-art (SOTA) performance solely through the accurate modeling of navigation information, without requiring auxiliary loss functions from perception tasks.

Carla Simulation Demos

Yielding to Cut-in Vehicle

Unprotected Left Turn

Method

Our method consists of two parts: Sequential navigation guidance and model. Sequential navigation guidance consists of navigation path and TBT information. The full pipeline of our model is divided into two phases: the multimodal feature fusion encoder and the transformer backbone of LLM.

Effectiveness of Navigation Information

We demonstrate the impact of introducing noise to the driving command on the predicted trajectory during the inference process of the Transfuser. The scenario involved an open intersection without traffic lights.

Teaser Image
Command Ablation Study

More Experiments Results

TODO

BibTeX

TODO