Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Driving

The nuCarla dataset has been introduced as a large-scale, nuScenes-style bird's-eye view perception dataset designed for the CARLA simulation environment. This dataset addresses the limitations of existing datasets that primarily support open-loop learning by providing a closed-loop simulation framework. nuCarla is fully compatible with the nuScenes format, allowing for the transfer of real-world perception models, and offers a scale comparable to nuScenes, enhancing the training of end-to-end autonomous driving models.

The PAVE dataset represents a significant advancement in the evaluation of autonomous vehicles (AVs), being the first end-to-end benchmark dataset collected entirely through autonomous driving in real-world conditions. It includes over 100 hours of naturalistic data from various production AV models, segmented into 32,727 key frames with synchronized camera images and high-precision GNSS/IMU data. This dataset aims to enhance the understanding of AV behavior and safety, providing crucial insights for future developments in autonomous driving technology.

The article discusses the limitations of current end-to-end autonomous driving systems, which overly depend on ego status, affecting their ability to generalize and understand scenes robustly. It introduces AdaptiveAD, a new architectural solution that employs a dual-branch structure to decouple scene perception from ego status. This approach aims to enhance the performance of autonomous driving systems by allowing for more effective scene-driven reasoning without the influence of ego status.

CAR-Scenes is a frame-level dataset designed for autonomous driving, facilitating the training and evaluation of vision-language models (VLMs) for scene-level understanding. The dataset comprises 5,192 annotated images from sources like Argoverse, Cityscapes, KITTI, and nuScenes, utilizing a comprehensive 28-key category/sub-category knowledge base. The annotations are generated through a GPT-4o-assisted pipeline with human verification, providing detailed attributes and supporting semantic retrieval and risk-aware scenario mining.