arXiv:2606.12396v1 Announce Type: new 
Abstract: Vision-language-action (VLA) models can describe scenes and reason about them in language, yet still struggle to ground their actions in the dense 3D world around them. Existing approaches either inject features from a frozen 3D foundation model without an objective that ensures the policy uses them, or constrain geometry with sparse box and map losses that provide no dense spatial signal. We introduce VLGA, the first vision-language-action model supervised to reconstruct the dense 3D world it drives through. VLGA introduces geometry as a fourth modality alongside vision, language, and action through a dedicated expert supervised by a per-pixel pointmap regression loss against LiDAR. Extensive experiments conducted on challenging nuScenes and Bench2Drive datasets for open-loop and closed-loop evaluations, respectively, show the superiority of VLGA over counterpart VLA methods. In particular, on open-loop nuScenes, VLGA sets a new state of the art among VLA methods without ego status, with the lowest L2 (0.50\,m average) and 3-second collision rate (0.18\%). On closed-loop Bench2Drive, VLGA attains the state-of-the-art driving score of 79.08, +0.71 over the strongest prior VLA, at comparable efficiency and comfort.

تم تقديم نموذج VLGA كإطار رائد للرؤية-اللغة-الإجراء مصمم لتحسين القيادة الذاتية من خلال دمج الهندسة كنوع رابع، إلى جانب الرؤية واللغة والإجراء. يتم الإشراف على هذا النموذج لإعادة بناء العالم ثلاثي الأبعاد الكثيف، مما يعالج القيود التي تواجهها الأساليب الحالية التي تكافح لتثبيت الإجراءات في البيئات المعقدة.

Se ha presentado el modelo VLGA como un marco pionero de visión-lenguaje-acción diseñado para mejorar la conducción autónoma al integrar la geometría como una cuarta modalidad, junto con la visión, el lenguaje y la acción. Este modelo está supervisado para reconstruir el denso mundo 3D, abordando las limitaciones de los enfoques existentes que luchan por anclar acciones en entornos complejos.

Le modèle VLGA a été introduit comme un cadre novateur de vision-langage-action conçu pour améliorer la conduite autonome en intégrant la géométrie comme quatrième modalité, aux côtés de la vision, du langage et de l'action. Ce modèle est supervisé pour reconstruire le monde 3D dense, répondant aux limitations des approches existantes qui peinent à ancrer les actions dans des environnements complexes.

The VLGA model has been introduced as a pioneering vision-language-action framework designed to enhance autonomous driving by integrating geometry as a fourth modality, alongside vision, language, and action. This model is supervised to reconstruct the dense 3D world, addressing the limitations of existing approaches that struggle to ground actions in complex environments.