arXiv:2503.22087v2 Announce Type: replace 
Abstract: 3D occupancy prediction has become a key perception task in autonomous driving, as it enables comprehensive scene understanding. Recent methods enhance this understanding by incorporating spatiotemporal information through multi-frame fusion, but they suffer from a trade-off: dense voxel-based representations provide high accuracy at significant computational cost, whereas sparse representations improve efficiency but lose spatial detail. To mitigate this trade-off, we introduce DuOcc, which employs a dual aggregation strategy that retains dense voxel representations to preserve spatial fidelity while maintaining high efficiency. DuOcc consists of two key components: (i) Stream-based Voxel Aggregation, which recurrently accumulates voxel features over time and refines them to suppress warping-induced distortions, preserving a clear separation between occupied and free space. (ii) Query-guided Aggregation, which complements the limitations of voxel accumulation by selectively injecting instance-level query features into the voxel regions occupied by dynamic objects. Experiments on the widely used Occ3D-nuScenes and SurroundOcc datasets demonstrate that DuOcc achieves state-of-the-art performance in real-time settings, while reducing memory usage by over 40% compared to prior methods.

تم تقديم نهج جديد لتوقع الإشغال ثلاثي الأبعاد، يسمى DuOcc، والذي يستخدم استراتيجية تجميع مزدوجة لتعزيز فهم المشهد في القيادة الذاتية. تجمع هذه الطريقة بين تجميع الفوكسل القائم على التدفق وتجميع موجه بالاستعلام لتحقيق توازن بين الدقة والكفاءة الحاسوبية، مع الحفاظ على تمثيلات فوكسل كثيفة مع تقليل التشوهات في البيانات المكانية.

Se ha introducido un nuevo enfoque para la predicción de ocupación 3D, denominado DuOcc, que utiliza una estrategia de agregación dual para mejorar la comprensión de escenas en la conducción autónoma. Este método combina la agregación de voxels basada en flujo y la agregación guiada por consultas para equilibrar la compensación entre precisión y eficiencia computacional, manteniendo representaciones de voxels densos mientras minimiza las distorsiones en los datos espaciales.

Une nouvelle approche de la prédiction d'occupation 3D, nommée DuOcc, a été introduite, utilisant une stratégie d'agrégation duale pour améliorer la compréhension des scènes dans la conduite autonome. Cette méthode combine l'agrégation de voxels basée sur le flux et l'agrégation guidée par requête pour équilibrer le compromis entre précision et efficacité computationnelle, en conservant des représentations de voxels denses tout en minimisant les distorsions dans les données spatiales.

A new approach to 3D occupancy prediction, named DuOcc, has been introduced, which utilizes a dual aggregation strategy to enhance scene understanding in autonomous driving. This method combines stream-based voxel aggregation and query-guided aggregation to balance the trade-off between accuracy and computational efficiency, retaining dense voxel representations while minimizing distortions in spatial data.

Stream and Query-guided Feature Aggregation for Efficient and Effective 3D Occupancy Prediction

arXiv:2512.03370v1 Announce Type: new 
Abstract: We introduce ShelfGaussian, an open-vocabulary multi-modal Gaussian-based 3D scene understanding framework supervised by off-the-shelf vision foundation models (VFMs). Gaussian-based methods have demonstrated superior performance and computational efficiency across a wide range of scene understanding tasks. However, existing methods either model objects as closed-set semantic Gaussians supervised by annotated 3D labels, neglecting their rendering ability, or learn open-set Gaussian representations via purely 2D self-supervision, leading to degraded geometry and limited to camera-only settings. To fully exploit the potential of Gaussians, we propose a Multi-Modal Gaussian Transformer that enables Gaussians to query features from diverse sensor modalities, and a Shelf-Supervised Learning Paradigm that efficiently optimizes Gaussians with VFM features jointly at 2D image and 3D scene levels. We evaluate ShelfGaussian on various perception and planning tasks. Experiments on Occ3D-nuScenes demonstrate its state-of-the-art zero-shot semantic occupancy prediction performance. ShelfGaussian is further evaluated on an unmanned ground vehicle (UGV) to assess its in the-wild performance across diverse urban scenarios. Project website: https://lunarlab-gatech.github.io/ShelfGaussian/.

تم تقديم ShelfGaussian كإطار لفهم المشاهد ثلاثية الأبعاد يعتمد على Gaussian متعدد الوسائط وذو مفردات مفتوحة، مستفيدًا من نماذج رؤية الأساس المتاحة لتحسين الأداء والكفاءة في مجموعة متنوعة من مهام فهم المشهد. يتناول هذا الإطار قيود الأساليب الحالية من خلال تمكين Gaussian من استعلام الميزات من عدة أنواع من المستشعرات وتحسينها على مستويات 2D و3D.

ShelfGaussian se ha introducido como un marco de comprensión de escenas 3D basado en Gaussianos multimodales y de vocabulario abierto, aprovechando modelos de visión de fundación disponibles para mejorar el rendimiento y la eficiencia en diversas tareas de comprensión de escenas. Este marco aborda las limitaciones de los métodos existentes al permitir que los Gaussianos consulten características de múltiples modalidades de sensores y optimizarlos tanto a niveles 2D como 3D.

ShelfGaussian a été introduit comme un cadre de compréhension de scène 3D basé sur des Gaussiens multi-modaux et à vocabulaire ouvert, tirant parti de modèles de fondation visuels disponibles pour améliorer la performance et l'efficacité dans diverses tâches de compréhension de scène. Ce cadre répond aux limitations des méthodes existantes en permettant aux Gaussiens d'interroger des caractéristiques de multiples modalités de capteurs et en les optimisant à la fois aux niveaux 2D et 3D.

ShelfGaussian has been introduced as an open-vocabulary multi-modal Gaussian-based framework for 3D scene understanding, leveraging off-the-shelf vision foundation models to enhance performance and efficiency in various scene understanding tasks. This framework addresses limitations of existing methods by enabling Gaussians to query features from multiple sensor modalities and optimizing them at both 2D and 3D levels.

Stream and Query-guided Feature Aggregation for Efficient and Effective 3D Occupancy Prediction

Was this article worth reading? Share it

LucidQuery AI

Deptho.ai

Mapfit