arXiv:2511.15396v1 Announce Type: new 
Abstract: Recent progress in self- and weakly supervised occupancy estimation has largely relied on 2D projection or rendering-based supervision, which suffers from geometric inconsistencies and severe depth bleeding. We thus introduce ShelfOcc, a vision-only method that overcomes these limitations without relying on LiDAR. ShelfOcc brings supervision into native 3D space by generating metrically consistent semantic voxel labels from video, enabling true 3D supervision without any additional sensors or manual 3D annotations. While recent vision-based 3D geometry foundation models provide a promising source of prior knowledge, they do not work out of the box as a prediction due to sparse or noisy and inconsistent geometry, especially in dynamic driving scenes. Our method introduces a dedicated framework that mitigates these issues by filtering and accumulating static geometry consistently across frames, handling dynamic content and propagating semantic information into a stable voxel representation. This data-centric shift in supervision for weakly/shelf-supervised occupancy estimation allows the use of essentially any SOTA occupancy model architecture without relying on LiDAR data. We argue that such high-quality supervision is essential for robust occupancy learning and constitutes an important complementary avenue to architectural innovation. On the Occ3D-nuScenes benchmark, ShelfOcc substantially outperforms all previous weakly/shelf-supervised methods (up to a 34% relative improvement), establishing a new data-driven direction for LiDAR-free 3D scene understanding.

ShelfOcc هي طريقة جديدة لتقدير الإشغال تعتمد فقط على الرؤية، مما يلغي الحاجة إلى تقنية LiDAR. من خلال توليد تسميات فوكسيلي سيميائية متسقة مقياسياً من الفيديو، توفر إشرافًا ثلاثي الأبعاد حقيقيًا دون الحاجة إلى أجهزة استشعار إضافية أو تعليقات ثلاثية الأبعاد يدوية. تتناول هذه الطريقة مشكلات عدم التناسق الهندسي وتسرب العمق التي توجد عادةً في طرق الإسقاط ثنائية الأبعاد، خاصة في البيئات الديناميكية أثناء القيادة.

ShelfOcc es un nuevo método de estimación de ocupación que se basa únicamente en la visión, eliminando la necesidad de tecnología LiDAR. Al generar etiquetas voxel semánticas métricamente consistentes a partir de video, proporciona supervisión 3D real sin sensores adicionales ni anotaciones manuales. Este enfoque aborda problemas de inconsistencias geométricas y desbordamiento de profundidad que se encuentran comúnmente en métodos de proyección 2D, especialmente en entornos de conducción dinámica.

ShelfOcc est une nouvelle méthode d'estimation d'occupation basée uniquement sur la vision, éliminant le besoin de technologie LiDAR. En générant des étiquettes voxel sémantiques métriquement cohérentes à partir de vidéos, elle fournit une supervision 3D réelle sans capteurs supplémentaires ni annotations manuelles. Cette approche résout les problèmes d'incohérences géométriques et de profondeur dans les méthodes de projection 2D, en particulier dans des environnements de conduite dynamiques.

ShelfOcc is a new vision-only method for occupancy estimation that eliminates the need for LiDAR technology. By generating metrically consistent semantic voxel labels from video, it provides true 3D supervision without additional sensors or manual annotations. This approach addresses issues of geometric inconsistencies and depth bleeding commonly found in 2D projection methods, particularly in dynamic driving environments.

ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation

arXiv:2505.20951v4 Announce Type: replace 
Abstract: Camera-based 3D semantic occupancy prediction offers an efficient and cost-effective solution for perceiving surrounding scenes in autonomous driving. However, existing works rely on explicit occupancy state inference, leading to numerous incorrect feature assignments, and insufficient samples restrict the learning of occupancy class inference. To address these challenges, we propose leveraging \textbf{D}epth awareness and \textbf{S}emantic aid to boost camera-based 3D semantic \textbf{Occ}upancy prediction (\textbf{DSOcc}). We jointly perform occupancy state and occupancy class inference, where soft occupancy confidence is calculated by non-learning method and multiplied with image features to make voxels aware of depth, enabling adaptive implicit occupancy state inference. Instead of enhancing feature learning, we directly utilize well-trained image semantic segmentation and fuse multiple frames with their occupancy probabilities to aid occupancy class inference, thereby enhancing robustness. Experimental results demonstrate that DSOcc achieves state-of-the-art performance on the SemanticKITTI dataset among camera-based methods and achieves competitive performance on the SSCBench-KITTI-360 and Occ3D-nuScenes datasets. Code will be released on github.

تم تقديم DSOcc كنهج جديد لتحسين توقع الإشغال السيميائي ثلاثي الأبعاد القائم على الكاميرا من خلال دمج الوعي بالعمق والمساعدة السيميائية، مما يعالج التحديات في استنتاج حالة الإشغال وتعلم الفئات. تهدف هذه الطريقة إلى تحسين دقة إدراك المشاهد في تطبيقات القيادة الذاتية من خلال استخدام ثقة الإشغال الناعمة ودمج إطارات متعددة مع احتمالات الإشغال.

DSOcc se ha presentado como un enfoque novedoso para mejorar la predicción de ocupación semántica 3D basada en cámara, integrando la conciencia de profundidad y la ayuda semántica, abordando los desafíos en la inferencia del estado de ocupación y el aprendizaje de clases. Este método busca mejorar la precisión de la percepción de escenas en aplicaciones de conducción autónoma al utilizar la confianza de ocupación suave y fusionar múltiples cuadros con probabilidades de ocupación.

DSOcc a été introduit comme une nouvelle approche pour améliorer la prédiction d'occupation sémantique 3D basée sur la caméra en intégrant la conscience de la profondeur et l'aide sémantique, abordant les défis dans l'inférence d'état d'occupation et l'apprentissage de classe. Cette méthode vise à améliorer la précision de la perception des scènes dans les applications de conduite autonome en utilisant la confiance d'occupation douce et en fusionnant plusieurs images avec des probabilités d'occupation.

DSOcc has been introduced as a novel approach to enhance camera-based 3D semantic occupancy prediction by integrating depth awareness and semantic aid, addressing challenges in occupancy state inference and class learning. This method aims to improve the accuracy of scene perception in autonomous driving applications by utilizing soft occupancy confidence and fusing multiple frames with occupancy probabilities.

DSOcc: Leveraging Depth Awareness and Semantic Aid to Boost Camera-Based 3D Semantic Occupancy Prediction

arXiv:2511.17221v1 Announce Type: new 
Abstract: Learning 3D scene geometry and semantics from images is a core challenge in computer vision and a key capability for autonomous driving. Since large-scale 3D annotation is prohibitively expensive, recent work explores self-supervised learning directly from sensor data without manual labels. Existing approaches either rely on 2D rendering consistency, where 3D structure emerges only implicitly, or on discretized voxel grids from accumulated lidar point clouds, limiting spatial precision and scalability. We introduce QueryOcc, a query-based self-supervised framework that learns continuous 3D semantic occupancy directly through independent 4D spatio-temporal queries sampled across adjacent frames. The framework supports supervision from either pseudo-point clouds derived from vision foundation models or raw lidar data. To enable long-range supervision and reasoning under constant memory, we introduce a contractive scene representation that preserves near-field detail while smoothly compressing distant regions. QueryOcc surpasses previous camera-based methods by 26% in semantic RayIoU on the self-supervised Occ3D-nuScenes benchmark while running at 11.6 FPS, demonstrating that direct 4D query supervision enables strong self-supervised occupancy learning. https://research.zenseact.com/publications/queryocc/

تم تقديم QueryOcc كإطار عمل قائم على الاستقلال الذاتي للتعلم الذاتي الذي يتعلم الإشغال السيماني ثلاثي الأبعاد بشكل مستمر مباشرة من بيانات المستشعر، مما يعالج التحديات المتعلقة بالهندسة السيمانية والمشاهد ثلاثية الأبعاد في رؤية الكمبيوتر، خاصة لتطبيقات القيادة الذاتية.

Se ha presentado QueryOcc como un marco de auto-supervisión basado en consultas que aprende la ocupación semántica 3D continua directamente a partir de datos de sensores, abordando los desafíos de la geometría y la semántica de escenas 3D en la visión por computadora, especialmente para aplicaciones de conducción autónoma.

QueryOcc a été introduit comme un cadre d'auto-supervision basé sur des requêtes qui apprend l'occupation sémantique 3D continue directement à partir des données de capteurs, répondant aux défis de la géométrie et de la sémantique des scènes 3D dans la vision par ordinateur, en particulier pour les applications de conduite autonome.

QueryOcc has been introduced as a query-based self-supervised framework that learns continuous 3D semantic occupancy directly from sensor data, addressing the challenges of 3D scene geometry and semantics in computer vision, particularly for autonomous driving applications.

ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation

Was this article worth reading? Share it

IACrea

Attentive AI

Cloudalizr