arXiv:2511.13031v2 Announce Type: replace 
Abstract: Vision-based 3D Semantic Scene Completion (SSC) has received growing attention due to its potential in autonomous driving. While most existing approaches follow an ego-centric paradigm by aggregating and diffusing features over the entire scene, they often overlook fine-grained object-level details, leading to semantic and geometric ambiguities, especially in complex environments. To address this limitation, we propose Ocean, an object-centric prediction framework that decomposes the scene into individual object instances to enable more accurate semantic occupancy prediction. Specifically, we first employ a lightweight segmentation model, MobileSAM, to extract instance masks from the input image. Then, we introduce a 3D Semantic Group Attention module that leverages linear attention to aggregate object-centric features in 3D space. To handle segmentation errors and missing instances, we further design a Global Similarity-Guided Attention module that leverages segmentation features for global interaction. Finally, we propose an Instance-aware Local Diffusion module that improves instance features through a generative process and subsequently refines the scene representation in the BEV space. Extensive experiments on the SemanticKITTI and SSCBench-KITTI360 benchmarks demonstrate that Ocean achieves state-of-the-art performance, with mIoU scores of 17.40 and 20.28, respectively.

تناقش الورقة المعنونة 'Towards 3D Object-Centric Feature Learning for Semantic Scene Completion' التقدم في إكمال المشاهد السيمانتية ثلاثية الأبعاد المعتمدة على الرؤية، خاصة لتطبيقات القيادة الذاتية. تركز الطرق التقليدية غالبًا على نماذج ذاتية المركز، متجاهلة التفاصيل الدقيقة على مستوى الكائن، مما يؤدي إلى غموض سيميائي وهندسي، خاصة في البيئات المعقدة. لحل هذه المشكلة، نقترح Ocean، وهو إطار توقع مركزي للكائنات يقوم بتفكيك المشهد إلى كائنات فردية لتمكين توقعات أكثر دقة للاحتلال السيماني. نستخدم أولاً نموذج تقسيم خفيف الوزن، MobileSAM، لاستخراج أقنعة الكائنات من الصورة المدخلة.

El artículo titulado 'Towards 3D Object-Centric Feature Learning for Semantic Scene Completion' discute los avances en la finalización semántica de escenas 3D basada en visión, especialmente para aplicaciones de conducción autónoma. Los métodos tradicionales a menudo se centran en paradigmas egocéntricos, descuidando los detalles a nivel de objeto, lo que puede resultar en ambigüedades en entornos complejos. La solución propuesta, llamada Ocean, utiliza un modelo de segmentación ligero llamado MobileSAM para extraer máscaras de instancia e incorpora un módulo de Atención Semántica 3D para mejo…

L'article intitulé 'Towards 3D Object-Centric Feature Learning for Semantic Scene Completion' aborde les avancées dans la complétion sémantique de scènes 3D basée sur la vision, en particulier pour les applications de conduite autonome. Les méthodes traditionnelles se concentrent souvent sur des paradigmes égocentriques, négligeant les détails au niveau des objets, ce qui peut entraîner des ambiguïtés dans des environnements complexes. La solution proposée, nommée Ocean, utilise un modèle de segmentation léger appelé MobileSAM pour extraire des masques d'instance et intègre un module d'attenti…

The paper titled 'Towards 3D Object-Centric Feature Learning for Semantic Scene Completion' discusses advancements in vision-based 3D Semantic Scene Completion (SSC), particularly for autonomous driving applications. Traditional methods often focus on ego-centric paradigms, neglecting detailed object-level information, which can result in ambiguities in complex environments. The proposed solution, named Ocean, utilizes a lightweight segmentation model called MobileSAM to extract instance masks and incorporates a 3D Semantic Group Attention module to enhance object-centric feature aggregation i…

Towards 3D Object-Centric Feature Learning for Semantic Scene Completion

arXiv:2505.20951v4 Announce Type: replace 
Abstract: Camera-based 3D semantic occupancy prediction offers an efficient and cost-effective solution for perceiving surrounding scenes in autonomous driving. However, existing works rely on explicit occupancy state inference, leading to numerous incorrect feature assignments, and insufficient samples restrict the learning of occupancy class inference. To address these challenges, we propose leveraging \textbf{D}epth awareness and \textbf{S}emantic aid to boost camera-based 3D semantic \textbf{Occ}upancy prediction (\textbf{DSOcc}). We jointly perform occupancy state and occupancy class inference, where soft occupancy confidence is calculated by non-learning method and multiplied with image features to make voxels aware of depth, enabling adaptive implicit occupancy state inference. Instead of enhancing feature learning, we directly utilize well-trained image semantic segmentation and fuse multiple frames with their occupancy probabilities to aid occupancy class inference, thereby enhancing robustness. Experimental results demonstrate that DSOcc achieves state-of-the-art performance on the SemanticKITTI dataset among camera-based methods and achieves competitive performance on the SSCBench-KITTI-360 and Occ3D-nuScenes datasets. Code will be released on github.

تم تقديم DSOcc كنهج جديد لتحسين توقع الإشغال السيميائي ثلاثي الأبعاد القائم على الكاميرا من خلال دمج الوعي بالعمق والمساعدة السيميائية، مما يعالج التحديات في استنتاج حالة الإشغال وتعلم الفئات. تهدف هذه الطريقة إلى تحسين دقة إدراك المشاهد في تطبيقات القيادة الذاتية من خلال استخدام ثقة الإشغال الناعمة ودمج إطارات متعددة مع احتمالات الإشغال.

DSOcc se ha presentado como un enfoque novedoso para mejorar la predicción de ocupación semántica 3D basada en cámara, integrando la conciencia de profundidad y la ayuda semántica, abordando los desafíos en la inferencia del estado de ocupación y el aprendizaje de clases. Este método busca mejorar la precisión de la percepción de escenas en aplicaciones de conducción autónoma al utilizar la confianza de ocupación suave y fusionar múltiples cuadros con probabilidades de ocupación.

DSOcc a été introduit comme une nouvelle approche pour améliorer la prédiction d'occupation sémantique 3D basée sur la caméra en intégrant la conscience de la profondeur et l'aide sémantique, abordant les défis dans l'inférence d'état d'occupation et l'apprentissage de classe. Cette méthode vise à améliorer la précision de la perception des scènes dans les applications de conduite autonome en utilisant la confiance d'occupation douce et en fusionnant plusieurs images avec des probabilités d'occupation.

DSOcc has been introduced as a novel approach to enhance camera-based 3D semantic occupancy prediction by integrating depth awareness and semantic aid, addressing challenges in occupancy state inference and class learning. This method aims to improve the accuracy of scene perception in autonomous driving applications by utilizing soft occupancy confidence and fusing multiple frames with occupancy probabilities.

DSOcc: Leveraging Depth Awareness and Semantic Aid to Boost Camera-Based 3D Semantic Occupancy Prediction

arXiv:2510.10471v2 Announce Type: replace-cross 
Abstract: Environmental perception systems are crucial for high-precision mapping and autonomous navigation, with LiDAR serving as a core sensor providing accurate 3D point cloud data. Efficiently processing unstructured point clouds while extracting structured semantic information remains a significant challenge. In recent years, numerous pseudo-image-based representation methods have emerged to balance efficiency and performance by fusing 3D point clouds with 2D grids. However, the fundamental inconsistency between the pseudo-image representation and the original 3D information critically undermines 2D-3D feature fusion, posing a primary obstacle for coherent information fusion and leading to poor feature discriminability. This work proposes DAGLFNet, a pseudo-image-based semantic segmentation framework designed to extract discriminative features. It incorporates three key components: first, a Global-Local Feature Fusion Encoding (GL-FFE) module to enhance intra-set local feature correlation and capture global contextual information; second, a Multi-Branch Feature Extraction (MB-FE) network to capture richer neighborhood information and improve the discriminability of contour features; and third, a Feature Fusion via Deep Feature-guided Attention (FFDFA) mechanism to refine cross-channel feature fusion precision. Experimental evaluations demonstrate that DAGLFNet achieves mean Intersection-over-Union (mIoU) scores of 69.9% and 78.7% on the validation sets of SemanticKITTI and nuScenes, respectively. The method achieves an excellent balance between accuracy and efficiency.

تم تقديم DAGLFNet كإطار جديد للتجزئة الدلالية المستندة إلى الصور الزائفة، حيث يتناول التحديات المتعلقة بمعالجة سحب النقاط غير المنظمة من LiDAR بكفاءة أثناء استخراج المعلومات الدلالية المنظمة. يتضمن هذا الإطار ترميز دمج الميزات العالمية والمحلية لتحسين تمييز الميزات، وهو أمر حاسم للتطبيقات في أنظمة الإدراك البيئي.

DAGLFNet se ha presentado como un nuevo marco para la segmentación semántica basada en pseudo-imágenes, abordando los desafíos de procesar de manera eficiente nubes de puntos LiDAR no estructuradas mientras se extraen información semántica estructurada. Este marco incorpora un codificador de fusión de características globales y locales para mejorar la discriminabilidad de las características, lo cual es crucial para aplicaciones en sistemas de percepción ambiental.

DAGLFNet a été introduit comme un nouveau cadre pour la segmentation sémantique basée sur des pseudo-images, s'attaquant aux défis de traitement efficace des nuages de points LiDAR non structurés tout en extrayant des informations sémantiques structurées. Ce cadre intègre un encodage de fusion de caractéristiques globales et locales pour améliorer la discriminabilité des caractéristiques, ce qui est crucial pour les applications dans les systèmes de perception environnementale.

DAGLFNet has been introduced as a novel framework for pseudo-image-based semantic segmentation, addressing the challenges of efficiently processing unstructured LiDAR point clouds while extracting structured semantic information. This framework incorporates a Global-Local Feature Fusion Encoding to enhance feature discriminability, which is crucial for applications in environmental perception systems.

Towards 3D Object-Centric Feature Learning for Semantic Scene Completion

Was this article worth reading? Share it

Semantic Pen

Octofy

OverScene