arXiv:2503.22204v2 Announce Type: replace 
Abstract: Open-vocabulary querying in 3D space is crucial for enabling more intelligent perception in applications such as robotics, autonomous systems, and augmented reality. However, most existing methods rely on 2D pixel-level parsing, leading to multi-view inconsistencies and poor 3D object retrieval. Moreover, they are limited to static scenes and struggle with dynamic scenes due to the complexities of motion modeling. In this paper, we propose Segment then Splat, a 3D-aware open vocabulary segmentation approach for both static and dynamic scenes based on Gaussian Splatting. Segment then Splat reverses the long established approach of "segmentation after reconstruction" by dividing Gaussians into distinct object sets before reconstruction. Once reconstruction is complete, the scene is naturally segmented into individual objects, achieving true 3D segmentation. This design eliminates both geometric and semantic ambiguities, as well as Gaussian-object misalignment issues in dynamic scenes. It also accelerates the optimization process, as it eliminates the need for learning a separate language field. After optimization, a CLIP embedding is assigned to each object to enable open-vocabulary querying. Extensive experiments one various datasets demonstrate the effectiveness of our proposed method in both static and dynamic scenarios.

تقدم ورقة جديدة طريقة مبتكرة للاستعلام عن المفردات المفتوحة في الفضاء ثلاثي الأبعاد، مما يعالج قيود الأساليب الحالية المعتمدة على مستوى بكسل ثنائي الأبعاد. هذه الابتكار مهم لأنه يعزز الإدراك في الروبوتات والأنظمة المستقلة والواقع المعزز، مما يسمح بالتعامل بشكل أفضل مع المشاهد الديناميكية وتحسين استرجاع الكائنات ثلاثية الأبعاد. من خلال التغلب على عدم التناسق بين وجهات النظر المتعددة، يمكن أن تؤدي هذه الأبحاث إلى تقنيات أكثر ذكاءً واستجابة في تطبيقات متنوعة.

Un nuevo artículo presenta un método innovador para la consulta de vocabulario abierto en el espacio 3D, abordando las limitaciones de los enfoques existentes basados en píxeles 2D. Esta innovación es significativa ya que mejora la percepción en robótica, sistemas autónomos y realidad aumentada, permitiendo un mejor manejo de escenas dinámicas y mejorando la recuperación de objetos 3D. Al superar las inconsistencias de múltiples vistas, esta investigación podría conducir a tecnologías más inteligentes y reactivas en diversas aplicaciones.

Un nouvel article présente une méthode révolutionnaire pour l'interrogation en vocabulaire ouvert dans l'espace 3D, répondant aux limites des approches existantes basées sur les pixels 2D. Cette innovation est importante car elle améliore la perception dans la robotique, les systèmes autonomes et la réalité augmentée, permettant une meilleure gestion des scènes dynamiques et améliorant la récupération d'objets 3D. En surmontant les incohérences multi-vues, cette recherche pourrait conduire à des technologies plus intelligentes et réactives dans diverses applications.

A new paper introduces a groundbreaking method for open-vocabulary querying in 3D space, addressing the limitations of existing 2D pixel-level approaches. This innovation is significant as it enhances perception in robotics, autonomous systems, and augmented reality, allowing for better handling of dynamic scenes and improving 3D object retrieval. By overcoming multi-view inconsistencies, this research could lead to more intelligent and responsive technologies in various applications.

Segment then Splat: Unified 3D Open-Vocabulary Segmentation via Gaussian Splatting

One More Thing in AI – Your Shortcut to AI Mastery

Segment then Splat: Unified 3D Open-Vocabulary Segmentation via Gaussian Splatting

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Deptho.ai

Artefacts.ai

Open Source Surveillance

Attentive AI

Ready to build your own newsroom?