arXiv:2605.26383v1 Announce Type: new 
Abstract: Object re-identification (ReID) in egocentric kitchen videos is challenging due to rapid viewpoint changes, frequent occlusions, cluttered scenes, and large intra-class appearance variations. Objects may leave and re-enter the field of view, and the large diversity of instances with limited annotations makes supervised ReID difficult to scale, motivating zero-shot approaches. We study zero-shot object ReID on the EPIC-Kitchens benchmark, where the goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features. We first evaluate five state-of-the-art feature extractors, including Vision-Language Models (VLMs) - CLIP, DINOv2, DreamSim, I-JEPA, and SAM3 - and show that zero-shot methods fail, with the best baseline achieving only 45.3% mAP. We then propose an Enhanced SAM3 ReID Pipeline, a zero-shot multi-stage method built around SAM3 segmentation as the core component. Stage 1 uses SAM3 to suppress background clutter. Stage 2 fuses embeddings from SAM3, DINOv2, and CLIP into a single L2-normalized descriptor. Stage 3 augments cosine similarity with mask-shape IoU for geometric consistency, and Stage 4 applies k-reciprocal re-ranking. The full pipeline improves performance by 7.5% mAP to 52.8%.

قدمت دراسة جديدة طريقة لإعادة تحديد الكائنات بدون تعلم (ReID) لفيديوهات المطبخ الذاتية، حيث تتناول تحديات مثل التغيرات السريعة في الزوايا والاحتلالات. يستخدم خط أنابيب SAM3 المحسن لإعادة تحديد الكائنات تقنية تقسيم SAM3 لمطابقة حالات الطعام وأدوات المطبخ عبر الإطارات، محققًا أداءً أفضل مقارنة بالطرق الحالية.

Un nuevo estudio ha presentado un método de re-identificación de objetos sin aprendizaje (ReID) para videos de cocina egocéntricos, abordando desafíos como cambios rápidos de perspectiva y oclusiones. El Pipeline Mejorado SAM3 ReID utiliza la segmentación SAM3 para emparejar instancias de alimentos y herramientas de cocina a través de los fotogramas, logrando un mejor rendimiento en comparación con los métodos existentes.

Une nouvelle étude a introduit une méthode de ré-identification d'objet sans apprentissage (ReID) pour les vidéos de cuisine égocentriques, abordant des défis tels que les changements rapides de point de vue et les occlusions. Le pipeline Enhanced SAM3 ReID utilise la segmentation SAM3 pour faire correspondre les instances d'aliments et d'outils de cuisine à travers les images, atteignant de meilleures performances par rapport aux méthodes existantes.

A new study has introduced a zero-shot object re-identification (ReID) method for egocentric kitchen videos, addressing challenges such as rapid viewpoint changes and occlusions. The Enhanced SAM3 ReID Pipeline utilizes SAM3 segmentation to match food and kitchen-tool instances across frames, achieving improved performance over existing methods.

Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion

arXiv:2606.14389v1 Announce Type: new 
Abstract: Simultaneous 3D reconstruction and 6D object pose estimation from a single monocular image is an inherently ill-posed problem. In industrial settings, however, multiple instances of an object are often randomly arranged in bins, implicitly providing several views of the same object within a single image. We show that this implicit multi-view geometry can be exploited to simultaneously reconstruct the object in 3D and estimate the 6D pose of each visible object instance. We present MooMIns, a new Gaussian-splatting-based approach that inverts the original Gaussian splatting formulation: instead of rendering a single scene from multiple cameras, we render multiple object instances from a single camera. Our method is initialized with SAM3 instance segmentation masks and a modified Structure from Motion (SfM) pipeline. In contrast to learned monocular depth estimation, we perform true geometry-based reconstruction from image evidence, avoiding hallucinations caused by training data priors. We evaluate MooMIns on synthetic and real bin-picking scenarios, and demonstrate accurate reconstruction of previously unseen objects as well as reliable pose estimation of individual instance

تم تقديم MooMIns كنهج جديد لإعادة بناء ثلاثية الأبعاد المتزامنة وتقدير وضعية 6D من صورة أحادية العين، مستفيدًا من الهندسة متعددة المناظر الضمنية التي توفرها عدة حالات من الكائنات المرتبة في البيئات الصناعية. تستخدم هذه الطريقة تقنية Gaussian-splatting وتبدأ باستخدام أقنعة تقسيم الكائنات SAM3 جنبًا إلى جنب مع خط أنابيب معدل من الهيكل من الحركة.

MooMIns se ha presentado como un enfoque novedoso para la reconstrucción 3D simultánea y la estimación de pose 6D a partir de una única imagen monocular, aprovechando la geometría multi-vista implícita proporcionada por múltiples instancias de objetos dispuestos en entornos industriales. Este método utiliza Gaussian-splatting y se inicializa con máscaras de segmentación de instancias SAM3 junto con un pipeline modificado de Estructura a partir del Movimiento.

MooMIns a été présenté comme une nouvelle approche pour la reconstruction 3D simultanée et l'estimation de pose 6D à partir d'une seule image monoculaire, exploitant la géométrie multi-vues implicite fournie par plusieurs instances d'objets disposés dans des environnements industriels. Cette méthode utilise le Gaussian-splatting et est initialisée avec des masques de segmentation d'instance SAM3 ainsi qu'un pipeline de Structure from Motion modifié.

MooMIns has been introduced as a novel approach for simultaneous 3D reconstruction and 6D object pose estimation using a single monocular image, leveraging the implicit multi-view geometry provided by multiple instances of objects arranged in industrial settings. This method utilizes Gaussian-splatting and is initialized with SAM3 instance segmentation masks alongside a modified Structure from Motion pipeline.

MooMIns -- Monocular 3D Reconstruction and Object Pose Estimation from Multiple Instances

arXiv:2606.14299v1 Announce Type: cross 
Abstract: Vision-Language Models (VLMs) such as CLIP have become a standard backbone for open-vocabulary recognition, yet their zero-shot predictions remain vulnerable to distribution shifts encountered at deployment. Test-Time Adaptation (TTA) has recently been extended to CLIP as a lightweight solution, leading to a rapidly growing body of TTA4CLIP methods. However, empirical progress in this area has largely outpaced our understanding of what truly drives adaptation, where their gains originate, and under which shifts they remain reliable. In this paper, we take a step back from the pursuit of state-of-the-art accuracy and conduct a systematic controlled study of TTA4CLIP. We first organize existing methods into three unified paradigms according to what is updated at test time. We then introduce TTABC, an open-source TTA Benchmark for CLIP, which standardizes evaluation protocols and integrates more than 20 representative methods. Our controlled empirical analysis focuses on three key areas. First, we determine the driving factors in parameter-based methods, revealing that adaptation gains are primarily driven by test-time evidence and reliable proxies rather than heavy optimization. Second, we explore evidence utilization beyond heavy parameter tuning, showing that competitive and efficient performance can be achieved through cross- or current-sample evidence and lightweight prototype updates. Finally, we demonstrate that there is no silver bullet for TTA: no single adaptation paradigm is universally optimal, and the preferred paradigm depends on the nature of shift. We hope our benchmark and study provide a clearer understanding of the current TTA4CLIP landscape and establish a foundation for further research.

أظهرت دراسة حديثة دراسة منهجية لتكيف وقت الاختبار (TTA) لنموذج CLIP، وهو نموذج رؤية-لغة (VLM)، مما يبرز الحاجة إلى فهم أفضل لآليات التكيف وفعاليتها تحت تغيرات التوزيع المختلفة. تقدم الدراسة TTABC، وهو معيار مفتوح المصدر يهدف إلى توحيد بروتوكولات التقييم ويجمع أكثر من 20 طريقة لـ TTA4CLIP.

Un estudio reciente ha examinado de manera sistemática la Adaptación en Tiempo de Prueba (TTA) para CLIP, un Modelo de Visión-Lenguaje (VLM), destacando la necesidad de una mejor comprensión de los mecanismos de adaptación y su efectividad ante diversos cambios de distribución. El estudio introduce TTABC, un banco de pruebas de código abierto que estandariza los protocolos de evaluación e incorpora más de 20 métodos para TTA4CLIP.

Une étude récente a examiné de manière systématique l'adaptation au moment du test (TTA) pour CLIP, un modèle de vision-langage (VLM), soulignant la nécessité d'une meilleure compréhension des mécanismes d'adaptation et de leur efficacité face à divers changements de distribution. L'étude introduit TTABC, un benchmark open-source qui standardise les protocoles d'évaluation et intègre plus de 20 méthodes pour TTA4CLIP.

A recent study has systematically examined Test-Time Adaptation (TTA) for CLIP, a Vision-Language Model (VLM), highlighting the need for a better understanding of adaptation mechanisms and their effectiveness under various distribution shifts. The study introduces TTABC, an open-source benchmark that standardizes evaluation protocols and incorporates over 20 methods for TTA4CLIP.

Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion

Was this article worth reading? Share it

Ready to build your own newsroom?