arXiv:2507.01463v4 Announce Type: replace 
Abstract: Instance segmentation of novel objects instances in RGB images, given some example images for each object, is a well known problem in computer vision. Designing a model general enough to be employed for all kinds of novel objects without (re-) training has proven to be a difficult task. To handle this, we present a new training-free framework, called: Novel Object Cyclic Threshold based Instance Segmentation (NOCTIS). NOCTIS integrates two pre-trained models: Grounded-SAM 2 for object proposals with precise bounding boxes and corresponding segmentation masks; and DINOv2 for robust class and patch embeddings, due to its zero-shot capabilities. Internally, the proposal-object matching is realized by determining an object matching score based on the similarity of the class embeddings and the average maximum similarity of the patch embeddings with a new cyclic thresholding (CT) mechanism that mitigates unstable matches caused by repetitive textures or visually similar patterns. Beyond CT, NOCTIS introduces: (i) an appearance score that is unaffected by object selection bias; (ii) the usage of the average confidence of the proposals' bounding box and mask as a scoring component; and (iii) an RGB-only pipeline that performs even better than RGB-D ones. We empirically show that NOCTIS, without further training/fine tuning, outperforms the best RGB and RGB-D methods regarding the mean AP score on the seven core datasets of the BOP 2023 challenge for the "Model-based 2D segmentation of unseen objects" task.

تقديم NOCTIS، إطار عمل جديد لتجزئة الكائنات، يعالج تحدي تحديد حالات الكائنات الجديدة في الصور RGB دون الحاجة إلى إعادة التدريب. من خلال دمج نماذج مدربة مسبقًا مثل Grounded-SAM 2 وDINOv2، يعزز NOCTIS دقة اقتراحات الكائنات وقدرات التجزئة من خلال آلية جديدة للعتبة الدورية.

La introducción de NOCTIS, un nuevo marco para la segmentación de instancias, aborda el desafío de identificar instancias de objetos novedosos en imágenes RGB sin necesidad de reentrenamiento. Al integrar modelos preentrenados como Grounded-SAM 2 y DINOv2, NOCTIS mejora la precisión de las propuestas de objetos y las capacidades de segmentación a través de un nuevo mecanismo de umbral cíclico.

L'introduction de NOCTIS, un nouveau cadre pour la segmentation d'instances, répond au défi d'identifier des instances d'objets nouveaux dans des images RGB sans nécessiter de réentraînement. En intégrant des modèles pré-entraînés tels que Grounded-SAM 2 et DINOv2, NOCTIS améliore la précision des propositions d'objets et des capacités de segmentation grâce à un nouveau mécanisme de seuil cyclique.

The introduction of NOCTIS, a novel framework for instance segmentation, addresses the challenge of identifying novel object instances in RGB images without the need for retraining. By integrating pre-trained models such as Grounded-SAM 2 and DINOv2, NOCTIS enhances object proposal accuracy and segmentation capabilities through a new cyclic thresholding mechanism.

NOCTIS: Novel Object Cyclic Threshold based Instance Segmentation

arXiv:2507.14137v3 Announce Type: replace 
Abstract: We present Franca (pronounced Fran-ka): free one; the first fully open-source (data, code, weights) vision foundation model that matches and in many cases surpasses the performance of state-of-the-art proprietary models, e.g., DINOv2, CLIP, SigLIPv2, etc. Our approach is grounded in a transparent training pipeline inspired by Web-SSL and uses publicly available data: ImageNet-21K and a subset of ReLAION-2B. Beyond model release, we tackle critical limitations in SSL clustering methods. While modern models rely on assigning image features to large codebooks via clustering algorithms like Sinkhorn-Knopp, they fail to account for the inherent ambiguity in clustering semantics. To address this, we introduce a parameter-efficient, multi-head clustering projector based on nested Matryoshka representations. This design progressively refines features into increasingly fine-grained clusters without increasing the model size, enabling both performance and memory efficiency. Additionally, we propose a novel positional disentanglement strategy that explicitly removes positional biases from dense representations, thereby improving the encoding of semantic content. This leads to consistent gains on several downstream benchmarks, demonstrating the utility of cleaner feature spaces. Our contributions establish a new standard for transparent, high-performance vision models and open a path toward more reproducible and generalizable foundation models for the broader AI community. The code and model checkpoints are available at https://github.com/valeoai/Franca.

تم تقديم فرانكا، أول نموذج مفتوح المصدر بالكامل في مجال الرؤية، والذي يظهر أداءً يعادل أو يتجاوز نماذج الملكية مثل DINOv2 و CLIP. يستخدم هذا النموذج خط أنابيب تدريب شفاف ومجموعات بيانات متاحة للجمهور، مما يعالج القيود في طرق التجميع الحالية في التعلم الذاتي من خلال نهج جديد للتجميع المتداخل على غرار ماتريوشكا.

Franca, el primer modelo de fundación visual completamente de código abierto, ha sido presentado, mostrando un rendimiento que iguala o supera a modelos propietarios como DINOv2 y CLIP. Este modelo utiliza un pipeline de entrenamiento transparente y conjuntos de datos disponibles públicamente, abordando las limitaciones de los métodos de agrupamiento en el aprendizaje auto-supervisado a través de un nuevo enfoque de agrupamiento anidado tipo Matryoshka.

Franca, le premier modèle de fondation visuelle entièrement open-source, a été présenté, affichant des performances qui égalent ou dépassent celles de modèles propriétaires comme DINOv2 et CLIP. Ce modèle utilise un pipeline d'entraînement transparent et des ensembles de données disponibles publiquement, abordant les limitations des méthodes de clustering en apprentissage auto-supervisé grâce à une nouvelle approche de clustering imbriqué de type Matryoshka.

Franca, the first fully open-source vision foundation model, has been introduced, showcasing performance that matches or exceeds proprietary models like DINOv2 and CLIP. This model utilizes a transparent training pipeline and publicly available datasets, addressing limitations in current self-supervised learning clustering methods through a novel nested Matryoshka clustering approach.

NOCTIS: Novel Object Cyclic Threshold based Instance Segmentation

Was this article worth reading? Share it

LucidQuery AI

Blunge

CreativePixel

4o Image Gen

GPTHumanizer

Attentive AI

Ready to build your own newsroom?