Collaborative Learning with Multiple Foundation Models for Source-Free Domain Adaptation

A recent study investigates the resilience of neuro-inspired multi-modal vision-language models (VLMs) against membership inference attacks, which can lead to privacy leakage of sensitive training data. The research introduces a neuroscience-inspired topological regularization framework to analyze the vulnerability of these models to privacy attacks, highlighting a gap in existing literature that primarily focuses on unimodal systems.

The recent introduction of AnchorOPT marks a significant advancement in prompt learning methodologies, particularly for CLIP models. This framework enhances the adaptability of anchor tokens by allowing them to learn dynamically from task-specific data and optimizing their positional relationships with soft tokens based on the training context.

Earth-Adapter has been introduced as a novel Parameter-Efficient Fine-Tuning (PEFT) method specifically designed to address challenges in Remote Sensing (RS) scenarios, particularly the handling of artifacts that affect image features. This method employs a Mixture of Frequency Adaptation process that integrates Discrete Fourier Transformation to effectively separate artifacts from original features.

A novel framework named GAMA++ has been introduced to enhance geometry-aware domain adaptation, addressing issues of disentanglement and rigid perturbation schemes that affect performance. This method employs latent space disentanglement and an adaptive contrastive perturbation strategy tailored to class-specific needs, achieving state-of-the-art results on benchmarks like DomainNet, Office-Home, and VisDA.

A novel framework named MAADA (Manifold-Aware Adversarial Data Augmentation) has been introduced to tackle the challenges of transfer learning under domain shift, effectively decomposing adversarial perturbations into on-manifold and off-manifold components. This approach enhances model robustness and generalization by minimizing geodesic discrepancies between source and target data manifolds, as demonstrated through experiments on DomainNet, VisDA, and Office-Home.

WeatherDiffusion has been introduced as a diffusion-based framework that enables controllable weather editing in intrinsic space, utilizing an inverse renderer to estimate material properties and scene geometry from input images. This framework enhances the ability to manipulate weather conditions in generated images through an intrinsic map-aware attention mechanism and CLIP-space interpolation.

A new study introduces the Interleaved Multi-Domain Identity Curriculum (IMIC), enabling models to perform object recognition, face recognition from varying image qualities, and person recognition in a unified embedding space without significant catastrophic forgetting. This approach was tested on foundation models DINOv3, CLIP, and EVA-02, demonstrating comparable performance to domain experts across all tasks.

The stable-pretraining library has been introduced as a modular and performance-optimized tool for foundation model research, built on PyTorch, Lightning, Hugging Face, and TorchMetrics. This library aims to simplify self-supervised learning (SSL) by providing essential utilities and enhancing the visibility of training dynamics through comprehensive logging.