VIVAT: Virtuous Improving VAE Training through Artifact Mitigation

AlignBench has been introduced as a benchmark for evaluating fine-grained image-text alignment using synthetic image-caption pairs, addressing limitations in existing models like CLIP that rely on rule-based perturbations or short captions. This benchmark allows for a more detailed assessment of visual-language models (VLMs) by annotating each sentence for correctness.

CT-GLIP, a new 3D Grounded Language-Image Pretrained model, has been introduced to enhance the alignment of CT scans with radiology reports, addressing limitations in existing methods that rely on global embeddings. This model constructs fine-grained CT-report pairs to improve cross-modal contrastive learning, enabling better identification of organs and abnormalities in a zero-shot manner.

A new Mixture-of-Ranks (MoR) architecture has been proposed for one-step real-world image super-resolution (Real-ISR), integrating sparse Mixture-of-Experts (MoE) to enhance the adaptability of models in reconstructing high-resolution images from degraded samples. This approach utilizes a fine-grained expert partitioning strategy, treating each rank in Low-Rank Adaptation (LoRA) as an independent expert, thereby improving the model's ability to capture heterogeneous characteristics of real-world images.

A recent study explored the feasibility of zero-shot self-supervised learning for reconstructing magnetic resonance cholangiopancreatography (MRCP) images, aiming to reduce breath-hold times during scans. The research involved 11 healthy volunteers and compared zero-shot reconstruction with traditional methods, achieving significant acceleration in image acquisition times.

A new framework named Prompt-OT has been introduced to enhance the adaptation of vision-language models (VLMs) like CLIP, addressing challenges related to overfitting and zero-shot generalization during fine-tuning. This optimal transport-guided approach preserves the structural consistency of feature distributions between pre-trained and fine-tuned models, ensuring effective prompt learning.

The introduction of Lagrangian-Optimized Robust Embeddings (LORE) presents a new unsupervised adversarial fine-tuning framework aimed at enhancing the robustness of visual encoders against adversarial perturbations. This framework addresses critical limitations in existing fine-tuning strategies, particularly their instability and suboptimal trade-offs between robustness and accuracy on clean data.