CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios

AlignBench has been introduced as a benchmark for evaluating fine-grained image-text alignment using synthetic image-caption pairs, addressing limitations in existing models like CLIP that rely on rule-based perturbations or short captions. This benchmark allows for a more detailed assessment of visual-language models (VLMs) by annotating each sentence for correctness.

A new Mixture-of-Ranks (MoR) architecture has been proposed for one-step real-world image super-resolution (Real-ISR), integrating sparse Mixture-of-Experts (MoE) to enhance the adaptability of models in reconstructing high-resolution images from degraded samples. This approach utilizes a fine-grained expert partitioning strategy, treating each rank in Low-Rank Adaptation (LoRA) as an independent expert, thereby improving the model's ability to capture heterogeneous characteristics of real-world images.

A new study has introduced a random forest-based method for out-of-distribution detection in lung cancer segmentation, utilizing a Swin Transformer model pretrained on over 10,000 3D CT scans. This approach aims to enhance the accuracy of identifying cancerous lesions in CT images, particularly in scenarios where data may not conform to expected distributions.

A new paper introduces VIVAT, a systematic approach designed to mitigate common artifacts in the training of Variational Autoencoders (VAEs), which are crucial for generative computer vision. The study identifies five prevalent artifacts and proposes modifications to improve VAE performance, achieving state-of-the-art results in image reconstruction metrics and enhancing text-to-image generation quality.

A new framework named Prompt-OT has been introduced to enhance the adaptation of vision-language models (VLMs) like CLIP, addressing challenges related to overfitting and zero-shot generalization during fine-tuning. This optimal transport-guided approach preserves the structural consistency of feature distributions between pre-trained and fine-tuned models, ensuring effective prompt learning.

The introduction of Lagrangian-Optimized Robust Embeddings (LORE) presents a new unsupervised adversarial fine-tuning framework aimed at enhancing the robustness of visual encoders against adversarial perturbations. This framework addresses critical limitations in existing fine-tuning strategies, particularly their instability and suboptimal trade-offs between robustness and accuracy on clean data.