GMAT: Grounded Multi-Agent Clinical Description Generation for Text Encoder in Vision-Language MIL for Whole Slide Image Classification

The article presents SLAM-AGS, a new pretraining framework for computational cytology that addresses two significant challenges: the unreliability and high cost of obtaining instance-level labels, and the extremely low witness rates. SLAM-AGS optimizes a weakly supervised similarity objective on slide-negative patches alongside a self-supervised contrastive objective on slide-positive patches. This approach enhances performance on downstream tasks. Additionally, Adaptive Gradient Surgery is employed to stabilize learning and prevent model collapse, resulting in improved bag-level predictions o…

DepthVision is a multimodal framework designed to enhance Vision-Language Models (VLMs) by utilizing LiDAR data without requiring architectural modifications or retraining. It synthesizes RGB-like images from sparse LiDAR point clouds using a conditional GAN and integrates a Luminance-Aware Modality Adaptation (LAMA) module to dynamically adjust image quality based on ambient lighting. This innovation aims to improve the reliability of autonomous vehicles in challenging visual conditions, such as darkness or motion blur.

Recent advancements in autonomous driving have investigated the application of vision-language models (VLMs) in visual question answering (VQA) frameworks for driving decision-making. However, these methods often rely on handcrafted prompts and exhibit inconsistent performance, which hampers their effectiveness in real-world scenarios. This study assesses state-of-the-art open-source VLMs on high-level decision-making tasks using ego-view visual inputs, revealing significant limitations in their ability to provide reliable, context-aware decisions.

The paper introduces VLM3D, a novel framework that utilizes vision-language models (VLMs) to enhance text-to-3D generation. It addresses two major limitations in current models: the lack of fine-grained semantic alignment and inadequate 3D spatial understanding. VLM3D employs a dual-query critic signal to evaluate both semantic fidelity and geometric coherence, significantly improving the generation process. The framework demonstrates its effectiveness across different paradigms, marking a step forward in 3D generation technology.

Foundation models (FMs) are revolutionizing medical image analysis by leveraging large datasets of unlabeled data. Unlike traditional methods that depend on manually annotated examples, FMs are pre-trained to extract general visual features, which can be fine-tuned for specific clinical tasks with minimal supervision. This review explores the development and application of FMs in pathology, radiology, and ophthalmology, synthesizing insights from over 150 studies. It highlights the components of FM pipelines and discusses challenges and future research directions.

This study evaluates the reasoning capabilities of large language models (LLMs) in reconstructing ordered procedural sequences from shuffled steps, using a dataset of food recipes. The research highlights the importance of correct sequencing for task success and assesses various LLMs under zero-shot and few-shot conditions. A comprehensive evaluation framework is introduced, utilizing metrics such as Kendall's Tau and Normalized Edit Distance. Findings indicate that model performance decreases with longer sequences, revealing challenges in processing complex procedures.

The paper discusses the challenges of test-time prompt tuning for vision-language models, highlighting the issue of prompt optimization bias that can lead to suboptimal performance in downstream tasks. It identifies two main causes: the model's focus on entropy minimization, which may overlook prediction accuracy, and data misalignment between visual and textual modalities. To address these issues, the authors propose a new method called Doubly Debiased Test-Time Prompt Tuning, aimed at improving model performance in zero-shot settings.

The article discusses the introduction of InData, a dataset aimed at enhancing the security of large language model (LLM) agents used for data analysis. Traditional methods allow LLMs to generate and execute code directly on databases, which poses security risks, especially with sensitive data. InData proposes a solution by restricting LLMs from direct code generation and requiring them to use a predefined set of secure tools. The dataset includes questions of varying difficulty to assess multi-step reasoning capabilities of LLMs.