MAFM^3: Modular Adaptation of Foundation Models for Multi-Modal Medical AI

Foundation models (FMs) are revolutionizing medical image analysis by leveraging large datasets of unlabeled data. Unlike traditional methods that depend on manually annotated examples, FMs are pre-trained to extract general visual features, which can be fine-tuned for specific clinical tasks with minimal supervision. This review explores the development and application of FMs in pathology, radiology, and ophthalmology, synthesizing insights from over 150 studies. It highlights the components of FM pipelines and discusses challenges and future research directions.

The X-VMamba model introduces a controllability-based interpretability framework for State Space Models (SSMs), particularly the Mamba architecture. This framework aims to clarify how Vision SSMs process spatial information, which has been a challenge due to the absence of transparent mechanisms. The proposed methods include a Jacobian-based approach for any SSM architecture and a Gramian-based method for diagonal SSMs, both designed to enhance understanding of internal state dynamics while maintaining computational efficiency.

The paper titled "\Phi eat: Physically-Grounded Feature Representation" introduces a new visual backbone designed to enhance self-supervised learning in vision tasks. Current self-supervised features often mix high-level semantics with low-level physical factors, which can limit their effectiveness in tasks requiring physical reasoning. The proposed \Phi eat model focuses on material identity and employs a pretraining strategy that contrasts spatial crops and physical augmentations of materials under various conditions, aiming to improve feature robustness.

Diabetic Retinopathy (DR) is a major cause of preventable blindness, making early detection essential for reducing global vision loss. Recent advancements in deep learning have significantly improved DR screening, evolving from basic convolutional neural networks to sophisticated methodologies that tackle issues like class imbalance and label scarcity. This survey synthesizes findings from over 50 studies and 20 datasets, highlighting methodological innovations and ongoing challenges in validation and reproducibility.