Libra-MIL: Multimodal Prototypes Stereoscopic Infused with Task-specific Language Priors for Few-shot Whole Slide Image Classification

The introduction of Multimodal Prototype-based Multi-Instance Learning (MIL) marks a significant advancement in the field of computational pathology, particularly for the classification of Whole Slide Images (WSIs). Traditional methods often face challenges due to the high computational costs associated with analyzing giga-pixel images and the limitations of existing vision-language models that typically employ unidirectional guidance. By proposing a bidirectional interaction model, this new approach enhances the synergy between visual and textual data, allowing for a more nuanced understanding of pathological entities. The use of large language models to generate task-specific descriptions is crucial, as it addresses the bias present in instance-level descriptions that lack fine-grained medical knowledge. This innovation not only improves model interpretability but also facilitates the learning of generalizable features, which is essential for effective few-shot learning in medical co…