arXiv:2510.26151v1 Announce Type: new 
Abstract: Large annotated datasets are essential for training robust Computer-Aided Diagnosis (CAD) models for breast cancer detection or risk prediction. However, acquiring such datasets with fine-detailed annotation is both costly and time-consuming. Vision-Language Models (VLMs), such as CLIP, which are pre-trained on large image-text pairs, offer a promising solution by enhancing robustness and data efficiency in medical imaging tasks. This paper introduces a novel Multi-View Mammography and Language Model for breast cancer classification and risk prediction, trained on a dataset of paired mammogram images and synthetic radiology reports. Our MV-MLM leverages multi-view supervision to learn rich representations from extensive radiology data by employing cross-modal self-supervision across image-text pairs. This includes multiple views and the corresponding pseudo-radiology reports. We propose a novel joint visual-textual learning strategy to enhance generalization and accuracy performance over different data types and tasks to distinguish breast tissues or cancer characteristics(calcification, mass) and utilize these patterns to understand mammography images and predict cancer risk. We evaluated our method on both private and publicly available datasets, demonstrating that the proposed model achieves state-of-the-art performance in three classification tasks: (1) malignancy classification, (2) subtype classification, and (3) image-based cancer risk prediction. Furthermore, the model exhibits strong data efficiency, outperforming existing fully supervised or VLM baselines while trained on synthetic text reports and without the need for actual radiology reports.

تقدم دراسة جديدة نموذج MV-MLM، الذي يجمع بين تصوير الثدي متعدد الزوايا ومعالجة اللغة لتحسين تشخيص سرطان الثدي وتوقع المخاطر. هذه الابتكار مهم لأنه يعالج التحدي المتمثل في الحصول على مجموعات بيانات كبيرة ومشروحة، والتي غالبًا ما تكون مكلفة وتستغرق وقتًا طويلاً. من خلال الاستفادة من نماذج الرؤية-اللغة مثل CLIP، يعزز MV-MLM الكفاءة والدقة في مهام التصوير الطبي، مما قد يؤدي إلى نتائج أفضل للمرضى وفحص أكثر فعالية لسرطان الثدي.

Un nuevo estudio presenta MV-MLM, un modelo que combina mamografía de múltiples vistas con procesamiento de lenguaje para mejorar el diagnóstico y la predicción de riesgos de cáncer de mama. Esta innovación es significativa porque aborda el desafío de adquirir grandes conjuntos de datos anotados, que a menudo son costosos y requieren mucho tiempo. Al aprovechar modelos de visión-lenguaje como CLIP, MV-MLM mejora la eficiencia y precisión de las tareas de imagen médica, lo que podría llevar a mejores resultados para los pacientes y a un cribado del cáncer más efectivo.

Une nouvelle étude présente MV-MLM, un modèle qui combine la mammographie multi-vues avec le traitement du langage pour améliorer le diagnostic et la prédiction des risques de cancer du sein. Cette innovation est importante car elle répond au défi de l'acquisition de grands ensembles de données annotées, souvent coûteux et longs à obtenir. En s'appuyant sur des modèles de vision-langage comme CLIP, MV-MLM améliore l'efficacité et la précision des tâches d'imagerie médicale, ce qui pourrait conduire à de meilleurs résultats pour les patients et à un dépistage du cancer plus efficace.

A new study introduces MV-MLM, a model that combines multi-view mammography with language processing to improve breast cancer diagnosis and risk prediction. This innovation is significant because it addresses the challenge of acquiring large, annotated datasets, which are often expensive and time-consuming. By leveraging Vision-Language Models like CLIP, MV-MLM enhances the efficiency and accuracy of medical imaging tasks, potentially leading to better patient outcomes and more effective cancer screening.

MV-MLM: Bridging Multi-View Mammography and Language for Breast Cancer Diagnosis and Risk Prediction

arXiv:2601.08336v1 Announce Type: new 
Abstract: Automatic integration of whole slide images (WSIs) and gene expression profiles has demonstrated substantial potential in precision clinical diagnosis and cancer progression studies. However, most existing studies focus on individual gene sequences and slide level classification tasks, with limited attention to spatial transcriptomics and patch level applications. To address this limitation, we propose a multimodal network, BioMorphNet, which automatically integrates tissue morphological features and spatial gene expression to support tissue classification and differential gene analysis. For considering morphological features, BioMorphNet constructs a graph to model the relationships between target patches and their neighbors, and adjusts the response strength based on morphological and molecular level similarity, to better characterize the tumor microenvironment. In terms of multimodal interactions, BioMorphNet derives clinical pathway features from spatial transcriptomic data based on a predefined pathway database, serving as a bridge between tissue morphology and gene expression. In addition, a novel learnable pathway module is designed to automatically simulate the biological pathway formation process, providing a complementary representation to existing clinical pathways. Compared with the latest morphology gene multimodal methods, BioMorphNet's average classification metrics improve by 2.67%, 5.48%, and 6.29% for prostate cancer, colorectal cancer, and breast cancer datasets, respectively. BioMorphNet not only classifies tissue categories within WSIs accurately to support tumor localization, but also analyzes differential gene expression between tissue categories based on prediction confidence, contributing to the discovery of potential tumor biomarkers.

تم اقتراح شبكة متعددة الوسائط جديدة تُدعى BioMorphNet لتحسين تصنيف الأنسجة وتحليل صور الشرائح الكاملة من خلال دمج الميزات الشكلية والتعبير الجيني المكاني، مما يعالج القيود في منهجيات البحث الحالية حول السرطان.

Se ha propuesto una nueva red multimodal llamada BioMorphNet para mejorar la clasificación de tejidos y el análisis de imágenes de diapositivas completas mediante la integración de características morfológicas y expresión génica espacial, abordando las limitaciones de las metodologías actuales en la investigación del cáncer.

Un nouveau réseau multimodal nommé BioMorphNet a été proposé pour améliorer la classification des tissus et l'analyse des images à grande échelle en intégrant des caractéristiques morphologiques et l'expression génique spatiale, répondant aux limitations des méthodologies de recherche sur le cancer actuelles.

A new multimodal network named BioMorphNet has been proposed to enhance tissue classification and whole-slide image analysis by integrating morphological features and spatial gene expression, addressing limitations in current cancer research methodologies.

Tissue Classification and Whole-Slide Images Analysis via Modeling of the Tumor Microenvironment and Biological Pathways

One More Thing in AI – Your Shortcut to AI Mastery

MV-MLM: Bridging Multi-View Mammography and Language for Breast Cancer Diagnosis and Risk Prediction

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

FastML

OpenL Translator

LangWatch

The Visualizer

Ready to build your own newsroom?