arXiv:2406.10519v3 Announce Type: replace 
Abstract: Masked Autoencoders (MAEs) have been shown to be effective in pre-training Vision Transformers (ViTs) for natural and medical image analysis problems. By reconstructing missing pixel/voxel information in visible patches, a ViT encoder can aggregate contextual information for downstream tasks. But, existing MAE pre-training methods, which were specifically developed with the ViT architecture, lack the ability to capture geometric shape and spatial information, which is critical for medical image segmentation tasks. In this paper, we propose a novel extension of known MAEs for self pre-training (i.e., models pre-trained on the same target dataset) for 3D medical image segmentation. (1) We propose a new topological loss to preserve geometric shape information by computing topological signatures of both the input and reconstructed volumes, learning geometric shape information. (2) We introduce a pre-text task that predicts the positions of the centers and eight corners of 3D crops, enabling the MAE to aggregate spatial information. (3) We extend the MAE pre-training strategy to a hybrid state-of-the-art (SOTA) medical image segmentation architecture and co-pretrain it alongside the ViT. (4) We develop a fine-tuned model for downstream segmentation tasks by complementing the pre-trained ViT encoder with our pre-trained SOTA model. Extensive experiments on five public 3D segmentation datasets show the effectiveness of our new approach.

تقدم هذه الورقة نهجًا جديدًا للتدريب الذاتي باستخدام مشفرات تلقائية مقنعة (MAEs) تدرك الطوبولوجيا والفضاء لعملية تقسيم الصور الطبية ثلاثية الأبعاد. تعزز الطريقة المقترحة قدرة محولات الرؤية (ViTs) على التقاط المعلومات الهندسية والفضائية، والتي تعتبر حاسمة للتقسيم الدقيق. يتم تقديم خسارة طوبولوجية جديدة للحفاظ على المعلومات الهندسية، مما يحسن أداء MAEs في مهام التصوير الطبي.

Este artículo presenta un nuevo enfoque de auto-preentrenamiento utilizando autoencoders enmascarados (MAE) conscientes de la topología y la espacialidad para la segmentación de imágenes médicas en 3D. El método propuesto mejora la capacidad de los Transformers de Visión (ViTs) para capturar información geométrica y espacial, que son cruciales para una segmentación precisa. Se introduce una nueva pérdida topológica para preservar la información de forma geométrica, mejorando el rendimiento de los MAE en tareas de imagen médica.

Cet article présente une nouvelle approche de pré-entraînement autonome utilisant des autoencodeurs masqués (MAE) sensibles à la topologie et à la spatialité pour la segmentation d'images médicales 3D. La méthode proposée améliore la capacité des Transformers de Vision (ViT) à capturer des informations géométriques et spatiales, essentielles pour une segmentation précise. Une nouvelle perte topologique est introduite pour préserver les informations de forme géométrique, améliorant ainsi les performances des MAE dans les tâches d'imagerie médicale.

This paper introduces a novel approach to self pre-training using topology- and spatiality-aware Masked Autoencoders (MAEs) for 3D medical image segmentation. The proposed method enhances the ability of Vision Transformers (ViTs) to capture geometric shape and spatial information, which are crucial for accurate segmentation. A new topological loss is introduced to preserve geometric shape information, improving the performance of MAEs in medical imaging tasks.

Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentation

arXiv:2601.08499v1 Announce Type: new 
Abstract: Large models such as Vision Transformers (ViTs) have demonstrated remarkable superiority over smaller architectures like ResNet in few-shot classification, owing to their powerful representational capacity. However, fine-tuning such large models demands extensive GPU memory and prolonged training time, making them impractical for many real-world low-resource scenarios. To bridge this gap, we propose EfficientFSL, a query-only fine-tuning framework tailored specifically for few-shot classification with ViT, which achieves competitive performance while significantly reducing computational overhead. EfficientFSL fully leverages the knowledge embedded in the pre-trained model and its strong comprehension ability, achieving high classification accuracy with an extremely small number of tunable parameters. Specifically, we introduce a lightweight trainable Forward Block to synthesize task-specific queries that extract informative features from the intermediate representations of the pre-trained model in a query-only manner. We further propose a Combine Block to fuse multi-layer outputs, enhancing the depth and robustness of feature representations. Finally, a Support-Query Attention Block mitigates distribution shift by adjusting prototypes to align with the query set distribution. With minimal trainable parameters, EfficientFSL achieves state-of-the-art performance on four in-domain few-shot datasets and six cross-domain datasets, demonstrating its effectiveness in real-world applications.

يقدم EfficientFSL إطارًا للتعديل يعتمد فقط على الاستفسارات لتحسين تصنيف القليل من العينات باستخدام المحولات البصرية (ViTs)، مما يعزز الأداء مع تقليل كبير في المتطلبات الحاسوبية. تستفيد هذه الطريقة من قدرات النموذج المدرب مسبقًا، محققة دقة عالية مع عدد قليل من المعلمات القابلة للتعديل.

EfficientFSL presenta un marco de ajuste solo de consultas para Transformadores de Visión (ViTs), mejorando la clasificación de pocos disparos mientras reduce significativamente las demandas computacionales. Este enfoque aprovecha las capacidades del modelo preentrenado, logrando alta precisión con un número mínimo de parámetros.

EfficientFSL introduit un cadre de réglage uniquement basé sur les requêtes pour les Transformateurs de Vision (ViTs), améliorant la classification à faible échantillon tout en réduisant considérablement les exigences computationnelles. Cette approche exploite les capacités du modèle pré-entraîné, atteignant une haute précision avec un nombre minimal de paramètres.

EfficientFSL introduces a query-only fine-tuning framework for Vision Transformers (ViTs), enhancing few-shot classification while significantly reducing computational demands. This approach leverages the pre-trained model's capabilities, achieving high accuracy with minimal parameters.

EfficientFSL: Enhancing Few-Shot Classification via Query-Only Tuning in Vision Transformers

arXiv:2601.08602v1 Announce Type: new 
Abstract: Vision modeling has advanced rapidly with Transformers, whose attention mechanisms capture visual dependencies but lack a principled account of how semantic information propagates spatially. We revisit this problem from a wave-based perspective: feature maps are treated as spatial signals whose evolution over an internal propagation time (aligned with network depth) is governed by an underdamped wave equation. In this formulation, spatial frequency-from low-frequency global layout to high-frequency edges and textures-is modeled explicitly, and its interaction with propagation time is controlled rather than implicitly fixed. We derive a closed-form, frequency-time decoupled solution and implement it as the Wave Propagation Operator (WPO), a lightweight module that models global interactions in O(N log N) time-far lower than attention. Building on WPO, we propose a family of WaveFormer models as drop-in replacements for standard ViTs and CNNs, achieving competitive accuracy across image classification, object detection, and semantic segmentation, while delivering up to 1.6x higher throughput and 30% fewer FLOPs than attention-based alternatives. Furthermore, our results demonstrate that wave propagation introduces a complementary modeling bias to heat-based methods, effectively capturing both global coherence and high-frequency details essential for rich visual semantics. Codes are available at: https://github.com/ZishanShu/WaveFormer.

تقدم دراسة جديدة WaveFormer، وهو نهج لنمذجة الرؤية يستخدم معادلة موجية للتحكم في تطور خرائط الميزات بمرور الوقت، مما يعزز نمذجة الترددات المكانية والتفاعلات في البيانات المرئية. توفر هذه الطريقة حلاً مغلقًا يتم تنفيذه كعامل انتشار الموجات (WPO)، الذي يعمل بكفاءة أكبر من آليات الانتباه التقليدية.

Un nuevo estudio presenta WaveFormer, un enfoque de modelado visual que utiliza una ecuación de onda para gobernar la evolución de los mapas de características a lo largo del tiempo, mejorando así la modelación de frecuencias espaciales e interacciones en los datos visuales. Este método ofrece una solución en forma cerrada implementada como el Operador de Propagación de Onda (WPO), que opera de manera más eficiente que los mecanismos de atención tradicionales.

Une nouvelle étude présente WaveFormer, une approche de modélisation visuelle qui utilise une équation d'onde pour gouverner l'évolution des cartes de caractéristiques dans le temps, améliorant ainsi la modélisation des fréquences spatiales et des interactions dans les données visuelles. Cette méthode offre une solution sous forme fermée mise en œuvre comme l'Opérateur de Propagation d'Onde (WPO), qui fonctionne plus efficacement que les mécanismes d'attention traditionnels.

A new study introduces WaveFormer, a vision modeling approach that utilizes a wave equation to govern the evolution of feature maps over time, enhancing the modeling of spatial frequencies and interactions in visual data. This method offers a closed-form solution implemented as the Wave Propagation Operator (WPO), which operates more efficiently than traditional attention mechanisms.

WaveFormer: Frequency-Time Decoupled Vision Modeling with Wave Equation

One More Thing in AI – Your Shortcut to AI Mastery

Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentation

Was this article worth reading? Share it

One More Thing in AI

Artefacts.ai

ClipCutAi

Supametas.AI

Z3D

Attentive AI

Ready to build your own newsroom?