arXiv:2511.19765v1 Announce Type: new 
Abstract: Weakly supervised semantic segmentation (WSSS) must learn dense masks from noisy, under-specified cues. We revisit the SegFormer decoder and show that three small, synergistic changes make weak supervision markedly more effective-without altering the MiT backbone or relying on heavy post-processing. Our method, CrispFormer, augments the decoder with: (1) a boundary branch that supervises thin object contours using a lightweight edge head and a boundary-aware loss; (2) an uncertainty-guided refiner that predicts per-pixel aleatoric uncertainty and uses it to weight losses and gate a residual correction of the segmentation logits; and (3) a dynamic multi-scale fusion layer that replaces static concatenation with spatial softmax gating over multi-resolution features, optionally modulated by uncertainty. The result is a single-pass model that preserves crisp boundaries, selects appropriate scales per location, and resists label noise from weak cues. Integrated into a standard WSSS pipeline (seed, student, and EMA relabeling), CrispFormer consistently improves boundary F-score, small-object recall, and mIoU over SegFormer baselines trained on the same seeds, while adding minimal compute. Our decoder-centric formulation is simple to implement, broadly compatible with existing SegFormer variants, and offers a reproducible path to higher-fidelity masks from image-level supervision.

تم تقديم إطار خفيف الوزن من المحولات للتمييز الدلالي الضعيف الإشراف، يسمى CrispFormer، مما يعزز من وحدة فك التشفير SegFormer من خلال ثلاث تعديلات مبتكرة. تشمل هذه التعديلات فرعًا للحدود لرسم خطوط الكائنات، ومُحسِّنًا موجهًا بالشك لتوقع الشك على مستوى البكسل، وطبقة دمج متعددة المقاييس ديناميكية لتحسين تكامل الميزات.

Se ha introducido un nuevo marco ligero de transformador para la segmentación semántica débilmente supervisada, llamado CrispFormer, que mejora el decodificador SegFormer con tres modificaciones innovadoras. Estas incluyen una rama de contorno para los contornos de los objetos, un refinador guiado por la incertidumbre para la incertidumbre a nivel de píxel y una capa de fusión multiescala dinámica para una mejor integración de características.

Un nouveau cadre léger de transformateur pour la segmentation sémantique faiblement supervisée, nommé CrispFormer, a été introduit, améliorant le décodeur SegFormer avec trois modifications innovantes. Celles-ci incluent une branche de contour pour les contours d'objets, un raffineur guidé par l'incertitude pour l'incertitude au niveau des pixels, et une couche de fusion multi-échelle dynamique pour une meilleure intégration des caractéristiques.

A new lightweight transformer framework for weakly supervised semantic segmentation, named CrispFormer, has been introduced, enhancing the SegFormer decoder with three innovative modifications. These include a boundary branch for object contours, an uncertainty-guided refiner for pixel-level uncertainty, and a dynamic multi-scale fusion layer for improved feature integration.

Lightweight Transformer Framework for Weakly Supervised Semantic Segmentation

arXiv:2511.20501v1 Announce Type: cross 
Abstract: Accurate extraction and segmentation of the cerebral arteries from digital subtraction angiography (DSA) sequences is essential for developing reliable clinical management models of complex cerebrovascular diseases. Conventional loss functions often rely solely on pixel-wise overlap, overlooking the geometric and physical consistency of vascular boundaries, which can lead to fragmented or unstable vessel predictions. To overcome this limitation, we propose a novel \textit{Physics-Informed Loss} (PIL) that models the interaction between the predicted and ground-truth boundaries as an elastic process inspired by dislocation theory in materials physics. This formulation introduces a physics-based regularization term that enforces smooth contour evolution and structural consistency, allowing the network to better capture fine vascular geometry. The proposed loss is integrated into several segmentation architectures, including U-Net, U-Net++, SegFormer, and MedFormer, and evaluated on two public benchmarks: DIAS and DSCA. Experimental results demonstrate that PIL consistently outperforms conventional loss functions such as Cross-Entropy, Dice, Active Contour, and Surface losses, achieving superior sensitivity, F1 score, and boundary coherence. These findings confirm that the incorporation of physics-based boundary interactions into deep neural networks improves both the precision and robustness of vascular segmentation in dynamic angiographic imaging. The implementation of the proposed method is publicly available at https://github.com/irfantahir301/Physicsis_loss.

تم اقتراح وظيفة خسارة مستنيرة بالفيزياء (PIL) جديدة لتحسين تقسيم الشرايين الدماغية في تسلسلات تصوير الأوعية الدموية بالطرح الرقمي (DSA). تتناول هذه الطريقة الجديدة قيود وظائف الخسارة التقليدية التي غالبًا ما تتجاهل الاتساق الهندسي والفيزيائي لحدود الأوعية الدموية، مما يؤدي إلى تحسين دقة تنبؤات الأوعية.

Se ha propuesto una nueva función de pérdida informada por la física (PIL) para mejorar la segmentación de las arterias cerebrales en secuencias de angiografía por sustracción digital (DSA). Este nuevo enfoque aborda las limitaciones de las funciones de pérdida tradicionales que a menudo no tienen en cuenta la consistencia geométrica y física de los límites vasculares, lo que conduce a una mayor precisión en las predicciones de los vasos.

Une nouvelle fonction de perte informée par la physique (PIL) a été proposée pour améliorer la segmentation des artères cérébrales dans les séquences d'angiographie par soustraction numérique (DSA). Cette nouvelle approche répond aux limites des fonctions de perte traditionnelles qui ne tiennent souvent pas compte de la cohérence géométrique et physique des frontières vasculaires, conduisant à une meilleure précision dans les prédictions des vaisseaux.

A novel Physics-Informed Loss (PIL) function has been proposed to enhance the segmentation of cerebral arteries in digital subtraction angiography (DSA) sequences. This new approach addresses the limitations of traditional loss functions that often fail to account for the geometric and physical consistency of vascular boundaries, leading to improved accuracy in vessel predictions.

Lightweight Transformer Framework for Weakly Supervised Semantic Segmentation

Was this article worth reading? Share it