arXiv:2511.19765v1 Announce Type: new 
Abstract: Weakly supervised semantic segmentation (WSSS) must learn dense masks from noisy, under-specified cues. We revisit the SegFormer decoder and show that three small, synergistic changes make weak supervision markedly more effective-without altering the MiT backbone or relying on heavy post-processing. Our method, CrispFormer, augments the decoder with: (1) a boundary branch that supervises thin object contours using a lightweight edge head and a boundary-aware loss; (2) an uncertainty-guided refiner that predicts per-pixel aleatoric uncertainty and uses it to weight losses and gate a residual correction of the segmentation logits; and (3) a dynamic multi-scale fusion layer that replaces static concatenation with spatial softmax gating over multi-resolution features, optionally modulated by uncertainty. The result is a single-pass model that preserves crisp boundaries, selects appropriate scales per location, and resists label noise from weak cues. Integrated into a standard WSSS pipeline (seed, student, and EMA relabeling), CrispFormer consistently improves boundary F-score, small-object recall, and mIoU over SegFormer baselines trained on the same seeds, while adding minimal compute. Our decoder-centric formulation is simple to implement, broadly compatible with existing SegFormer variants, and offers a reproducible path to higher-fidelity masks from image-level supervision.

تم تقديم إطار خفيف الوزن من المحولات للتمييز الدلالي الضعيف الإشراف، يسمى CrispFormer، مما يعزز من وحدة فك التشفير SegFormer من خلال ثلاث تعديلات مبتكرة. تشمل هذه التعديلات فرعًا للحدود لرسم خطوط الكائنات، ومُحسِّنًا موجهًا بالشك لتوقع الشك على مستوى البكسل، وطبقة دمج متعددة المقاييس ديناميكية لتحسين تكامل الميزات.

Se ha introducido un nuevo marco ligero de transformador para la segmentación semántica débilmente supervisada, llamado CrispFormer, que mejora el decodificador SegFormer con tres modificaciones innovadoras. Estas incluyen una rama de contorno para los contornos de los objetos, un refinador guiado por la incertidumbre para la incertidumbre a nivel de píxel y una capa de fusión multiescala dinámica para una mejor integración de características.

Un nouveau cadre léger de transformateur pour la segmentation sémantique faiblement supervisée, nommé CrispFormer, a été introduit, améliorant le décodeur SegFormer avec trois modifications innovantes. Celles-ci incluent une branche de contour pour les contours d'objets, un raffineur guidé par l'incertitude pour l'incertitude au niveau des pixels, et une couche de fusion multi-échelle dynamique pour une meilleure intégration des caractéristiques.

A new lightweight transformer framework for weakly supervised semantic segmentation, named CrispFormer, has been introduced, enhancing the SegFormer decoder with three innovative modifications. These include a boundary branch for object contours, an uncertainty-guided refiner for pixel-level uncertainty, and a dynamic multi-scale fusion layer for improved feature integration.

Lightweight Transformer Framework for Weakly Supervised Semantic Segmentation

Was this article worth reading? Share it

LucidQuery AI

ClipCutAi

The Visualizer

Fakeface

Sprello

sync. labs

Ready to build your own newsroom?