arXiv:2506.15976v2 Announce Type: replace 
Abstract: Mamba, a State Space Model (SSM) that accelerates training by recasting recurrence as a parallel scan, has recently emerged as a linearly-scaling alternative to self-attention. Because of its unidirectional nature, each state in Mamba only has information of its previous states and is blind to states after. Current Mamba-based computer-vision methods typically overcome this by augmenting Mamba's global forward scan with a global backward scan, forming a bi-directional scan to restore a full receptive field. However, this operation doubles the computational load, eroding much of the efficiency advantage that originally Mamba have. To eliminate this extra scans, we introduce LBMamba, a locally bi-directional SSM block that embeds a lightweight locally backward scan inside the forward scan and executes it in per-thread registers. Building on LBMamba, we present LBVim, a backbone that alternates scan directions every two layers to recover a global receptive field without extra backward sweeps. We validate our approach on both natural images and whole slide images (WSIs) and show that it constantly offers a superior performance-throughput trade-off. Under the same throughput, LBVim achieves 0.8% to 1.6% higher top-1 accuracy on the ImageNet-1K classification dataset, 0.6% to 2.7% higher mIoU on the ADE20K semantic segmentation dataset, 0.9% higher APb and 1.1% higher APm on the COCO detection dataset. Our method also boosts the accuracy of four SOTA Mamba models, namely VMamba, LocalVim, PlainMamba and Adventurer, by 0.5% to 3.4%. We integrate LBMamba into the SOTA pathology multiple instance learning (MIL) model, MambaMIL, which is unidirectional. Experiments on 3 public WSI classification datasets show that our method achieves a relative improvement of up to 3.06% better AUC, 3.39% better F1, 1.67% better accuracy. Our code is available at https://github.com/cvlab-stonybrook/LBMamba.

تقديم LBMamba، نموذج الحالة الفضائية (SSM) ثنائي الاتجاه محليًا، يمثل تقدمًا كبيرًا في كفاءة التدريب لمهام رؤية الكمبيوتر. من خلال دمج مسح خلفي خفيف الوزن داخل المسح الأمامي، يلغي LBMamba الحاجة إلى عمليات مسح إضافية، مما يحافظ على الكفاءة الحاسوبية مع تحسين الأداء. يتم التحقق من هذه الابتكار من خلال تحسينات في مقاييس الدقة على مجموعات بيانات مثل ImageNet-1K وADE20K، مما يظهر تأثيره المحتمل على تطبيقات الذكاء الاصطناعي.

La introducción de LBMamba, un modelo de espacio de estados (SSM) bidireccional local, representa un avance significativo en la eficiencia del entrenamiento para tareas de visión por computadora. Al integrar un escaneo hacia atrás ligero dentro del escaneo hacia adelante, LBMamba elimina la necesidad de escaneos adicionales, manteniendo así la eficiencia computacional mientras mejora el rendimiento. Esta innovación se valida a través de métricas de precisión mejoradas en conjuntos de datos como ImageNet-1K y ADE20K, mostrando su impacto potencial en aplicaciones de IA.

L'introduction de LBMamba, un modèle d'état spatial (SSM) bi-directionnel local, représente une avancée significative en matière d'efficacité d'entraînement pour les tâches de vision par ordinateur. En intégrant un scan arrière léger dans le scan avant, LBMamba élimine le besoin de scans supplémentaires, maintenant ainsi l'efficacité computationnelle tout en améliorant les performances. Cette innovation est validée par des métriques de précision améliorées sur des ensembles de données tels qu'ImageNet-1K et ADE20K, montrant son impact potentiel sur les applications d'IA.

The introduction of LBMamba, a locally bi-directional State Space Model (SSM), marks a significant advancement in training efficiency for computer vision tasks. By embedding a lightweight backward scan within the forward scan, LBMamba eliminates the need for additional scans, thus maintaining computational efficiency while enhancing performance. This innovation is validated through improved accuracy metrics on datasets like ImageNet-1K and ADE20K, showcasing its potential impact on AI applications.

LBMamba: Locally Bi-directional Mamba

Was this article worth reading? Share it

Ready to build your own newsroom?