arXiv:2511.17929v1 Announce Type: new 
Abstract: Temporal Action Detection (TAD) aims to identify and localize actions by determining their starting and ending frames within untrimmed videos. Recent Structured State-Space Models such as Mamba have demonstrated potential in TAD due to their long-range modeling capability and linear computational complexity. On the other hand, structured state-space models often face two key challenges in TAD, namely, decay of temporal context due to recursive processing and self-element conflict during global visual context modeling, which become more severe while handling long-span action instances. Additionally, traditional methods for TAD struggle with detecting long-span action instances due to a lack of global awareness and inefficient detection heads. This paper presents MambaTAD, a new state-space TAD model that introduces long-range modeling and global feature detection capabilities for accurate temporal action detection. MambaTAD comprises two novel designs that complement each other with superior TAD performance. First, it introduces a Diagonal-Masked Bidirectional State-Space (DMBSS) module which effectively facilitates global feature fusion and temporal action detection. Second, it introduces a global feature fusion head that refines the detection progressively with multi-granularity features and global awareness. In addition, MambaTAD tackles TAD in an end-to-end one-stage manner using a new state-space temporal adapter(SSTA) which reduces network parameters and computation cost with linear complexity. Extensive experiments show that MambaTAD achieves superior TAD performance consistently across multiple public benchmarks.

تم تقديم MambaTAD، وهو نموذج جديد من نماذج الفضاءات الزمنية للكشف عن الإجراءات الزمنية (TAD)، لتحسين تحديد وتحديد مواقع الإجراءات في مقاطع الفيديو غير المقطوعة. يتناول هذا النموذج التحديات التي تواجهها نماذج الفضاءات الزمنية الهيكلية التقليدية، مثل تدهور السياق الزمني وصراع العناصر، خاصة في حالات الإجراءات طويلة المدى.

MambaTAD, un nuevo modelo de espacio de estados para la Detección de Acciones Temporales (TAD), ha sido presentado para mejorar la identificación y localización de acciones en videos no editados. Este modelo aborda los desafíos que enfrentan los modelos de espacio de estados estructurados tradicionales, como la degradación del contexto temporal y el conflicto de elementos, especialmente en instancias de acciones de largo alcance.

MambaTAD, un nouveau modèle d'état-espaces pour la Détection d'Actions Temporelles (TAD), a été introduit pour améliorer l'identification et la localisation des actions dans des vidéos non découpées. Ce modèle répond aux défis rencontrés par les modèles d'état-espaces structurés traditionnels, tels que la dégradation du contexte temporel et le conflit d'éléments, en particulier dans les instances d'actions à long terme.

MambaTAD, a new state-space model for Temporal Action Detection (TAD), has been introduced to enhance the identification and localization of actions in untrimmed videos. This model addresses challenges faced by traditional structured state-space models, such as temporal context decay and self-element conflict, particularly in long-span action instances.

MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detection

arXiv:2501.06138v3 Announce Type: replace 
Abstract: Temporal Action Detection (TAD) in untrimmed videos poses significant challenges, particularly for Activities of Daily Living (ADL) requiring models to (1) process long-duration videos, (2) capture temporal variations in actions, and (3) simultaneously detect dense overlapping actions. Existing CNN and Transformer-based approaches, struggle to jointly capture fine-grained detail and long-range structure at scale. State-space Model (SSM) based Mamba offers powerful long-range modeling, but naive application to TAD collapses fine-grained temporal structure and fails to account for the challenges inherent to TAD. To this end, we propose Multi-Scale Temporal Mamba (MS-Temba), which extends Mamba to TAD with newly introduced dilated SSMs. Each Temba block, comprising dilated SSMs coupled with our proposed additional losses, enables the learning of discriminative representations across temporal scales. A lightweight Multi-scale Mamba Fuser then unifies these multi-scale features via SSM-based aggregation, yielding precise action-boundary localization. With only 17M parameters, MS-Temba achieves state-of-the-art performance on densely labeled ADL benchmarks TSU & Charades, and further generalizes to long-form video summarization, setting new state-of-the-art results on TVSum & SumMe.

يقدم نموذج MS-Temba، وهو نموذج متعدد المقاييس للزمان، حلاً للتحديات الكبيرة في اكتشاف الأحداث الزمنية (TAD) في مقاطع الفيديو غير المقطوعة، وخاصة في الأنشطة اليومية. يعزز هذا النموذج القدرة على معالجة مقاطع الفيديو الطويلة، والتقاط التغيرات الزمنية، واكتشاف الأحداث المتداخلة بشكل فعال من خلال استخدام نماذج الفضاء الزمني (SSMs) المتوسعة.

La introducción de MS-Temba, un modelo Multi-Scale Temporal Mamba, aborda desafíos significativos en la Detección de Acciones Temporales (TAD) para videos no editados, especialmente en Actividades de la Vida Diaria (ADL). Este modelo mejora la capacidad de procesar videos de larga duración, capturar variaciones temporales y detectar acciones superpuestas de manera efectiva mediante el uso de Modelos de Espacio de Estado (SSMs) dilatados.

L'introduction de MS-Temba, un modèle Multi-Scale Temporal Mamba, répond à des défis significatifs dans la Détection d'Actions Temporelles (TAD) pour des vidéos non montées, en particulier dans les Activités de la Vie Quotidienne (ADL). Ce modèle améliore la capacité à traiter des vidéos de longue durée, à capturer des variations temporelles et à détecter efficacement des actions qui se chevauchent grâce à l'utilisation de Modèles d'Espace d'État (SSMs) dilatés.

The introduction of MS-Temba, a Multi-Scale Temporal Mamba model, addresses significant challenges in Temporal Action Detection (TAD) for untrimmed videos, particularly in Activities of Daily Living (ADL). This model enhances the ability to process long-duration videos, capture temporal variations, and detect overlapping actions effectively through the use of dilated State-space Models (SSMs).

MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detection

Was this article worth reading? Share it

LucidQuery AI

Supametas.AI

Tombot Spark

Attentive AI

MindStudio

LangWatch

Ready to build your own newsroom?