arXiv:2511.10241v2 Announce Type: replace 
Abstract: Spatio-Temporal Video Grounding (STVG) aims to localize a spatio-temporal tube that corresponds to a given language query in an untrimmed video. This is a challenging task since it involves complex vision-language understanding and spatiotemporal reasoning. Recent works have explored weakly-supervised setting in STVG to eliminate reliance on fine-grained annotations like bounding boxes or temporal stamps. However, they typically follow a simple late-fusion manner, which generates tubes independent of the text description, often resulting in failed target identification and inconsistent target tracking. To address this limitation, we propose a Tube-conditioned Reconstruction with Mutual Constraints (\textbf{TubeRMC}) framework that generates text-conditioned candidate tubes with pre-trained visual grounding models and further refine them via tube-conditioned reconstruction with spatio-temporal constraints. Specifically, we design three reconstruction strategies from temporal, spatial, and spatio-temporal perspectives to comprehensively capture rich tube-text correspondences. Each strategy is equipped with a Tube-conditioned Reconstructor, utilizing spatio-temporal tubes as condition to reconstruct the key clues in the query. We further introduce mutual constraints between spatial and temporal proposals to enhance their quality for reconstruction. TubeRMC outperforms existing methods on two public benchmarks VidSTG and HCSTVG. Further visualization shows that TubeRMC effectively mitigates both target identification errors and inconsistent tracking.

تقدم المقالة TubeRMC، وهو إطار مصمم لتحديد الفيديو الزمني المكاني (STVG)، والذي يهدف إلى تحديد الأنابيب الزمنية المكانية في مقاطع الفيديو غير المقطوعة بناءً على استفسارات لغوية. يعالج هذا الإطار القيود في الأساليب الحالية ذات الإشراف الضعيف التي غالبًا ما تفشل في تحديد وتتبع الأهداف بدقة بسبب نهجها البسيط في الدمج المتأخر. يستخدم TubeRMC أنابيب مرشحة مشروطة بالنص ويقوم بتحسينها من خلال قيود زمنية مكانية.

El artículo presenta TubeRMC, un marco diseñado para el Spatio-Temporal Video Grounding (STVG), que busca localizar tubos espaciotemporales en videos no editados basados en consultas lingüísticas. Este marco aborda las limitaciones de los métodos existentes de supervisión débil que a menudo fallan en identificar y rastrear objetivos con precisión debido a su enfoque simplista de fusión tardía. TubeRMC utiliza tubos candidatos condicionados por texto y los refina a través de restricciones espaciotemporales.

L'article présente TubeRMC, un cadre conçu pour le Spatio-Temporal Video Grounding (STVG), qui vise à localiser des tubes spatio-temporels dans des vidéos non découpées en fonction de requêtes linguistiques. Ce cadre répond aux limitations des méthodes faiblement supervisées existantes qui échouent souvent à identifier et suivre avec précision les cibles en raison de leur approche simpliste de fusion tardive. TubeRMC utilise des tubes candidats conditionnés par le texte et les affine grâce à des contraintes spatio-temporelles.

The article presents TubeRMC, a framework designed for Spatio-Temporal Video Grounding (STVG), which aims to localize spatio-temporal tubes in untrimmed videos based on language queries. The framework addresses limitations in existing weakly-supervised methods that often fail to accurately identify and track targets due to their simplistic late-fusion approach. TubeRMC utilizes text-conditioned candidate tubes and refines them through spatio-temporal constraints.

TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding

Was this article worth reading? Share it

LucidQuery AI

Clarity.Tube

VideoDubber Video Translator

Postugc

Dubsmart LLC

TubeMemo

Ready to build your own newsroom?