arXiv:2508.00518v2 Announce Type: replace 
Abstract: Spatiotemporal video grounding aims to localize target entities in videos based on textual queries. While existing research has made significant progress in exocentric videos, the egocentric setting remains relatively underexplored, despite its growing importance in applications such as augmented reality and robotics. In this work, we conduct a systematic analysis of the discrepancies between egocentric and exocentric videos, revealing key challenges such as shorter object durations, sparser trajectories, smaller object sizes, and larger positional shifts. To address these challenges, we introduce EgoMask, the first pixel-level benchmark for fine-grained spatiotemporal grounding in egocentric videos. It is constructed by our proposed automatic annotation pipeline, which annotates referring expressions and object masks across short-, medium-, and long-term videos. Additionally, we create EgoMask-Train, a large-scale training dataset to facilitate model development. Experiments demonstrate that the state-of-the-art spatiotemporal grounding models perform poorly on our benchmark EgoMask, but fine-tuning on EgoMask-Train yields significant improvements, while preserving performance on exocentric datasets. Our work thus provides essential resources and insights for advancing egocentric video understanding. Our code is available at https://github.com/LaVi-Lab/EgoMask .

دراسة جديدة تقدم EgoMask، وهو أول معيار على مستوى البكسل للتأصيل المكاني الزمني الدقيق في مقاطع الفيديو الذاتية، حيث تتناول تحديات مثل فترات وجود الكائنات الأقصر والمسارات الأكثر ندرة. تسلط هذه الدراسة الضوء على الفجوات بين مقاطع الفيديو الذاتية والخارجية، والتي تم استكشافها بشكل أقل على الرغم من أهميتها في مجالات مثل الواقع المعزز والروبوتات.

Un nuevo estudio presenta EgoMask, el primer marco de referencia a nivel de píxel para la localización espaciotemporal detallada en videos egocéntricos, abordando desafíos como la duración más corta de los objetos y trayectorias más escasas. Esta investigación destaca las discrepancias entre videos egocéntricos y exocéntricos, que han sido menos explorados a pesar de su relevancia en campos como la realidad aumentada y la robótica.

Une nouvelle étude présente EgoMask, le premier benchmark au niveau pixel pour le ancrage spatiotemporel détaillé dans les vidéos égocentriques, abordant des défis tels que des durées d'objets plus courtes et des trajectoires plus rares. Cette recherche met en lumière les différences entre les vidéos égocentriques et exocentriques, qui ont été moins explorées malgré leur pertinence dans des domaines tels que la réalité augmentée et la robotique.

A new study introduces EgoMask, the first pixel-level benchmark for fine-grained spatiotemporal grounding in egocentric videos, addressing challenges such as shorter object durations and sparser trajectories. This research highlights the discrepancies between egocentric and exocentric videos, which have been less explored despite their relevance in fields like augmented reality and robotics.

Fine-grained Spatiotemporal Grounding on Egocentric Videos

Was this article worth reading? Share it

LucidQuery AI

Postugc

Videolulu

AiReelGenerator.com

Videotok

Com.locatelloapp

Ready to build your own newsroom?