arXiv:2511.01427v1 Announce Type: new 
Abstract: Single object tracking aims to localize target object with specific reference modalities (bounding box, natural language or both) in a sequence of specific video modalities (RGB, RGB+Depth, RGB+Thermal or RGB+Event.). Different reference modalities enable various human-machine interactions, and different video modalities are demanded in complex scenarios to enhance tracking robustness. Existing trackers are designed for single or several video modalities with single or several reference modalities, which leads to separate model designs and limits practical applications. Practically, a unified tracker is needed to handle various requirements. To the best of our knowledge, there is still no tracker that can perform tracking with these above reference modalities across these video modalities simultaneously. Thus, in this paper, we present a unified tracker, UniSOT, for different combinations of three reference modalities and four video modalities with uniform parameters. Extensive experimental results on 18 visual tracking, vision-language tracking and RGB+X tracking benchmarks demonstrate that UniSOT shows superior performance against modality-specific counterparts. Notably, UniSOT outperforms previous counterparts by over 3.0\% AUC on TNL2K across all three reference modalities and outperforms Un-Track by over 2.0\% main metric across all three RGB+X video modalities.

يقدم UniSOT إطارًا مبتكرًا لتتبع كائن واحد يعزز الطريقة التي نتفاعل بها مع التكنولوجيا. من خلال استخدام مجموعة متنوعة من أوضاع الإشارة مثل الصناديق المحيطة واللغة الطبيعية، بالإضافة إلى عدة أوضاع فيديو مثل RGB والتصوير الحراري، يهدف هذا الإطار إلى تحسين دقة التتبع في البيئات المعقدة. هذه الخطوة مهمة لأنها تفتح آفاقًا جديدة للتفاعلات بين الإنسان والآلة، مما يجعل التكنولوجيا أكثر حداثة وفعالية في التطبيقات الواقعية.

UniSOT presenta un marco innovador para el seguimiento de un solo objeto que mejora la forma en que interactuamos con la tecnología. Al utilizar diversas modalidades de referencia como cuadros delimitadores y lenguaje natural, junto con múltiples modalidades de video como RGB e imágenes térmicas, este marco tiene como objetivo mejorar la precisión del seguimiento en entornos complejos. Este avance es significativo ya que abre nuevas posibilidades para las interacciones humano-máquina, haciendo que la tecnología sea más intuitiva y efectiva en aplicaciones del mundo real.

UniSOT présente un cadre innovant pour le suivi d'objets uniques qui améliore notre interaction avec la technologie. En utilisant diverses modalités de référence comme les boîtes englobantes et le langage naturel, ainsi que plusieurs modalités vidéo telles que l'imagerie RGB et thermique, ce cadre vise à améliorer la précision du suivi dans des environnements complexes. Cette avancée est significative car elle ouvre de nouvelles possibilités pour les interactions homme-machine, rendant la technologie plus intuitive et efficace dans des applications réelles.

UniSOT introduces an innovative framework for single object tracking that enhances the way we interact with technology. By utilizing various reference modalities like bounding boxes and natural language, along with multiple video modalities such as RGB and thermal imaging, this framework aims to improve tracking accuracy in complex environments. This advancement is significant as it opens up new possibilities for human-machine interactions, making technology more intuitive and effective in real-world applications.

UniSOT: A Unified Framework for Multi-Modality Single Object Tracking

Was this article worth reading? Share it

Srefs

Attentive AI

Dubsmart LLC