arXiv:2511.19436v1 Announce Type: cross 
Abstract: We present VDC-Agent, a self-evolving framework for Video Detailed Captioning that requires neither human annotations nor larger teacher models. The agent forms a closed loop of caption generation, principle-guided scoring (score and textual suggestions), and prompt refinement. When caption quality regresses, a self-reflection path leverages the previous chain-of-thought to amend the update. Running this process on unlabeled videos produces trajectories of (caption, score) pairs. We convert the trajectories into preference tuples and filter out samples with JSON parsing errors, resulting in VDC-Agent-19K, which contains 18,886 automatically constructed pairs. We then fine-tune the base MLLM on this dataset using an easy-to-hard curriculum direct preference optimization. Built on Qwen2.5-VL-7B-Instruct, our VDC-Agent-7B attains state-of-the-art performance on the VDC benchmark with 49.08% average accuracy and 2.50 score, surpassing specialized video captioners and improving over the base model by +5.13% accuracy and +0.27 score at similar inference cost.

تم تقديم VDC-Agent كإطار ذاتي التطور لتفصيل الفيديو، مما يلغي الحاجة إلى التعليقات البشرية أو نماذج المعلم الأكبر. يعمل من خلال حلقة مغلقة لتوليد التعليقات، وتقييمها، وتنقيح المحفزات، مما يسمح بتحسين مستمر في جودة التعليقات. لقد نجح الإطار في إنشاء مجموعة بيانات تحتوي على 18,886 زوجًا من التعليقات والتقييمات، مما أدى إلى تحقيق أداء رائد في معيار VDC.

Se ha presentado VDC-Agent como un marco auto-evolutivo para la subtitulación detallada de videos, eliminando la necesidad de anotaciones humanas o modelos maestros más grandes. Funciona a través de un ciclo cerrado de generación de subtítulos, puntuación y refinamiento de indicaciones, permitiendo una mejora continua en la calidad de los subtítulos. El marco ha logrado generar un conjunto de datos de 18,886 pares de subtítulos-puntuación, alcanzando un rendimiento de vanguardia en el benchmark VDC.

VDC-Agent a été présenté comme un cadre auto-évolutif pour la légende détaillée de vidéos, éliminant le besoin d'annotations humaines ou de modèles enseignants plus grands. Il fonctionne à travers une boucle fermée de génération de légendes, de notation et de raffinement d'invites, permettant une amélioration continue de la qualité des légendes. Le cadre a réussi à générer un ensemble de données de 18 886 paires légende-score, menant à des performances de pointe sur le benchmark VDC.

VDC-Agent has been introduced as a self-evolving framework for Video Detailed Captioning, eliminating the need for human annotations or larger teacher models. It operates through a closed loop of caption generation, scoring, and prompt refinement, allowing for continuous improvement in caption quality. The framework has successfully generated a dataset of 18,886 caption-score pairs, leading to state-of-the-art performance on the VDC benchmark.

VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflection

arXiv:2512.06663v1 Announce Type: new 
Abstract: Large Vision-Language Models (LVLMs) have demonstrated remarkable success in a broad range of vision-language tasks, such as general visual question answering and optical character recognition (OCR). However, their performance on perception-centric tasks -- such as object detection, semantic segmentation, and depth estimation -- remains significantly inferior to that of task-specific expert models. For example, Qwen2.5-VL-7B-Instruct achieves only 19% mAP on COCO2017 val, particularly struggling with dense scenes and small object recall. In this work, we introduce Chain-of-Thought for Detection (CoT4Det), a simple but efficient strategy that reformulates perception tasks into three interpretable steps: classification, counting, and grounding -- each more naturally aligned with the reasoning capabilities of LVLMs. Extensive experiments demonstrate that our method significantly improves perception performance without compromising general vision language capabilities. With a standard Qwen2.5-VL-7B-Instruct, CoT4Det boosts mAP from 19.0% to 33.0% on COCO2017 val and achieves competitive results across a variety of perception benchmarks, outperforming baselines by +2% on RefCOCO series and 19% on Flickr30k entities.

تهدف مقدمة CoT4Det، وهو إطار تفكير متسلسل، إلى تحسين أداء نماذج اللغة والرؤية الكبيرة (LVLM) في المهام الموجهة نحو الإدراك مثل اكتشاف الكائنات والتجزئة الدلالية، والتي كانت سابقًا متخلفة عن النماذج المتخصصة في المهام. يعيد هذا الإطار صياغة هذه المهام في ثلاث خطوات قابلة للتفسير: التصنيف، العد، والتثبيت.

La introducción de CoT4Det, un marco de pensamiento encadenado, tiene como objetivo mejorar el rendimiento de los Modelos de Lenguaje-Visión de Gran Escala (LVLM) en tareas orientadas a la percepción, como la detección de objetos y la segmentación semántica, que anteriormente habían quedado rezagadas en comparación con los modelos específicos para tareas. Este marco reformula estas tareas en tres pasos interpretables: clasificación, conteo y anclaje.

L'introduction de CoT4Det, un cadre de réflexion en chaîne, vise à améliorer les performances des grands modèles de vision-langage (LVLM) sur des tâches orientées vers la perception telles que la détection d'objets et la segmentation sémantique, qui ont précédemment été en retard par rapport aux modèles spécifiques à des tâches. Ce cadre reformule ces tâches en trois étapes interprétables : classification, comptage et ancrage.

The introduction of CoT4Det, a Chain-of-Thought framework, aims to enhance the performance of Large Vision-Language Models (LVLMs) on perception-oriented tasks such as object detection and semantic segmentation, which have previously lagged behind task-specific models. This framework reformulates these tasks into three interpretable steps: classification, counting, and grounding.

VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflection

Was this article worth reading? Share it

Chattermate

VideoDubber Video Translator

VidBoard AI