arXiv:2511.02778v1 Announce Type: cross 
Abstract: Code has emerged as a precise and executable medium for reasoning and action in the agent era. Yet, progress has largely focused on language-centric tasks such as program synthesis and debugging, leaving visual-centric coding underexplored. Inspired by how humans reason over sketches, we advocate SVG code as a compact, interpretable, and executable visual representation. We introduce VCode, a benchmark that reframes multimodal understanding as code generation: given an image, a model must produce SVG that preserves symbolic meaning for downstream reasoning. VCode covers three domains - general commonsense (MM-Vet), professional disciplines (MMMU), and visual-centric perception (CV-Bench). To assess symbolic fidelity, we propose CodeVQA, a novel evaluation protocol in which a policy model answers questions over rendered SVGs; correct answers indicate faithful symbolic preservation. Empirically, frontier VLMs struggle to generate faithful SVGs, revealing a persistent gap between language-centric and visual-centric coding. To close this gap, we introduce VCoder, an agentic framework that augments VLMs along two axes: (i) Thinking with Revision, which iteratively analyzes discrepancies and refines SVG code; and (ii) Acting with Visual Tools, where detectors and parsers supply structured cues such as objects, shapes, and text beyond the model's intrinsic capacity. Across benchmarks, frontier VLMs with strong reasoning capabilities score well overall yet remain limited in professional knowledge and 3D reasoning. VCoder delivers a 12.3-point overall gain over the top-performing Claude-4-Opus. Human studies show that both humans and VLMs perform worse on rendered SVGs, their consistency reveals the promise of symbolic visual representation. The benchmark and code are available at https://github.com/CSU-JPG/VCode.

VCode هو معيار مبتكر يركز على الترميز متعدد الوسائط، باستخدام SVG كتمثيل بصري. هذه المقاربة مهمة لأنها تحول التركيز من المهام البرمجية التي تركز على اللغة إلى منظور أكثر تركيزًا على الصورة، مما يعكس كيف يفكر البشر غالبًا في الرسومات. من خلال تقديم هذا المعيار الجديد، يهدف VCode إلى تعزيز فهمنا لكيفية دمج العناصر البصرية في البرمجة، مما قد يؤدي إلى طرق برمجة أكثر حدسية وفعالية.

VCode es un innovador benchmark que se centra en la codificación multimodal, utilizando SVG como representación visual. Este enfoque es significativo porque desplaza el foco de las tareas de codificación centradas en el lenguaje hacia una perspectiva más centrada en lo visual, que refleja cómo los humanos a menudo razonan con bocetos. Al introducir este nuevo benchmark, VCode busca mejorar nuestra comprensión de cómo se pueden integrar los elementos visuales en la codificación, lo que podría llevar a métodos de programación más intuitivos y efectivos.

VCode est un benchmark innovant qui se concentre sur le codage multimodal, en utilisant notamment le SVG comme représentation visuelle. Cette approche est importante car elle déplace l'accent des tâches de codage centrées sur le langage vers une perspective plus centrée sur le visuel, ce qui reflète la manière dont les humains raisonnent souvent avec des croquis. En introduisant ce nouveau benchmark, VCode vise à améliorer notre compréhension de la manière dont les éléments visuels peuvent être intégrés dans le codage, ce qui pourrait conduire à des méthodes de programmation plus intuitives et efficaces.

VCode is an innovative benchmark that focuses on multimodal coding, particularly using SVG as a visual representation. This approach is significant because it shifts the focus from traditional language-centric coding tasks to a more visual-centric perspective, which mirrors how humans often reason with sketches. By introducing this new benchmark, VCode aims to enhance our understanding of how visual elements can be integrated into coding, potentially leading to more intuitive and effective programming methods.

VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation

Was this article worth reading? Share it

Ready to build your own newsroom?