arXiv:2605.29462v1 Announce Type: new 
Abstract: The emergence of Large Vision-Language Models (LVLMs) has substantially expanded model capabilities beyond text-only understanding, enabling unified inference across both visual and textual modalities and supporting a broader range of real-world applications. To comprehensively evaluate the perception, understanding, reasoning, and cognition capabilities of LVLMs throughout the entire financial business workflow in Chinese contexts, we introduce CFMME, a novel Chinese financial multimodal evaluation benchmark. CFMME comprises 6,052 instances spanning from fundamental academic knowledge to complex real-world applications, covering eight primary financial image modalities and four core multimodal tasks. On CFMME, we conduct a thorough evaluation of representative LVLMs. The results show that the state-of-the-art model attains an overall accuracy of 66.11\% on the question answering task and an average score of 77.18 on the detection, recognition, and information extraction tasks, indicating substantial room for improvement in current LVLMs. In addition, we conduct detailed analyses of error causes, cross-modal capabilities, and multi-orientation settings, yielding valuable insights for future research. We hope that CFMME will spur further progress in LVLMs, especially by improving their performance on multiple multimodal tasks in the financial domain.

تم تقديم معيار جديد يسمى CFMME لتقييم نماذج اللغة والرؤية الكبيرة (LVLM) في سياق التطبيقات المالية الصينية. يتضمن هذا المعيار 6,052 حالة تتراوح من المعرفة الأكاديمية الأساسية إلى السيناريوهات المعقدة في العالم الحقيقي، تغطي مجموعة متنوعة من أنماط الصور المالية والمهام متعددة الوسائط. يهدف المعيار إلى تقييم قدرات الإدراك والفهم والتفكير والإدراك لهذه النماذج.

Se ha introducido un nuevo benchmark llamado CFMME para evaluar los Modelos de Lenguaje-Vision de Gran Escala (LVLM) en el contexto de las aplicaciones financieras chinas. Este conjunto de datos incluye 6,052 instancias que van desde conocimientos académicos básicos hasta escenarios complejos del mundo real, abarcando diversas modalidades de imágenes financieras y tareas multimodales. El benchmark tiene como objetivo evaluar las capacidades de percepción, comprensión, razonamiento y cognición de estos modelos.

Un nouveau benchmark appelé CFMME a été introduit pour évaluer les grands modèles de vision-langage (LVLM) dans le contexte des applications financières chinoises. Ce jeu de données comprend 6 052 instances allant des connaissances académiques de base à des scénarios réels complexes, couvrant diverses modalités d'images financières et des tâches multimodales. Le benchmark vise à évaluer les capacités de perception, de compréhension, de raisonnement et de cognition de ces modèles.

A new benchmark called CFMME has been introduced to evaluate Large Vision-Language Models (LVLMs) in the context of Chinese financial applications. This dataset includes 6,052 instances that range from basic academic knowledge to complex real-world scenarios, covering various financial image modalities and multimodal tasks. The benchmark aims to assess the perception, understanding, reasoning, and cognition capabilities of these models.