arXiv:2503.07265v3 Announce Type: replace 
Abstract: Text-to-Image (T2I) models are capable of generating high-quality artistic creations and visual content. However, existing research and evaluation standards predominantly focus on image realism and shallow text-image alignment, lacking a comprehensive assessment of complex semantic understanding and world knowledge integration in text-to-image generation. To address this challenge, we propose \textbf{WISE}, the first benchmark specifically designed for \textbf{W}orld Knowledge-\textbf{I}nformed \textbf{S}emantic \textbf{E}valuation. WISE moves beyond simple word-pixel mapping by challenging models with 1000 meticulously crafted prompts across 25 subdomains in cultural common sense, spatio-temporal reasoning, and natural science. To overcome the limitations of traditional CLIP metric, we introduce \textbf{WiScore}, a novel quantitative metric for assessing knowledge-image alignment. Through comprehensive testing of 20 models (10 dedicated T2I models and 10 unified multimodal models) using 1,000 structured prompts spanning 25 subdomains, our findings reveal significant limitations in their ability to effectively integrate and apply world knowledge during image generation, highlighting critical pathways for enhancing knowledge incorporation and application in next-generation T2I models. Code and data are available at \href{https://github.com/PKU-YuanGroup/WISE}{PKU-YuanGroup/WISE}.

يقدم المقال WISE، وهو معيار جديد لتقييم نماذج تحويل النص إلى صورة (T2I)، مع التركيز على الفهم الدلالي المعقد ودمج المعرفة العالمية. على عكس التقييمات الحالية التي تركز على واقعية الصورة ومحاذاة النص والصورة السطحية، يتحدى WISE النماذج بـ 1000 طلب عبر مجالات فرعية مختلفة. بالإضافة إلى ذلك، يقدم WiScore، وهو مقياس كمي جديد لتقييم محاذاة المعرفة والصورة.

El artículo presenta WISE, un nuevo estándar para evaluar modelos de generación de imágenes a partir de texto (T2I), centrado en la comprensión semántica compleja y la integración del conocimiento del mundo. A diferencia de las evaluaciones existentes que se enfocan en el realismo de las imágenes y la alineación básica texto-imagen, WISE desafía a los modelos con 1000 indicaciones en varios subdominios. Además, se introduce WiScore, una nueva métrica para evaluar la alineación entre conocimiento e imagen.

L'article présente WISE, une nouvelle référence pour évaluer les modèles de génération d'images à partir de texte (T2I), en mettant l'accent sur la compréhension sémantique complexe et l'intégration des connaissances du monde. Contrairement aux évaluations existantes qui privilégient le réalisme des images et l'alignement texte-image de base, WISE défie les modèles avec 1000 invites dans divers sous-domaines. De plus, il présente WiScore, une nouvelle métrique pour évaluer l'alignement des connaissances et des images.

The article introduces WISE, a new benchmark for evaluating Text-to-Image (T2I) models, focusing on complex semantic understanding and world knowledge integration. Unlike existing evaluations that emphasize image realism and basic text-image alignment, WISE challenges models with 1000 prompts across various subdomains. Additionally, it presents WiScore, a new metric for assessing knowledge-image alignment, addressing limitations of traditional evaluation methods.

WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation

arXiv:2511.20561v1 Announce Type: new 
Abstract: Recent years have witnessed significant progress in Unified Multimodal Models, yet a fundamental question remains: Does understanding truly inform generation? To investigate this, we introduce UniSandbox, a decoupled evaluation framework paired with controlled, synthetic datasets to avoid data leakage and enable detailed analysis. Our findings reveal a significant understanding-generation gap, which is mainly reflected in two key dimensions: reasoning generation and knowledge transfer. Specifically, for reasoning generation tasks, we observe that explicit Chain-of-Thought (CoT) in the understanding module effectively bridges the gap, and further demonstrate that a self-training approach can successfully internalize this ability, enabling implicit reasoning during generation. Additionally, for knowledge transfer tasks, we find that CoT assists the generative process by helping retrieve newly learned knowledge, and also discover that query-based architectures inherently exhibit latent CoT-like properties that affect this transfer. UniSandbox provides preliminary insights for designing future unified architectures and training strategies that truly bridge the gap between understanding and generation. Code and data are available at https://github.com/PKU-YuanGroup/UniSandBox

شهدت النماذج متعددة الوسائط الموحدة تقدمًا كبيرًا مؤخرًا، مما أثار تساؤلات حول ما إذا كانت الفهم حقًا يُعلم التوليد. تهدف UniSandbox، وهي إطار تقييم مفصول، إلى معالجة هذا الأمر من خلال استخدام مجموعات بيانات اصطناعية محكومة لتحليل الفجوة بين الفهم والتوليد، خاصة في مهام توليد التفكير ونقل المعرفة.

Los avances recientes en Modelos Multimodales Unificados han planteado la cuestión de si la comprensión realmente informa la generación. La introducción de UniSandbox, un marco de evaluación desacoplado, busca abordar esto utilizando conjuntos de datos sintéticos controlados para analizar la brecha entre comprensión y generación, especialmente en tareas de generación de razonamiento y transferencia de conocimiento.

Les avancées récentes dans les modèles multimodaux unifiés soulèvent la question de savoir si la compréhension informe la génération. L'introduction d'UniSandbox, un cadre d'évaluation découplé, vise à aborder cette question en utilisant des ensembles de données synthétiques contrôlés pour analyser l'écart entre compréhension et génération, en particulier dans les tâches de génération de raisonnement et de transfert de connaissances.

Recent advancements in Unified Multimodal Models have raised the question of whether understanding informs generation. The introduction of UniSandbox, a decoupled evaluation framework, aims to address this by utilizing controlled synthetic datasets to analyze the understanding-generation gap, particularly in reasoning generation and knowledge transfer tasks.

WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation

Was this article worth reading? Share it

Typewell

Humanize AI

AI Humanizer