arXiv:2511.10993v1 Announce Type: new 
Abstract: Text-to-image synthesis models require the ability to generate diverse images while maintaining stability. To overcome this challenge, a number of methods have been proposed, including the collection of prompt-image datasets and the integration of additional data modalities during training. Although these methods have shown promising results in general domains, they face limitations when applied to specialized fields such as medicine, where only limited types and insufficient amounts of data are available. We present CLUE (Controllable Latent space of Unprompted Embeddings), a generative model framework that achieves diverse generation while maintaining stability through fixed-format prompts without requiring any additional data. Based on the Stable Diffusion architecture, CLUE employs a Style Encoder that processes images and prompts to generate style embeddings, which are subsequently fed into a new second attention layer of the U-Net architecture. Through Kullback-Leibler divergence, the latent space achieves continuous representation of image features within Gaussian regions, independent of prompts. Performance was assessed on otitis media dataset. CLUE reduced FID to 9.30 (vs. 46.81) and improved recall to 70.29% (vs. 49.60%). A classifier trained on synthetic-only data at 1000% scale achieved an F1 score of 83.21% (vs. 73.83%). Combining synthetic data with equal amounts of real data achieved an F1 score of 94.76%, higher than when using only real data. On an external dataset, synthetic-only training achieved an F1 score of 76.77% (vs. 60.61%) at 1000% scale. The combined approach achieved an F1 score of 85.78%, higher than when using only the internal dataset. These results demonstrate that CLUE enables diverse yet stable image generation from limited datasets and serves as an effective data augmentation method for domain-specific applications.

يقدم المقال CLUE (الفضاء الكامن القابل للتحكم من التضمينات غير المدفوعة)، وهو إطار نموذج توليدي مصمم لتوليد الصور من النص. يهدف CLUE إلى إنتاج صور متنوعة مع الحفاظ على الاستقرار، باستخدام مطالبات بتنسيق ثابت دون الحاجة إلى بيانات إضافية. يعتمد على بنية Stable Diffusion، ويستخدم مشفر نمط لإنشاء تضمينات نمطية، والتي تتم معالجتها من خلال طبقة انتباه جديدة في U-Net. تعالج هذه الطريقة التحديات التي تواجه المجالات المتخصصة مثل الطب، حيث تكون البيانات غالبًا محدودة.

El artículo presenta CLUE (Controllable Latent space of Unprompted Embeddings), un marco de modelo generativo diseñado para la síntesis de texto a imagen. CLUE busca generar imágenes diversas mientras asegura la estabilidad, utilizando indicaciones de formato fijo sin necesidad de datos adicionales. Basado en la arquitectura de Stable Diffusion, incorpora un codificador de estilo para crear embeddings de estilo, que se procesan a través de una nueva capa de atención en el U-Net. Este enfoque aborda los desafíos enfrentados en campos especializados como la medicina, donde los datos suelen ser l…

L'article présente CLUE (Controllable Latent space of Unprompted Embeddings), un cadre de modèle génératif conçu pour la synthèse d'images à partir de texte. CLUE vise à générer des images diverses tout en garantissant la stabilité, en utilisant des invites au format fixe sans nécessiter de données supplémentaires. Basé sur l'architecture Stable Diffusion, il intègre un encodeur de style pour créer des embeddings de style, qui sont traités à travers une nouvelle couche d'attention dans le U-Net. Cette approche répond aux défis rencontrés dans des domaines spécialisés comme la médecine, où les …

The article presents CLUE (Controllable Latent space of Unprompted Embeddings), a generative model framework designed for text-to-image synthesis. CLUE aims to generate diverse images while ensuring stability, utilizing fixed-format prompts without the need for additional data. Built on the Stable Diffusion architecture, it incorporates a Style Encoder to create style embeddings, which are processed through a new attention layer in the U-Net. This approach addresses challenges faced in specialized fields like medicine, where data is often limited.

CLUE: Controllable Latent space of Unprompted Embeddings for Diversity Management in Text-to-Image Synthesis

Was this article worth reading? Share it

Ready to build your own newsroom?