arXiv:2412.17153v3 Announce Type: replace-cross 
Abstract: Autoregressive (AR) models have achieved state-of-the-art performance in text and image generation but suffer from slow generation due to the token-by-token process. We ask an ambitious question: can a pre-trained AR model be adapted to generate outputs in just one or two steps? If successful, this would significantly advance the development and deployment of AR models. We notice that existing works that try to speed up AR generation by generating multiple tokens at once fundamentally cannot capture the output distribution due to the conditional dependencies between tokens, limiting their effectiveness for few-step generation. To address this, we propose Distilled Decoding (DD), which uses flow matching to create a deterministic mapping from Gaussian distribution to the output distribution of the pre-trained AR model. We then train a network to distill this mapping, enabling few-step generation. DD doesn't need the training data of the original AR model, making it more practical. We evaluate DD on state-of-the-art image AR models and present promising results on ImageNet-256. For VAR, which requires 10-step generation, DD enables one-step generation (6.3$\times$ speed-up), with an acceptable increase in FID from 4.19 to 9.96. For LlamaGen, DD reduces generation from 256 steps to 1, achieving an 217.8$\times$ speed-up with a comparable FID increase from 4.11 to 11.35. In both cases, baseline methods completely fail with FID>100. DD also excels on text-to-image generation, reducing the generation from 256 steps to 2 for LlamaGen with minimal FID increase from 25.70 to 28.95. As the first work to demonstrate the possibility of one-step generation for image AR models, DD challenges the prevailing notion that AR models are inherently slow, and opens up new opportunities for efficient AR generation. The project website is at https://imagination-research.github.io/distilled-decoding.

تستكشف دراسة حديثة إمكانية تعديل النماذج الذاتية الانحدار لتوليد الصور في خطوة أو خطوتين فقط، وهو تحسين كبير مقارنة بالنهج التقليدي خطوة بخطوة. قد تحدث هذه الخطوة ثورة في كفاءة توليد الصور، مما يجعلها أسرع وأكثر عملية لمجموعة متنوعة من التطبيقات. إذا نجحت، فإنها لن تعزز فقط أداء هذه النماذج، بل ستوسع أيضًا من قابليتها للاستخدام في المجالات الإبداعية والتكنولوجية.

Un estudio reciente explora el potencial de adaptar modelos autorregresivos para generar imágenes en uno o dos pasos, una mejora significativa en comparación con el enfoque tradicional de token por token. Este avance podría revolucionar la eficiencia de la generación de imágenes, haciéndola más rápida y práctica para diversas aplicaciones. Si tiene éxito, no solo mejoraría el rendimiento de estos modelos, sino que también ampliaría su utilidad en campos creativos y tecnológicos.

Une étude récente explore le potentiel d'adaptation des modèles autorégressifs pour générer des images en une ou deux étapes, une amélioration significative par rapport à l'approche traditionnelle étape par étape. Cette avancée pourrait révolutionner l'efficacité de la génération d'images, la rendant plus rapide et plus pratique pour diverses applications. Si elle réussit, cela améliorerait non seulement les performances de ces modèles, mais élargirait également leur utilité dans les domaines créatifs et technologiques.

A recent study explores the potential of adapting autoregressive models to generate images in just one or two steps, a significant improvement over the traditional token-by-token approach. This advancement could revolutionize the efficiency of image generation, making it faster and more practical for various applications. If successful, it would not only enhance the performance of these models but also broaden their usability in creative fields and technology.

Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching

One More Thing in AI – Your Shortcut to AI Mastery

Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching

Was this article worth reading? Share it

One More Thing in AI

Https

Bulk Image Generation AI

AI Flow Chat

ClipCutAi

Blunge

Ready to build your own newsroom?