Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models with Flow Matching

Recent advancements in autoregressive (AR) models have demonstrated significant potential in image generation, surpassing diffusion-based methods in scalability and integration with multi-modal systems. However, challenges remain in extending AR models to general image editing due to inefficient conditioning, which can result in poor adherence to instructions and visual artifacts. To tackle these issues, the proposed SCAR method introduces Compressed Semantic Prefilling and Semantic Alignment Guidance, enhancing the fidelity of instructions during the autoregressive decoding process.

Vision-Language Models (VLMs) are emerging models that integrate visual content with natural language. Current methods typically fuse data either early in the encoding process or late through pooled embeddings. This paper introduces a lightweight fusion module utilizing cross-only, bidirectional attention layers to align hidden states from both modalities, enhancing understanding while keeping encoders non-causal. The proposed method aims to improve the performance of VLMs by leveraging the inherent structure of visual and textual data.

The paper titled 'Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning' introduces a new method called Bias-REstrained Prefix Representation FineTuning (BREP ReFT). This approach aims to enhance the mathematical reasoning capabilities of models by addressing the limitations of existing Representation finetuning (ReFT) methods, which struggle with mathematical tasks. The study demonstrates that BREP ReFT outperforms both standard ReFT and weight-based Parameter-Efficient finetuning (PEFT) methods through extensive experiments.

The paper titled 'Parameter-Efficient MoE LoRA for Few-Shot Multi-Style Editing' addresses the challenges faced by general image editing models when adapting to new styles. It proposes a novel few-shot style editing framework and introduces a benchmark dataset comprising five distinct styles. The framework utilizes a parameter-efficient multi-style Mixture-of-Experts Low-Rank Adaptation (MoE LoRA) that employs style-specific and style-shared routing mechanisms to fine-tune multiple styles effectively. This approach aims to enhance the performance of image editing models with minimal data.

The article presents a novel flow matching (FM)-based generative model for multiple-input multiple-output (MIMO) channel estimation. This approach addresses the slow sampling speed challenge associated with diffusion model (DM)-based schemes by formulating the channel estimation problem within the FM framework. The proposed method shows potential for superior channel estimation accuracy and significantly reduced sampling overhead compared to existing DM-based methods.

Higher-order Neural Additive Models (HONAMs) have been introduced as an advancement over Neural Additive Models (NAMs), which are known for their predictive performance and interpretability. HONAMs address the limitation of NAMs by effectively capturing feature interactions of arbitrary orders, enhancing predictive accuracy while maintaining interpretability, crucial for high-stakes applications. The source code for HONAM is publicly available on GitHub.

The study investigates the ability of transformer models to predict long steps in the Collatz sequence, a complex arithmetic function that maps odd integers to their successors. The accuracy of the models varies significantly depending on the base used for encoding, achieving up to 99.7% accuracy for bases 24 and 32, while dropping to 37% and 25% for bases 11 and 3. Despite these variations, all models exhibit a common learning pattern, accurately predicting inputs with similar residuals modulo 2^p.