TiDAR: Think in Diffusion, Talk in Autoregression

TiDAR represents a significant advancement in the field of AI language modeling, merging the strengths of diffusion language models and autoregressive (AR) models. Traditional AR models excel in quality but often fall short in speed, while diffusion models promise faster generation but can compromise on output quality. TiDAR addresses these challenges by introducing a hybrid architecture that allows for efficient token drafting through diffusion and final output sampling via autoregression, all within a single forward pass. This innovative approach not only enhances throughput and GPU utilization but also maintains high-quality outputs, outperforming speculative decoding methods. Evaluated against various AR models and diffusion variants, TiDAR demonstrates a strong balance between drafting and verification capacity, marking a notable step forward in generative AI.