Terminal Velocity Matching

A new approach called Cross-Resolution Phase-Aligned Attention (CRPA) has been introduced to address a critical failure in the use of rotary positional embeddings (RoPE) within Diffusion Transformers, particularly when handling mixed-resolution denoising. This issue arises from linear interpolation that leads to phase aliasing, causing instability in the attention mechanism and resulting in artifacts or collapse during processing.

The recent paper titled 'Rectified SpaAttn: Revisiting Attention Sparsity for Efficient Video Generation' addresses the challenges posed by attention computation in video generation, particularly the latency introduced by the quadratic complexity of Diffusion Transformers. The authors propose a new method, Rectified SpaAttn, which aims to improve attention allocation by rectifying biases in the attention weights assigned to critical and non-critical tokens.

A new framework named Plan-X has been introduced to enhance video generation through high-level semantic planning, addressing the limitations of existing Diffusion Transformers in visual synthesis. The framework incorporates a Semantic Planner, which utilizes multimodal language processing to interpret user intent and generate structured spatio-temporal semantic tokens for video creation.