Edit Flows: Flow Matching with Edit Operations

Edit Flows represents a breakthrough in non-autoregressive modeling by introducing a discrete flow over sequences through edit operations, which include insertions, deletions, and substitutions. This model is built upon a Continuous-time Markov Chain framework, enabling a more flexible and position-relative generation of sequences. The training method employed utilizes an expanded state space with auxiliary variables, enhancing the efficiency of the learning process. Empirical results demonstrate that Edit Flows significantly outperforms both traditional autoregressive models and mask-based models in various applications, including image captioning, text generation, and code generation. This advancement not only addresses the limitations of existing models but also sets a new standard for future developments in AI sequence generation.