Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenization

A new technique called Gaussian Quant (GQ) has been introduced to enhance the training of Vector Quantized Variational Autoencoders (VQ-VAE), which are used for compressing images into discrete tokens. This method allows for the conversion of a Gaussian VAE into a VQ-VAE without the need for extensive training, thereby simplifying the process and improving performance.

A new approach to Vision Transformers (ViTs) has been introduced, featuring a Jumbo token that enhances processing speed by reducing patch token width while increasing global token width. This innovation aims to address the slow performance of ViTs without compromising their generality or accuracy, making them more practical for various applications.

A new paper introduces an innovative dataset quantization method aimed at reducing storage and communication costs for large-scale datasets on resource-constrained edge devices. This approach focuses on compressing individual samples by minimizing intra-sample redundancy while retaining essential features, marking a shift from traditional inter-sample redundancy methods.

A new study proposes a structured initialization method for Vision Transformers (ViTs), aiming to integrate the strong inductive biases of Convolutional Neural Networks (CNNs) without altering the architecture. This approach is designed to enhance performance on small datasets while maintaining scalability as data increases.

The Iwin Transformer has been introduced as a novel hierarchical vision transformer that operates without position embeddings, utilizing interleaved window attention and depthwise separable convolution to enhance performance across various visual tasks. This architecture allows for direct fine-tuning from low to high resolution, achieving notable results such as 87.4% top-1 accuracy on ImageNet-1K.