SkyMoE: A Vision-Language Foundation Model for Enhancing Geospatial Interpretation with Mixture of Experts

A systematic analysis of unified multimodal models has been conducted, revealing significant insights into their components' compressibility and sensitivity. The study utilized training-free pruning methodologies to assess depth and width adjustments, particularly noting that understanding components are more compressible in generation tasks compared to generation components, which are sensitive to compression.

A new Mixture-of-Ranks (MoR) architecture has been proposed for one-step real-world image super-resolution (Real-ISR), integrating sparse Mixture-of-Experts (MoE) to enhance the adaptability of models in reconstructing high-resolution images from degraded samples. This approach utilizes a fine-grained expert partitioning strategy, treating each rank in Low-Rank Adaptation (LoRA) as an independent expert, thereby improving the model's ability to capture heterogeneous characteristics of real-world images.

A novel formulation for reinforcement learning (RL) with large language models (LLMs) has been proposed, focusing on optimizing true sequence-level rewards through a surrogate token-level objective in policy gradient methods like REINFORCE. The study emphasizes minimizing training-inference discrepancies and policy staleness to enhance the validity of this approach.

The recent introduction of NeKo, a Mixture-of-Experts (MoE) language model, aims to enhance post-recognition error correction across various modalities, including speech-to-text and vision-to-text. This model leverages a multi-task correction approach, allowing it to learn from diverse datasets while minimizing the increase in parameters typically associated with separate correction models.