NeKo: Cross-Modality Post-Recognition Error Correction with Tasks-Guided Mixture-of-Experts Language Model

A systematic analysis of unified multimodal models has been conducted, revealing significant insights into their components' compressibility and sensitivity. The study utilized training-free pruning methodologies to assess depth and width adjustments, particularly noting that understanding components are more compressible in generation tasks compared to generation components, which are sensitive to compression.

SkyMoE has been introduced as a Mixture-of-Experts (MoE) vision-language model designed to improve geospatial interpretation, particularly in remote sensing tasks. This model addresses the limitations of existing general-purpose vision-language models by employing an adaptive router that generates task-specific routing instructions, allowing for enhanced differentiation between various tasks and interpretation granularities.

A new Mixture-of-Ranks (MoR) architecture has been proposed for one-step real-world image super-resolution (Real-ISR), integrating sparse Mixture-of-Experts (MoE) to enhance the adaptability of models in reconstructing high-resolution images from degraded samples. This approach utilizes a fine-grained expert partitioning strategy, treating each rank in Low-Rank Adaptation (LoRA) as an independent expert, thereby improving the model's ability to capture heterogeneous characteristics of real-world images.

A novel formulation for reinforcement learning (RL) with large language models (LLMs) has been proposed, focusing on optimizing true sequence-level rewards through a surrogate token-level objective in policy gradient methods like REINFORCE. The study emphasizes minimizing training-inference discrepancies and policy staleness to enhance the validity of this approach.