Scaling Multimodal Search and Recommendation with Small Language Models via Upside-Down Reinforcement Learning

The Network of Theseus (NoT) introduces a novel approach in deep learning by allowing the transformation of a guide network architecture into a different target architecture while maintaining performance. This method challenges the traditional assumption that the architecture used during training must remain unchanged during inference, thereby offering flexibility in model design and optimization.

Large-scale Mixture of Experts (MoE) Large Language Models (LLMs) have emerged as leading open-weight models, but their random expert selection mechanism leads to significant data movement overhead. A recent study conducted comprehensive profiling across four state-of-the-art MoE models, revealing insights that can enhance future serving systems and reduce bottlenecks in multi-unit LLM serving.

Jina-VLM, a 2.4 billion parameter vision-language model, has been introduced, achieving state-of-the-art multilingual visual question answering capabilities among open 2B-scale VLMs. It integrates a SigLIP2 vision encoder with a Qwen3 language backbone, allowing for efficient processing of images at arbitrary resolutions.

A new framework called GRASP (GRouped Activation Shared Parameterization) has been introduced for parameter-efficient fine-tuning of transformers, allowing for the training of large pre-trained models by updating only a small subset of parameters. This method partitions token representations into groups, learning shared scaling and shifting vectors to enhance model performance while significantly reducing the number of trainable parameters.

A novel method called Dual LoRA has been proposed to enhance the performance of Low-Rank Adaptation (LoRA) in fine-tuning large language models (LLMs). This method introduces two distinct groups within low-rank matrices: a magnitude group for controlling the extent of parameter updates and a direction group for determining the update direction, thereby improving the adaptation process.

The Idea-Gated Transformer has been introduced as a novel architecture aimed at addressing the issue of 'Topic Drift' in Autoregressive Language Models (LLMs) during text generation. This model separates semantic planning from syntactic generation by utilizing an auxiliary 'Idea Head' that predicts future context, allowing for real-time vocabulary pruning to enhance coherence in generated text.

The recent introduction of DESIGNER, a design-logic-guided reasoning data synthesis pipeline, aims to enhance the capabilities of large language models (LLMs) in tackling complex, multidisciplinary questions. By leveraging extensive raw documents, DESIGNER generates high-difficulty questions that challenge LLMs' reasoning abilities across various disciplines.

OpenREAD is a newly proposed framework that enhances end-to-end autonomous driving by integrating a vision-language model with reinforced open-ended reasoning, addressing limitations in traditional supervised fine-tuning and reinforcement fine-tuning methods. This innovation aims to improve decision-making and planning in complex driving scenarios.