Emergent Bayesian Behaviour and Optimal Cue Combination in LLMs

The introduction of InEx presents a novel approach to mitigating hallucinations in large language models (LLMs) by employing a training-free, multi-agent framework that incorporates introspective reasoning and cross-modal collaboration. This method aims to enhance the reliability of multimodal LLMs (MLLMs) by autonomously refining responses through iterative verification processes.

Researchers have found that large language models (LLMs) make decisions about networking and friendship in ways that closely resemble human behavior, both in synthetic simulations and real-world contexts. This suggests that LLMs can replicate social decision-making processes similar to those of people.

A new quantization method called Four Over Six (4/6) has been introduced to enhance the NVFP4 quantization algorithm, which is crucial for the efficient training and inference of large language models (LLMs). This method evaluates multiple scale factors for blocks of values, aiming to reduce quantization errors that can lead to performance degradation during model training and inference.

The recent introduction of Mixture-of-Head attention (MoH) enhances the multi-head attention mechanism central to Transformer models, aiming to improve efficiency while maintaining or exceeding previous accuracy levels. This new architecture allows tokens to select relevant attention heads, thereby optimizing inference without increasing parameters.

Recent advancements in Diffusion Mixture-of-Experts (MoE) models have shifted focus from routing mechanisms to architectural configurations, revealing that factors like expert modules and attention encodings are crucial for model effectiveness. This systematic study emphasizes the importance of tuning these configurations to maximize performance in both latent and pixel-space diffusion frameworks.

A new method called ZIP-RC has been introduced to enhance the inference capabilities of large language models (LLMs) by enabling real-time prediction of reward and cost during generation. This approach addresses the limitations of existing test-time scaling methods, which often lead to increased costs and latency without providing adaptive inference capabilities.

Recent research introduces Semantically Equivalent and Coherent Attacks (SECA), a method designed to elicit hallucinations from Large Language Models (LLMs) through realistic prompt modifications that maintain semantic coherence. This approach addresses the limitations of previous adversarial attacks that often resulted in unrealistic prompts, thereby enhancing understanding of how hallucinations can occur in practical applications.

InnoGym has been introduced as the first benchmark and framework designed to systematically evaluate the innovation potential of AI agents, focusing on both performance gain and novelty across 18 tasks from real-world engineering and scientific domains. This initiative aims to address the limitations of existing benchmarks that primarily measure correctness without considering the diversity of methods behind solutions.