<a href="https://arxiv.org/abs/2512.24601">Recursive language models</a> (RLMs) are an inference technique developed by researchers at MIT CSAIL that treat long prompts as an external environment to the model. Instead of forcing the entire prompt into the model&#x27;s context window, the framework allows the LLM to programmatically examine, decompose, and recursively call itself over snippets of the text.Rather than expanding context windows or summarizing old information, the MIT team reframes long-context reasoning as a systems problem. By letting models treat prompts as something they can inspect with code, recursive language models allow LLMs to reason over millions of tokens without retraining. This offers enterprises a practical path to long-horizon tasks like codebase analysis, legal review, and multi-step reasoning that routinely break today’s models.Because the framework is designed as a wrapper around existing models, it can serve as a drop-in replacement for applications that make direct calls to LLMs.<h2>The LLM context problem</h2>While frontier models are becoming increasingly sophisticated at reasoning, their ability to process massive amounts of information is not scaling at the same rate. This bottleneck is driven by two distinct limitations: the hard physical constraint on how much text a model can process at once (context length) and &quot;context rot.&quot;The challenge, the researchers argue, is whether it’s possible to scale the effective context size of general-purpose LLMs by orders of magnitude without retraining them. This capability is becoming increasingly important for enterprise applications, where LLMs are adopted for long-horizon tasks requiring the processing of millions of tokens — a challenge Zhang argues can’t be solved by simply expanding context windows.&quot;There is an entropy argument that implies you need exponentially more data samples as you increase the effective context window size,&quot; Alex Zhang, a co-author of the paper, told VentureBeat. Current approaches to extending context often rely on compaction, where the model summarizes older parts of the conversation to free up space. However, this method fails for tasks requiring random access to specific details located in earlier parts of the prompt.<h2>How RLMs work</h2>The concept behind RLMs is drawn from &quot;out-of-core&quot; algorithms used in classical computing. These algorithms are designed to process datasets too large to fit into a computer&#x27;s main memory by keeping the data on a hard drive and fetching only the necessary chunks as needed.RLMs apply this logic to generative AI. Instead of feeding a long prompt directly into the neural network, the framework loads the text as a string variable inside a Python coding environment. The LLM is given general context about the data (such as the total character count) but does not &quot;see&quot; the text initially.Once the prompt is stored as a variable, the LLM acts as a programmer. It writes Python code to interact with the external variable, using standard commands to peek into the data. For example, the model might use regular expressions to search for specific keywords like &quot;Chapter 1&quot; or &quot;financial results.&quot;When the code execution finds a relevant snippet, the RLM pulls only that specific chunk into its active context window for analysis.For example, if the prompt is a massive book, the LLM might write a loop that identifies chapter boundaries and then triggers a sub-call to summarize each chapter individually.The architecture typically involves two agents. A &quot;root language model,&quot; often a capability-heavy model like GPT-5, acts as the orchestrator. It plans the approach, writes the code, and manages the data flow within the REPL environment. A &quot;recursive language model,&quot; often a faster and cheaper model, acts as the worker. The root LM calls this worker to process the specific text snippets isolated by the code.Because the prompt resides in the environment&#x27;s memory rather than the model&#x27;s context window, the system can handle inputs far larger than the model&#x27;s training limit. Importantly, to the end-user, the RLM behaves exactly like a standard model: It accepts a string and returns an answer. This allows enterprise teams to swap standard API calls for RLMs. For developers looking to experiment, the RLM code is currently available on <a href="https://github.com/ysz/recursive-llm">GitHub</a>. &quot;A key argument for RLMs is that most complex tasks can be decomposed into smaller, &#x27;local&#x27; sub-tasks,&quot; Zhang said. &quot;However, how to perform this context/problem decomposition is non-trivial, and the model must be capable of performing this.&quot;<h2>RLMs in action</h2>To validate the framework, the researchers tested RLMs against base models and other agentic approaches like CodeAct and summary agents across a variety of long-context tasks, including retrieval and multi-hop question answering.The results demonstrated strong performance gains at the 10 million+ token scale. On <a href="https://github.com/texttron/BrowseComp-Plus">BrowseComp-Plus</a>, a benchmark involving inputs of 6 to 11 million tokens, standard base models failed completely, scoring 0%. In contrast, the RLM powered by <a href="https://venturebeat.com/ai/openai-launches-gpt-5-not-agi-but-capable-of-generating-software-on-demand">GPT-5</a> achieved a score of 91.33%, significantly outperforming the Summary Agent (70.47%) and <a href="https://machinelearning.apple.com/research/codeact">CodeAct </a>(51%).The framework also excelled at tasks with high computational complexity. On OOLONG-Pairs, an information-dense reasoning benchmark where the difficulty scales quadratically with input length, base GPT-5 models failed catastrophically with a score of just 0.04%. The RLM achieved an F1 score (a balanced measure of precision and recall) of 58%, demonstrating emergent capabilities to handle dense tasks that paralyze standard models. Similarly, on code understanding tasks (CodeQA benchmark), the RLM more than doubled the performance of the base GPT-5 model, jumping from 24% to 62%.Regarding the context rot problem, the data showed that while the base GPT-5 performance degrades rapidly as task complexity increases, RLM performance holds steady, consistently outperforming the base model on contexts longer than 16,000 tokens.Despite the increased complexity of the workflow, RLMs often maintained comparable or lower average costs than the baselines. On the BrowseComp-Plus benchmark, the RLM was up to three times cheaper than the summarization baseline. However, the researchers noted that while median costs are low, RLM trajectories are &quot;long-tailed.&quot; Outlier runs can become expensive if the model gets stuck in loops or performs redundant verifications. While GPT-5 was conservative in its sub-calls, the open-source <a href="https://venturebeat.com/programming-development/qwen3-coder-480b-a35b-instruct-launches-and-it-might-be-the-best-coding-model-yet">Qwen3-Coder</a> model sometimes attempted thousands of sub-calls for simple tasks.&quot;Today, you likely will have to implement your own guardrails and logic to control RLM behavior,&quot; Zhang said. However, he hypothesizes that future models could be trained to manage their own compute budgets more effectively. Companies like Prime Intellect are planning to <a href="https://www.primeintellect.ai/blog/rlm">integrate RLM</a> into the training process of models, possibly addressing the edge cases where the model’s inference budget spikes.For enterprise architects deciding where to place their bets, the RLM framework offers a new tool for handling information-dense problems.&quot;I think RLMs are still extremely useful for chatbots (think long chat histories), but ultimately they argue for an alternative way of using LMs,&quot; Zhang said. &quot;I think RLMs work in tandem with standard retrieval methods like RAG; they do not serve as a replacement, and can be used in different settings or together.&quot;

طور باحثون في معهد ماساتشوستس للتكنولوجيا (MIT CSAIL) إطار عمل جديد يسمى نماذج اللغة التكرارية (RLM) يسمح للنماذج اللغوية الكبيرة (LLMs) بمعالجة ما يصل إلى 10 ملايين رمز دون التعرض لتدهور السياق. تتيح هذه الطريقة المبتكرة للنماذج اللغوية تحليل النصوص الطويلة وتفكيكها برمجيًا، مما يعزز قدراتها على التفكير دون الحاجة إلى إعادة التدريب.

Investigadores del MIT CSAIL han desarrollado un nuevo marco llamado Modelos de Lenguaje Recursivos (RLM) que permite a los modelos de lenguaje grandes (LLM) procesar hasta 10 millones de tokens sin experimentar degradación del contexto. Este enfoque innovador permite a los LLM analizar y descomponer programáticamente largas solicitudes, mejorando sus capacidades de razonamiento sin necesidad de reentrenamiento.

Des chercheurs du MIT CSAIL ont développé un nouveau cadre appelé Modèles de Langage Récursifs (RLM) qui permet aux grands modèles de langage (LLM) de traiter jusqu'à 10 millions de tokens sans subir de dégradation du contexte. Cette approche innovante permet aux LLM d'analyser et de décomposer programmatique des longues invites, améliorant ainsi leurs capacités de raisonnement sans nécessiter de réentraînement.

Researchers at MIT CSAIL have developed a new framework called Recursive Language Models (RLMs) that enables large language models (LLMs) to process up to 10 million tokens without experiencing context rot. This innovative approach allows LLMs to programmatically analyze and decompose long prompts, enhancing their reasoning capabilities without the need for retraining.

MIT’s new ‘recursive’ framework lets LLMs process 10 million tokens without context rot

Every year, NeurIPS produces hundreds of impressive papers, and a handful that subtly reset how practitioners think about scaling, evaluation and system design. In 2025, the most consequential works weren&#x27;t about a single <a href="https://venturebeat.com/orchestration/why-your-llm-bill-is-exploding-and-how-semantic-caching-can-cut-it-by-73">breakthrough model</a>. Instead, they challenged fundamental assumptions that academicians and corporations have quietly relied on: Bigger models mean better reasoning, RL creates new capabilities, attention is “solved” and generative models inevitably memorize.This year’s top papers collectively point to a deeper shift: AI progress is now constrained less by raw model capacity and more by architecture, training dynamics and evaluation strategy.Below is a technical deep dive into five of the most influential NeurIPS 2025 papers — and what they mean for anyone building real-world AI systems.<h2>1. LLMs are converging—and we finally have a way to measure it</h2>Paper: <a href="https://arxiv.org/abs/2510.22954">Artificial Hivemind: The Open-Ended Homogeneity of Language Models</a>For years, <a href="https://venturebeat.com/orchestration/why-which-api-do-i-call-is-the-wrong-question-in-the-llm-era">LLM evaluation</a> has focused on correctness. But in open-ended or ambiguous tasks like brainstorming, ideation or creative synthesis, there often is no single correct answer. The risk instead is homogeneity: Models producing the same “safe,” high-probability responses.This paper introduces Infinity-Chat, a benchmark designed explicitly to measure diversity and pluralism in open-ended generation. Rather than scoring answers as right or wrong, it measures:<ul><li>Intra-model collapse: How often the same model repeats itself</li><li>Inter-model homogeneity: How similar different models’ outputs are</li></ul>The result is uncomfortable but important: Across architectures and providers, models increasingly converge on similar outputs — even when multiple valid answers exist.<h2>Why this matters in practice</h2>For corporations, this reframes “alignment” as a trade-off. Preference tuning and safety constraints can quietly reduce diversity, leading to assistants that feel too safe, predictable or biased toward dominant viewpoints.Takeaway: If your product relies on creative or exploratory outputs, diversity metrics need to be first-class citizens. <h2>2. Attention isn’t finished — a simple gate changes everything</h2>Paper: <a href="https://arxiv.org/abs/2505.06708">Gated Attention for Large Language Models</a>Transformer attention has been treated as settled engineering. This paper proves it isn’t.The authors introduce a small architectural change: Apply a query-dependent sigmoid gate after scaled dot-product attention, per attention head. That’s it. No exotic kernels, no massive overhead.Across dozens of large-scale training runs — including dense and <a href="https://venturebeat.com/ai/mixture-of-recursions-delivers-2x-faster-inference-heres-how-to-implement-it">mixture-of-experts (MoE)</a> models trained on trillions of tokens — this gated variant:<ul><li>Improved stability</li><li>Reduced “attention sinks”</li><li>Enhanced long-context performance</li><li>Consistently outperformed vanilla attention</li></ul><h3>Why it works</h3>The gate introduces:<ul><li>Non-linearity in attention outputs</li><li>Implicit sparsity, suppressing pathological activations</li></ul>This challenges the assumption that attention failures are purely data or optimization problems.Takeaway: Some of the biggest LLM reliability issues may be architectural — not algorithmic — and solvable with surprisingly small changes.<h2>3. RL can scale — if you scale in depth, not just data</h2>Paper: <a href="https://arxiv.org/abs/2503.14858">1,000-Layer Networks for Self-Supervised Reinforcement Learnin</a>gConventional wisdom says RL doesn’t scale well without dense rewards or demonstrations. This paper reveals that that assumption is incomplete.By scaling network depth aggressively from typical 2 to 5 layers to nearly 1,000 layers, the authors demonstrate dramatic gains in self-supervised, goal-conditioned RL, with performance improvements ranging from 2X to 50X.The key isn’t brute force. It’s pairing depth with contrastive objectives, stable optimization regimes and goal-conditioned representations<h3>Why this matters beyond robotics</h3>For agentic systems and autonomous workflows, this suggests that representation depth — not just data or reward shaping — may be a critical lever for generalization and exploration.Takeaway: RL’s scaling limits may be architectural, not fundamental.<h2>4. Why diffusion models generalize instead of memorizing</h2>Paper: <a href="https://arxiv.org/abs/2505.17638">Why Diffusion Models Don&#x27;t Memorize: The Role of Implicit Dynamical Regularization in Training</a>Diffusion models are massively overparameterized, yet they often generalize remarkably well. This paper explains why.The authors identify two distinct training timescales:<ul><li>One where generative quality rapidly improves</li><li>Another — much slower — where memorization emerges</li></ul>Crucially, the memorization timescale grows linearly with dataset size, creating a widening window where models improve without overfitting.<h3>Practical implications</h3>This reframes early stopping and dataset scaling strategies. Memorization isn’t inevitable — it’s predictable and delayed.Takeaway: For diffusion training, dataset size doesn’t just improve quality — it actively delays overfitting.<h2>5. RL improves reasoning performance, not reasoning capacity </h2>Paper: <a href="https://arxiv.org/abs/2504.13837">Does Reinforcement Learning Really Incentivize Reasoning in LLMs?</a>Perhaps the most strategically important result of NeurIPS 2025 is also the most sobering.This paper rigorously tests whether reinforcement learning with verifiable rewards (RLVR) actually creates new reasoning abilities in LLMs — or simply reshapes existing ones.Their conclusion: RLVR primarily improves sampling efficiency, not reasoning capacity. At large sample sizes, the base model often already contains the correct reasoning trajectories.<h3>What this means for LLM training pipelines</h3>RL is better understood as:<ul><li>A distribution-shaping mechanism</li><li>Not a generator of fundamentally new capabilities</li></ul>Takeaway: To truly expand reasoning capacity, RL likely needs to be paired with mechanisms like teacher distillation or architectural changes — not used in isolation.<h2>The bigger picture: AI progress is becoming systems-limited </h2>Taken together, these papers point to a common theme:The bottleneck in <a href="https://venturebeat.com/infrastructure/agent-autonomy-without-guardrails-is-an-sre-nightmare">modern AI</a> is no longer raw model size — it’s system design.<ul><li>Diversity collapse requires new evaluation metrics</li><li>Attention failures require architectural fixes</li><li>RL scaling depends on depth and representation</li><li>Memorization depends on training dynamics, not parameter count</li><li>Reasoning gains depend on how distributions are shaped, not just optimized</li></ul>For builders, the message is clear: Competitive advantage is shifting from “who has the biggest model” to “who understands the system.”Maitreyi Chatterjee is a software engineer.Devansh Agarwal currently works as an ML engineer at FAANG.

في NeurIPS 2025، ظهرت أوراق بحثية مهمة تتحدى المعتقدات الراسخة في الذكاء الاصطناعي، مشيرة إلى أن التقدم يعتمد الآن أكثر على الهندسة المعمارية واستراتيجيات التقييم بدلاً من مجرد زيادة حجم النموذج. تشير هذه الأفكار إلى تحول في كيفية تطوير وتقييم أنظمة الذكاء الاصطناعي.

En NeurIPS 2025, surgieron artículos significativos que desafían creencias arraigadas en la IA, sugiriendo que los avances dependen más de la arquitectura y las estrategias de evaluación que del simple aumento del tamaño del modelo. Estas ideas indican un cambio en la forma en que se desarrollan y evalúan los sistemas de IA.

Lors de NeurIPS 2025, des articles significatifs ont remis en question des croyances bien ancrées en IA, suggérant que les avancées dépendent désormais davantage de l'architecture et des stratégies d'évaluation que de l'augmentation de la taille des modèles. Ces idées indiquent un changement dans la manière dont les systèmes d'IA sont développés et évalués.

At NeurIPS 2025, significant papers emerged that challenge long-held beliefs in AI, suggesting that advancements are now more dependent on architecture and evaluation strategies rather than merely increasing model size. These insights indicate a shift in how AI systems are developed and assessed.

Why reinforcement learning plateaus without representation depth (and other key takeaways from NeurIPS 2025)

The German AI startup Black Forest Labs (BFL), <a href="https://venturebeat.com/ai/stable-diffusion-creators-launch-black-forest-labs-secure-31m-for-flux-1-ai-image-generator">founded by former Stability AI engineers</a>, is continuing to build out its suite of open source AI image generators with the release of <a href="https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligence">FLUX.2 [klein]</a>, a new pair of small models — one open and one non-commercial — that emphasizes speed and lower compute requirements, with the models generating images in less than a second on a Nvidia GB200. The [klein] series, released yesterday, includes two primary parameter counts: 4 billion (4B) and 9 billion (9B).The model weights are available on <a href="https://huggingface.co/collections/black-forest-labs/flux2">Hugging Face</a> and code on <a href="https://github.com/black-forest-labs/flux2?tab=readme-ov-file">Github</a>.While the larger models in the FLUX.2 family ([max] and [pro]), <a href="https://venturebeat.com/ai/black-forest-labs-launches-flux-2-ai-image-models-to-challenge-nano-banana">released in November of 2025</a>, chase the limits of photorealism and &quot;grounding search&quot; capabilities, [klein] is designed specifically for consumer hardware and latency-critical workflows.In great news for enterprises, the 4B version is available under an Apache 2.0 license, meaning they — or any organization or developer — can use the [klein] models for their commercial purposes without paying BFL or any intermediaries a dime. However, a number of AI image and media creation platforms including <a href="https://fal.ai/models/fal-ai/flux-2/klein/4b/base">Fal.ai</a> have begun offering it for extremely low cost as well through their application programming interfaces (APIs) and as a direct-to-user tool. Already, it&#x27;s won strong praise from early users for its speed. What it lacks for in overall image quality, it seems to make up for in its fast generation capability, open license, affordability and small footprint — benefitting enterprises who want to run image models on their own hardware or at extremely low cost. So how did BFL do it and how can it benefit you? Read on to learn more.<h2>The &quot;Pareto Frontier&quot; of Latency</h2>The technical philosophy behind [klein] is what BFL documentation describes as defining the &quot;Pareto frontier&quot; for quality versus latency. In simple terms, they have attempted to squeeze the maximum possible visual fidelity into a model small enough to run on a home gaming PC without a noticeable lag.The performance metrics released by the company paint a picture of a model built for interactivity rather than just batch generation. According to Black Forest Labs&#x27; official figures, the [klein] models are capable of generating or editing images in under 0.5 seconds on modern hardware. Even on standard consumer GPUs like an RTX 3090 or 4070, the 4B model is designed to fit comfortably within approximately 13GB of VRAM.This speed is achieved through &quot;distillation,&quot; a process where a larger, more complex model &quot;teaches&quot; a smaller, more efficient one to approximate its outputs in fewer steps. The distilled [klein] variants require only four steps to generate an image. This effectively turns the generation process from a coffee-break task into a near-instantaneous one, enabling what BFL describes on X (formerly Twitter) as &quot;developing ideas from 0 → 1&quot; in real-time.<h2>Under the Hood: Unified Architecture</h2>Historically, image generation and image editing have often required different pipelines or complex adapters (like ControlNets). FLUX.2 [klein] attempts to unify these. The architecture natively supports text-to-image, single-reference editing, and multi-reference composition without needing to swap models.According to the documentation released on GitHub, the models support:<ul><li>Multi-Reference Editing: Users can upload up to four reference images (or ten in the playground) to guide the style or structure of the output.</li><li>Hex-Code Color Control: A frequent pain point for designers is getting &quot;that exact shade of red.&quot; The new models accept specific hex codes in prompts (e.g., #800020) to force precise color rendering.</li><li>Structured Prompting: The model parses JSON-like structured inputs for rigorously defined compositions, a feature clearly aimed at programmatic generation and enterprise pipelines.</li></ul><h3>The Licensing Split: Open Weights vs. Open Source</h3>For startups and developers building on top of BFL’s tech, understanding the licensing landscape of this release is critical. BFL has adopted a split strategy that separates &quot;hobbyist/research&quot; use from &quot;commercial infrastructure.&quot;<ol><li>FLUX.2 [klein] 4B: Released under Apache 2.0. This is a permissive free software license that allows for commercial use, modification, and redistribution. If you are building a paid app, a SaaS platform, or a game that integrates AI generation, you can use the 4B model royalty-free.</li><li>FLUX.2 [klein] 9B &amp; [dev]: Released under the FLUX Non-Commercial License. These weights are open for researchers and hobbyists to download and experiment with, but they cannot be used for commercial applications without a separate agreement.</li></ol>This distinction positions the 4B model as a direct competitor to other open-weights models like Stable Diffusion 3 Medium or SDXL, but with a more modern architecture and a permissive license that removes legal ambiguity for startups.<h3>Ecosystem Integration: ComfyUI and Beyond</h3>BFL is clearly aware that a model is only as good as the tools that run it. Coinciding with the model drop, the team released official workflow templates for <a href="https://www.comfy.org/">ComfyUI</a>, the node-based interface that has become the standard integrated development environment (IDE) for AI artists.The workflows—specifically <code>image_flux2_klein_text_to_image.json</code> and the editing variants—allow users to drag and drop the new capabilities into existing pipelines immediately.Community reaction on social media has centered on this workflow integration and the speed. In a post on X, the official Black Forest Labs account highlighted the model&#x27;s ability to &quot;rapidly explore a specific aesthetic,&quot; showcasing a video where the style of an image shifted instantly as the user scrubbed through options.<h3>Why It Matters For Enterprise AI Decision-Makers</h3>The release of FLUX.2 [klein] signals a maturation in the generative AI market, moving past the initial phase of novelty into a period defined by utility, integration, and speed.For Lead AI Engineers who are constantly juggling the need to balance speed with quality, this shift is pivotal. These professionals, who manage the full lifecycle of models from data preparation to deployment, often face the daily challenge of integrating rapidly evolving tools into existing workflows. The availability of a distilled 4B model under an Apache 2.0 license offers a practical solution for those focused on rapid deployment and fine-tuning to achieve specific business goals, allowing them to bypass the latency bottlenecks that typically plague high-fidelity image generation.For Senior AI Engineers focused on orchestration and automation, the implications are equally significant. These experts are responsible for building scalable AI pipelines and maintaining model integrity across different environments, often while working under strict budget constraints.The lightweight nature of the [klein] family directly addresses the challenge of implementing efficient systems with limited resources. By utilizing a model that fits within consumer-grade VRAM, orchestration specialists can architect cost-effective, local inference pipelines that avoid the heavy operational costs associated with massive proprietary models.Even for the Director of IT Security, the move toward capable, locally runnable open-weight models offers a distinct advantage. Tasked with protecting the organization from cyber threats and managing security operations with limited resources, reliance on external APIs for sensitive creative workflows can be a vulnerability. A high-quality model that runs locally allows security leaders to sanction AI tools that keep proprietary data within the corporate firewall, balancing the operational demands of the business with the robust security measures they are required to uphold.

أطلقت شركة بلاك فوريست لابز (BFL)، وهي شركة ناشئة ألمانية في مجال الذكاء الاصطناعي أسسها مهندسون سابقون في Stability AI، FLUX.2 [klein]، وهي مجموعة جديدة من مولدات الصور بالذكاء الاصطناعي مفتوحة المصدر التي يمكنها إنتاج الصور في أقل من ثانية على أجهزة Nvidia. يتضمن الإطلاق نموذجين بعدد معلمات يبلغ 4 مليارات و9 مليارات، متاحة على Hugging Face وGitHub.

Black Forest Labs (BFL), una startup alemana de IA fundada por exingenieros de Stability AI, ha lanzado FLUX.2 [klein], una nueva suite de generadores de imágenes de IA de código abierto que pueden producir imágenes en menos de un segundo en hardware de Nvidia. El lanzamiento incluye dos modelos con recuentos de parámetros de 4 mil millones y 9 mil millones, disponibles en Hugging Face y GitHub.

Black Forest Labs (BFL), une startup allemande d'IA fondée par d'anciens ingénieurs de Stability AI, a lancé FLUX.2 [klein], une nouvelle suite de générateurs d'images IA open-source capable de produire des images en moins d'une seconde sur du matériel Nvidia. Le lancement comprend deux modèles avec des comptes de paramètres de 4 milliards et 9 milliards, disponibles sur Hugging Face et GitHub.

Black Forest Labs (BFL), a German AI startup founded by former Stability AI engineers, has launched FLUX.2 [klein], a new suite of open-source AI image generators that can produce images in less than a second on Nvidia hardware. The release includes two models with parameter counts of 4 billion and 9 billion, available on Hugging Face and GitHub.

Black Forest Labs launches open source Flux.2 [klein] to generate AI images in less than a second

Researchers at Google have developed a technique that makes it easier for AI models to learn complex reasoning tasks that usually cause LLMs to hallucinate or fall apart. Instead of training LLMs through next-token prediction, their technique, called <a href="https://arxiv.org/abs/2512.20605">internal reinforcement learning</a> (internal RL), steers the model’s internal activations toward developing a high-level step-by-step solution for the input problem. Ultimately, this could provide a scalable path for creating autonomous agents that can handle complex reasoning and real-world robotics without needing constant, manual guidance.<h2>The limits of next-token prediction</h2><a href="https://venturebeat.com/infrastructure/deepseek-r1s-bold-bet-on-reinforcement-learning-how-it-outpaced-openai-at-3-of-the-cost">Reinforcement learning</a> plays a key role in post-training LLMs, particularly for complex reasoning tasks that require long-horizon planning. However, the problem lies in the architecture of these models. LLMs are autoregressive, meaning they generate sequences one token at a time. When these models explore new strategies during training, they do so by making small, random changes to the next single token or action. This exposes a deeper limitation: next-token prediction forces models to search for solutions at the wrong level of abstraction, making long-horizon reasoning inefficient even when the model “knows” what to do.This token-by-token approach works well for basic language modeling but breaks down in long-horizon tasks where rewards are sparse. If the model relies solely on random token-level sampling, the probability of stumbling upon the correct multi-step solution is infinitesimally small, &quot;on the order of one in a million,&quot; according to the researchers.The issue isn&#x27;t just that the models get confused; it’s that they get confused at the wrong level. In comments provided to VentureBeat, Yanick Schimpf, a co-author of the paper, notes that in a 20-step task, an agent can get lost in the minute details of a single step, or it can lose track of the overall goal.&quot;We argue that when facing a problem with some abstract structure... [goal-oriented exploration] is what you want,&quot; Schimpf said. By solving the problem at the abstract level first, the agent commits to a path, ensuring it doesn&#x27;t &quot;get lost in one of the reasoning steps&quot; and fail to complete the broader workflow.To address this, the field has long looked toward hierarchical reinforcement learning. HRL attempts to solve complex problems by decomposing them into a hierarchy of temporally abstract actions (high-level subroutines that represent different stages of the solution) rather than managing a task as a string of tokens. However, discovering these appropriate subroutines remains a longstanding challenge. Current HRL methods often fail to discover proper policies, frequently &quot;converging to degenerate options&quot; that do not represent meaningful behaviors. Even sophisticated modern methods like GRPO (a popular RL algorithm used for sparse-reward tasks) fail in complex environments because they cannot effectively bridge the gap between low-level execution and high-level planning. <h2>Steering the LLM&#x27;s internal thoughts</h2>To overcome these limitations, the Google team proposed internal RL. Advanced autoregressive models already &quot;know&quot; how to perform complex, multi-step tasks internally, even if they aren&#x27;t explicitly trained to do so. Because these complex behaviors are hidden inside the model&#x27;s residual stream (i.e., the numerical values that carry information through the network&#x27;s layers), the researchers introduced an &quot;internal neural network controller,&quot; or metacontroller. Instead of monitoring and changing the output token, the metacontroller controls the model’s behavior by applying changes to the model&#x27;s internal activations in the middle layers.This nudge steers the model into a specific useful state. The base model then automatically generates the sequence of individual steps needed to achieve that goal because it has already seen those patterns during its initial pretraining. The metacontroller operates through unsupervised learning and does not require human-labeled training examples. Instead, the researchers use a self-supervised framework where the model analyzes a full sequence of behavior and works backward to infer the hidden, high-level intent that best explains the actions.During the internal RL phase, the updates are applied to the metacontroller, which shifts training from next-token prediction to learning high-level actions that can lead to the solution.To understand the practical value of this, consider an enterprise agent tasked with code generation. Today, there is a difficult trade-off: You need &quot;low temperature&quot; (predictability) to get the syntax right, but &quot;high temperature&quot; (creativity) to solve the logic puzzle.&quot;Internal RL might facilitate this by allowing the model to explore the space of abstract actions, i.e. structuring logic and method calls, while delegating the token-level realization of those actions to the robust, lower-temperature distribution of the base model,&quot; Schimpf said. The agent explores the solution without breaking the syntax.The researchers investigated two methods for applying this controller. In the first, the base autoregressive model is pretrained on a behavioral dataset and then frozen, while the metacontroller is trained to steer the frozen model&#x27;s residual stream. In the second, the metacontroller and the base model are jointly optimized, with parameters of both networks updated simultaneously. <h2>Internal RL in action</h2>To evaluate the effectiveness of internal RL, the researchers ran experiments across hierarchical environments designed to stump traditional learners. These included a discrete grid world and a continuous control task where a quadrupedal &quot;ant&quot; robot must coordinate joint movements. Both environments used sparse rewards with very long action sequences.While baselines like GRPO and CompILE failed to learn the tasks within a million episodes due to the difficulty of credit assignment over long horizons, internal RL achieved high success rates with a small number of training episodes. By choosing high-level goals rather than tiny steps, the metacontroller drastically reduced the search space. This allowed the model to identify which high-level decisions led to success, making credit assignment efficient enough to solve the sparse reward problem.Notably, the researchers found that the &quot;frozen&quot; approach was superior. When the base model and metacontroller were co-trained from scratch, the system failed to develop meaningful abstractions. However, applied to a frozen model, the metacontroller successfully discovered key checkpoints without any human labels, perfectly aligning its internal switching mechanism with the ground-truth moments when an agent finished one subgoal and started the next.As the industry currently fixates on reasoning models that output verbose &quot;chains of thought&quot; to solve problems, Google’s research points toward a different, perhaps more efficient future.&quot;Our study joins a growing body of work suggesting that &#x27;internal reasoning&#x27; is not only feasible but potentially more efficient than token-based approaches,&quot; Schimpf said. &quot;Moreover, these silent &#x27;thoughts&#x27; can be decoupled from specific input modalities — a property that could be particularly relevant for the future of multi-modal AI.&quot;If internal reasoning can be guided without being externalized, the future of AI agents may hinge less on prompting strategies and more on how well we can access and steer what models already represent internally. For enterprises betting on autonomous systems that must plan, adapt, and act over long horizons, that shift could matter more than any new reasoning benchmark.

طور الباحثون في جوجل تقنية تُعرف باسم التعلم المعزز الداخلي (internal RL)، والتي تعزز قدرة نماذج الذكاء الاصطناعي على تعلم مهام التفكير المعقدة، مما يقلل من حالات الهلوسة التي تُلاحظ عادةً في نماذج اللغة الكبيرة (LLMs). هذه الطريقة تحول التركيز من التنبؤ بالرمز التالي إلى توجيه العمليات الداخلية للنموذج نحو حل منظم للمشكلات.

Investigadores de Google han desarrollado una técnica conocida como aprendizaje por refuerzo interno (internal RL), que mejora la capacidad de los modelos de IA para aprender tareas de razonamiento complejas, reduciendo los casos de alucinaciones que suelen observarse en los modelos de lenguaje grandes (LLMs). Este método cambia el enfoque de la predicción del siguiente token a guiar los procesos internos del modelo hacia una solución estructurada de problemas.

Des chercheurs de Google ont développé une technique appelée apprentissage par renforcement interne (internal RL), qui améliore la capacité des modèles d'IA à apprendre des tâches de raisonnement complexes, réduisant les cas d'hallucination généralement observés dans les grands modèles de langage (LLMs). Cette méthode déplace l'accent de la prédiction du prochain jeton vers l'orientation des processus internes du modèle vers une résolution de problèmes structurée.

Researchers at Google have developed a technique known as internal reinforcement learning (internal RL), which enhances the ability of AI models to learn complex reasoning tasks, reducing instances of hallucination typically seen in large language models (LLMs). This method shifts the focus from next-token prediction to guiding the model's internal processes towards structured problem-solving.