arXiv:2606.12400v1 Announce Type: new 
Abstract: Long input sequences are central to document understanding and multi-step reasoning in Large Language Models, yet the quadratic cost of attention makes inference both memory-intensive and slow. Context distillation mitigates this by compressing contextual information into model parameters, and recent work such as Doc-to-LoRA amortizes context distillation into a single forward pass that generates one LoRA adapter per document. However, producing a single monolithic adapter for all queries leads to irrelevant-query interference, limited compositional recall, and poor scalability to long-document reasoning. To address these challenges, we propose Doc-to-Atom (Doc2Atom), a compositional parametric memory framework that decomposes each document into semantically typed knowledge atoms. Each atom is compiled into an independent micro-LoRA adapter and a provenance retrieval key. At inference time, a lightweight query router selects and assembles only the relevant atoms into a query-specific adapter, which is then injected into a frozen base model. The entire system is trained end-to-end through a multi-objective distillation framework. Experiments on six diverse QA benchmarks demonstrate that Doc2Atom outperforms Doc-to-LoRA baselines while reducing the memory cost of document internalization.

يقدم Doc-to-Atom (Doc2Atom) إطار ذاكرة بارامترية تركيبية جديدة تقوم بتفكيك الوثائق إلى ذرات معرفة ذات نوع دلالي، مما يعالج عدم الكفاءة في النماذج الحالية في التعامل مع تسلسلات الإدخال الطويلة في نماذج اللغة الكبيرة (LLMs). يقوم هذا الإطار بإنشاء محولات micro-LoRA مستقلة لكل ذرة، مما يعزز فهم الوثائق والتفكير متعدد الخطوات.

La introducción de Doc-to-Atom (Doc2Atom) presenta un nuevo marco de memoria paramétrica composicional que descompone documentos en átomos de conocimiento tipificados semánticamente, abordando las ineficiencias de los modelos existentes en el manejo de largas secuencias de entrada en los Modelos de Lenguaje Grande (LLMs). Este marco genera adaptadores micro-LoRA independientes para cada átomo, mejorando la comprensión de documentos y el razonamiento en múltiples pasos.

L'introduction de Doc-to-Atom (Doc2Atom) présente un nouveau cadre de mémoire paramétrique compositionnelle qui décompose les documents en atomes de connaissance typés sémantiquement, répondant aux inefficacités des modèles existants dans le traitement de longues séquences d'entrée dans les grands modèles de langage (LLM). Ce cadre génère des adaptateurs micro-LoRA indépendants pour chaque atome, améliorant la compréhension des documents et le raisonnement en plusieurs étapes.

The introduction of Doc-to-Atom (Doc2Atom) presents a novel compositional parametric memory framework that decomposes documents into semantically typed knowledge atoms, addressing the inefficiencies of existing models in handling long input sequences in Large Language Models (LLMs). This framework generates independent micro-LoRA adapters for each atom, enhancing document understanding and multi-step reasoning.