arXiv:2605.09986v2 Announce Type: replace 
Abstract: Training a language model on data scattered across bandwidth-limited nodes that cannot be centralized is a setting that arises in clinical networks, enterprise knowledge bases, and scientific consortia. We study the regime in which data must remain distributed across nodes, and ask what statistical guarantees are in principle achievable under explicit bandwidth budgets; we aim to characterize what is provably possible, not to demonstrate a deployment-ready system. Existing theory treats either training-time consistency or inference-time calibration in isolation, and no prior work makes bandwidth a first-class statistical parameter. We analyze two protocols, Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for inference, as the analytical vehicles for our results. Our first main result is an explicit high-probability KL-consistency rate for FPLD with simultaneous dependence on node count $K$, per-node sample size $n$, quantization budget $B$, probe-set size $m$, and vocabulary size $V$; bandwidth enters only through an exponentially vanishing quantization term. Our second main result is a distribution-free marginal-coverage bound for FC-RAG, whose novel retrieval-bandwidth slack $\Delta_{\mathrm{RAG}} = f_{\max}\sqrt{K^{-2}\sum_i v(B_i)}$ makes per-node retrieval bandwidth a first-class statistical parameter, with arithmetic aggregation across $K$ nodes shrinking the slack as $K^{-1/2}$ in the per-node-uniform regime. A Pinsker-type corollary composes the two bounds into an end-to-end coverage guarantee. Synthetic experiments verify the predicted scaling along the bounds' parameters; small-scale experiments on a GPT-2 testbed illustrate that the qualitative bandwidth-accuracy tradeoff survives on a real language model. A deployment-scale empirical evaluation is out of scope.

دراسة حديثة بعنوان 'نماذج اللغة الفيدرالية تحت ميزانيات عرض النطاق الترددي: معدلات التقطير والتغطية المطابقة' تفحص تدريب نماذج اللغة عبر عقد محدودة النطاق الترددي، خاصة في الشبكات السريرية وقواعد المعرفة المؤسسية. تركز الدراسة على الضمانات الإحصائية الممكن تحقيقها تحت قيود عرض النطاق الترددي، مقدمة بروتوكولات للتدريب والاستدلال تعطي الأولوية للبيانات الموزعة.

Un estudio reciente titulado 'Modelos de Lenguaje Federados Bajo Presupuestos de Ancho de Banda: Tasas de Destilación y Cobertura Conformal' examina el entrenamiento de modelos de lenguaje a través de nodos con ancho de banda limitado, especialmente en entornos clínicos y empresariales. La investigación se centra en las garantías estadísticas alcanzables bajo restricciones de ancho de banda, introduciendo protocolos para el entrenamiento y la inferencia que priorizan los datos distribuidos.

Une étude récente intitulée 'Modèles de Langue Fédérés Sous Budgets de Bande Passante : Taux de Distillation et Couverture Conformale' examine l'entraînement de modèles de langue à travers des nœuds à bande passante limitée, en particulier dans des contextes cliniques et d'entreprise. La recherche se concentre sur les garanties statistiques réalisables sous des contraintes de bande passante, introduisant des protocoles pour l'entraînement et l'inférence qui privilégient les données distribuées.

A recent study titled 'Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage' examines the training of language models across bandwidth-limited nodes, particularly in clinical and enterprise settings. The research focuses on the statistical guarantees achievable under bandwidth constraints, introducing protocols for training and inference that prioritize distributed data.