arXiv:2511.16523v1 Announce Type: new 
Abstract: Federated learning (FL) enables clients to collaboratively train a shared model in a distributed manner, setting it apart from traditional deep learning paradigms. However, most existing FL research assumes consistent client participation, overlooking the practical scenario of dynamic participation (DPFL), where clients may intermittently join or leave during training. Moreover, no existing benchmarking framework systematically supports the study of DPFL-specific challenges. In this work, we present the first open-source framework explicitly designed for benchmarking FL models under dynamic client participation. Our framework provides configurable data distributions, participation patterns, and evaluation metrics tailored to DPFL scenarios. Using this platform, we benchmark four major categories of widely adopted FL models and uncover substantial performance degradation under dynamic participation. To address these challenges, we further propose Knowledge-Pool Federated Learning (KPFL), a generic plugin that maintains a shared knowledge pool across both active and idle clients. KPFL leverages dual-age and data-bias weighting, combined with generative knowledge distillation, to mitigate instability and prevent knowledge loss. Extensive experiments demonstrate the significant impact of dynamic participation on FL performance and the effectiveness of KPFL in improving model robustness and generalization.

يتيح التعلم الفيدرالي (FL) للعملاء تدريب نموذج مشترك بشكل تعاوني بطريقة موزعة، مما يميزه عن paradigms التعلم العميق التقليدية. تقدم هذه الدراسة إطار عمل مفتوح المصدر لتقييم نماذج FL تحت مشاركة العملاء الديناميكية (DPFL)، حيث تتناول التحديات المتعلقة بانضمام العملاء أو مغادرتهم بشكل متقطع أثناء التدريب. يوفر الإطار توزيعات بيانات قابلة للتكوين ومقاييس تقييم، مما يكشف عن تدهور كبير في الأداء تحت ظروف DPFL.

El aprendizaje federado (FL) permite a los clientes entrenar colaborativamente un modelo compartido de manera distribuida, diferenciándose de los paradigmas tradicionales de aprendizaje profundo. Esta investigación presenta un nuevo marco de código abierto para evaluar modelos de FL bajo participación dinámica de clientes (DPFL), abordando los desafíos de la participación intermitente de los clientes durante el entrenamiento. El marco ofrece distribuciones de datos configurables y métricas de evaluación, revelando una degradación significativa del rendimiento en modelos de FL bajo condiciones …

L'apprentissage fédéré (FL) permet aux clients de former collaborativement un modèle partagé de manière distribuée, se distinguant de l'apprentissage profond traditionnel. Cette recherche présente un nouveau cadre open-source pour évaluer les modèles FL sous participation dynamique des clients (DPFL), abordant les défis liés à l'entrée et à la sortie intermittentes des clients pendant la formation. Le cadre offre des distributions de données configurables et des métriques d'évaluation, révélant une dégradation significative des performances des modèles FL dans des conditions DPFL.

Federated learning (FL) allows clients to collaboratively train a shared model in a distributed manner, differing from traditional deep learning. This research introduces a new open-source framework for benchmarking FL models under dynamic client participation (DPFL), addressing the challenges of clients intermittently joining or leaving during training. The framework offers configurable data distributions and evaluation metrics, revealing significant performance degradation in FL models under DPFL conditions.

Dynamic Participation in Federated Learning: Benchmarks and a Knowledge Pool Plugin

arXiv:2511.16069v1 Announce Type: new 
Abstract: Federated Learning with Low-Rank Adaptation (LoRA) faces three critical challenges under client heterogeneity: (1) Initialization-Induced Instability due to random initialization misaligning client subspaces; (2) Rank Incompatibility and Aggregation Error when averaging LoRA parameters of different ranks, which biases the global model; and (3) exacerbated Client Drift under Non-IID Data, impairing generalization. To address these challenges, we propose ILoRA, a unified framework that integrates three core innovations: a QR-based orthonormal initialization to ensure all clients start in a coherent subspace; a Concatenated QR Aggregation mechanism that fuses heterogeneous-rank updates via concatenation and decomposition, preserving information while maintaining dimension alignment; and an AdamW optimizer with rank-aware control variates to correct local updates and mitigate client drift. Supported by theoretical convergence guarantees, extensive experiments on vision and NLP benchmarks demonstrate that ILoRA consistently achieves superior accuracy and convergence stability compared to existing federated LoRA methods.

ILoRA، أو التعلم الفيدرالي مع التكيف منخفض الرتبة، يتناول ثلاثة تحديات رئيسية تتعلق بتنوع العملاء: عدم الاستقرار الناتج عن التهيئة، عدم توافق الرتبة، وانحراف العملاء تحت بيانات غير IID. الإطار المقترح يدمج تهيئة قائمة على QR، وآلية تجميع QR متصلة، ومُحسّن AdamW مع متغيرات تحكم مدركة للرتبة. تهدف هذه الابتكارات إلى تحسين الاستقرار والأداء لنماذج التعلم الفيدرالي عبر بيئات العملاء المتنوعة.

ILoRA, o Aprendizaje Federado con Adaptación de Bajo Rango, aborda tres desafíos significativos en la heterogeneidad de los clientes: inestabilidad en la inicialización, incompatibilidad de rango y deriva del cliente bajo datos no IID. El marco propuesto integra una inicialización basada en QR, un mecanismo de agregación QR concatenado y un optimizador AdamW con variables de control conscientes del rango. Estas innovaciones buscan mejorar la estabilidad y el rendimiento de los modelos de aprendizaje federado en entornos de clientes diversos.

ILoRA, ou apprentissage fédéré avec adaptation à faible rang, s'attaque à trois défis majeurs liés à l'hétérogénéité des clients : l'instabilité d'initialisation, l'incompatibilité de rang et la dérive des clients sous des données non IID. Le cadre proposé intègre une initialisation basée sur QR, un mécanisme d'agrégation QR concaténé et un optimiseur AdamW avec des variables de contrôle sensibles au rang. Ces innovations visent à améliorer la stabilité et les performances des modèles d'apprentissage fédéré dans des environnements clients divers.

ILoRA, or Federated Learning with Low-Rank Adaptation, addresses three significant challenges in client heterogeneity: initialization instability, rank incompatibility, and client drift under non-IID data. The proposed framework integrates a QR-based initialization, a concatenated QR aggregation mechanism, and an AdamW optimizer with rank-aware control variates. These innovations aim to enhance the stability and performance of federated learning models across diverse client environments.

Dynamic Participation in Federated Learning: Benchmarks and a Knowledge Pool Plugin

Was this article worth reading? Share it