arXiv:2511.12695v1 Announce Type: cross 
Abstract: Federated Learning (FL) enables decentralized, privacy-preserving model training but struggles to balance global generalization and local personalization due to non-identical data distributions across clients. Personalized Fine-Tuning (PFT), a popular post-hoc solution, fine-tunes the final global model locally but often overfits to skewed client distributions or fails under domain shifts. We propose adapting Linear Probing followed by full Fine-Tuning (LP-FT), a principled centralized strategy for alleviating feature distortion (Kumar et al., 2022), to the FL setting. Through systematic evaluation across seven datasets and six PFT variants, we demonstrate LP-FT's superiority in balancing personalization and generalization. Our analysis uncovers federated feature distortion, a phenomenon where local fine-tuning destabilizes globally learned features, and theoretically characterizes how LP-FT mitigates this via phased parameter updates. We further establish conditions (e.g., partial feature overlap, covariate-concept shift) under which LP-FT outperforms standard fine-tuning, offering actionable guidelines for deploying robust personalization in FL.

يتيح التعلم الفيدرالي (FL) تدريب النماذج بشكل لامركزي مع الحفاظ على الخصوصية، لكنه يواجه تحديات في تحقيق التوازن بين التعميم العالمي والتخصيص المحلي بسبب اختلاف توزيعات البيانات بين العملاء. يُعتبر التخصيص الدقيق (PFT) طريقة شائعة بعدية تعدل النموذج العالمي محليًا، لكنها قد تؤدي إلى الإفراط في التكيف أو ضعف الأداء في ظل تغيرات المجال. تهدف الطريقة المقترحة، وهي الاستدلال الخطي متبوعًا بالتعديل الكامل (LP-FT)، إلى معالجة هذه المشكلات من خلال تقييم فعاليتها بشكل منهجي عبر مجموعات بيانات متعددة ونسخ PFT.

El Aprendizaje Federado (FL) permite el entrenamiento descentralizado de modelos mientras preserva la privacidad, pero enfrenta desafíos para equilibrar la generalización global y la personalización local debido a las distribuciones de datos variadas entre los clientes. La Personalización Fina (PFT) es un método post-hoc común que ajusta el modelo global localmente, pero puede llevar a un sobreajuste o un rendimiento deficiente ante cambios de dominio. El método propuesto, Probing Lineal seguido de un Ajuste Completo (LP-FT), busca abordar estos problemas evaluando sistemáticamente su efectivi…

L'apprentissage fédéré (FL) permet un entraînement décentralisé des modèles tout en préservant la vie privée, mais il rencontre des difficultés pour équilibrer la généralisation globale et la personnalisation locale en raison des distributions de données variées parmi les clients. La personnalisation fine (PFT) est une méthode post-hoc courante qui ajuste le modèle global localement, mais peut entraîner un surajustement ou de mauvaises performances en cas de changements de domaine. La méthode proposée, le Linear Probing suivi d'un Fine-Tuning complet (LP-FT), vise à résoudre ces problèmes en é…

Federated Learning (FL) allows for decentralized model training while preserving privacy, yet it faces challenges in achieving a balance between global generalization and local personalization due to varying data distributions among clients. Personalized Fine-Tuning (PFT) is a common post-hoc method that adjusts the global model locally but can lead to overfitting or poor performance under domain shifts. The proposed method, Linear Probing followed by full Fine-Tuning (LP-FT), aims to address these issues by systematically evaluating its effectiveness across multiple datasets and PFT variants.

A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning

arXiv:2601.08216v1 Announce Type: new 
Abstract: Federated learning protocols require repeated synchronization between clients and a central server, with convergence rates depending on learning rates, data heterogeneity, and client sampling. This paper asks whether iterative communication is necessary for distributed linear regression. We show it is not. We formulate federated ridge regression as a distributed equilibrium problem where each client computes local sufficient statistics -- the Gram matrix and moment vector -- and transmits them once. The server reconstructs the global solution through a single matrix inversion. We prove exact recovery: under a coverage condition on client feature matrices, one-shot aggregation yields the centralized ridge solution, not an approximation. For heterogeneous distributions violating coverage, we derive non-asymptotic error bounds depending on spectral properties of the aggregated Gram matrix. Communication reduces from $\mathcal{O}(Rd)$ in iterative methods to $\mathcal{O}(d^2)$ total; for high-dimensional settings, we propose and experimentally validate random projection techniques reducing this to $\mathcal{O}(m^2)$ where $m \ll d$. We establish differential privacy guarantees where noise is injected once per client, eliminating the composition penalty that degrades privacy in multi-round protocols. We further address practical considerations including client dropout robustness, federated cross-validation for hyperparameter selection, and comparison with gradient-based alternatives. Comprehensive experiments on synthetic heterogeneous regression demonstrate that one-shot fusion matches FedAvg accuracy while requiring up to $38\times$ less communication. The framework applies to kernel methods and random feature models but not to general nonlinear architectures.

أظهرت دراسة حديثة نهجًا مبتكرًا في الانحدار المتحد الفيدرالي، حيث أثبتت أن التواصل التكراري بين العملاء والخادم المركزي ليس ضروريًا لتحقيق الاسترداد الدقيق للحل المركزي. من خلال تجميع الإحصائيات الكافية من العملاء في إرسال واحد، يمكن للخادم إعادة بناء الحل العالمي من خلال عكس المصفوفة، مما يقلل بشكل كبير من عبء التواصل.

Un estudio reciente presenta un enfoque novedoso para la regresión ridge federada, demostrando que la comunicación iterativa entre los clientes y un servidor central no es necesaria para lograr la recuperación exacta de la solución centralizada. Al agregar estadísticas suficientes de los clientes en una sola transmisión, el servidor puede reconstruir la solución global mediante la inversión de matrices, reduciendo significativamente la sobrecarga de comunicación.

Une étude récente présente une approche novatrice de la régression ridge fédérée, démontrant que la communication itérative entre les clients et un serveur central n'est pas nécessaire pour obtenir une récupération exacte de la solution centralisée. En agrégeant les statistiques suffisantes des clients en une seule transmission, le serveur peut reconstruire la solution globale par inversion de matrice, réduisant ainsi considérablement la surcharge de communication.

A recent study introduces a novel approach to federated ridge regression, demonstrating that iterative communication between clients and a central server is unnecessary for achieving exact recovery of the centralized solution. By aggregating sufficient statistics from clients in a single transmission, the server can reconstruct the global solution through matrix inversion, significantly reducing communication overhead.

One-Shot Federated Ridge Regression: Exact Recovery via Sufficient Statistic Aggregation

arXiv:2311.12715v3 Announce Type: replace 
Abstract: Federated Learning is an important emerging distributed training paradigm that keeps data private on clients. It is now well understood that by controlling only a small subset of FL clients, it is possible to introduce a backdoor to a federated learning model, in the presence of certain attributes. In this paper, we present a new type of attack that compromises the fairness of the trained model. Fairness is understood to be the attribute-level performance distribution of a trained model. It is particularly salient in domains where, for example, skewed accuracy discrimination between subpopulations could have disastrous consequences. We find that by employing a threat model similar to that of a backdoor attack, an attacker is able to influence the aggregated model to have an unfair performance distribution between any given set of attributes. Furthermore, we find that this attack is possible by controlling only a single client. While combating naturally induced unfairness in FL has previously been discussed in depth, its artificially induced kind has been neglected. We show that defending against attacks on fairness should be a critical consideration in any situation where unfairness in a trained model could benefit a user who participated in its training.

تسلط الأبحاث الحديثة الضوء على نوع جديد من الهجمات على التعلم الفيدرالي (FL) الذي يهدد العدالة في النماذج المدربة، حيث يكشف أن السيطرة على عميل واحد فقط يمكن أن تؤثر على توزيع الأداء عبر سمات مختلفة. يثير هذا مخاوف بشأن نزاهة النماذج في التطبيقات الحساسة حيث تكون العدالة أمرًا حاسمًا.

Investigaciones recientes destacan un nuevo tipo de ataque en el Aprendizaje Federado (FL) que compromete la equidad de los modelos entrenados, revelando que controlar solo un cliente puede sesgar las distribuciones de rendimiento entre diversos atributos. Esto genera preocupaciones sobre la integridad de los modelos en aplicaciones sensibles donde la equidad es crítica.

Des recherches récentes mettent en lumière un nouveau type d'attaque sur l'apprentissage fédéré (FL) qui compromet l'équité des modèles entraînés, révélant que le contrôle d'un seul client peut fausser les distributions de performance entre divers attributs. Cela soulève des inquiétudes quant à l'intégrité des modèles dans des applications sensibles où l'équité est cruciale.

Recent research highlights a new type of attack on Federated Learning (FL) that compromises the fairness of trained models, revealing that controlling just one client can skew performance distributions across various attributes. This raises concerns about the integrity of models in sensitive applications where fairness is critical.

A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning

Was this article worth reading? Share it

LucidQuery AI

Hypertune

Octofy

DOO

Fyli

Project Feather

Ready to build your own newsroom?