An Efficient Gradient-Based Inference Attack for Federated Learning

A new framework named TrajSyn has been introduced to facilitate privacy-preserving dataset distillation from federated model trajectories, enabling effective server-side adversarial training without accessing raw client data. This innovation addresses the challenges posed by adversarial perturbations in deep learning models deployed on edge devices, particularly in Federated Learning settings where data privacy is paramount.

A new framework called Selective Subnetwork Distillation (SSD) has been proposed to enhance continual learning in sparse neural systems, specifically addressing the limitations of Sparse Distributed Memory Multi-Layer Perceptrons (SDMLP). SSD enables the identification and distillation of knowledge from high-activation neurons without relying on task labels or replay, thus preserving modularity while allowing for structural realignment.

A systematic study has been conducted on the privacy-utility relationship in post-training quantization (PTQ) of deep neural networks, focusing on three algorithms: AdaRound, BRECQ, and OBC. The research reveals that low-precision PTQs, specifically at 4-bit, 2-bit, and 1.58-bit levels, can significantly reduce privacy leakage while maintaining model performance across datasets like CIFAR-10, CIFAR-100, and TinyImageNet.

A new framework addressing Data Reconstruction Attacks (DRA) in Federated Learning (FL) systems has been introduced, focusing on quantifying the risk associated with these attacks through a metric called Invertibility Loss (InvLoss). This framework aims to provide a theoretical basis for understanding and mitigating the risks posed by adversaries who can infer sensitive training data from local clients.

A new study presents REAL (Representation Enhanced Analytic Learning), a method designed to improve exemplar-free class-incremental learning (EFCIL) by addressing issues of representation and knowledge utilization in existing analytic continual learning frameworks. REAL employs a dual-stream pretraining approach followed by a representation-enhancing distillation process to create a more effective classifier during class-incremental learning.

A new one-cycle structured pruning framework has been proposed, integrating pre-training, pruning, and fine-tuning into a single training cycle, which aims to enhance efficiency while maintaining accuracy. This method identifies an optimal sub-network early in the training process, utilizing norm-based group saliency criteria and structured sparsity regularization to improve performance.