Task-Aware Multi-Expert Architecture For Lifelong Deep Learning

A new gradient-based membership inference attack for federated learning has been introduced, leveraging the temporal evolution of last-layer gradients across multiple federated rounds. This method does not require access to private datasets and is designed to address both semi-honest and malicious adversaries, expanding the scope of potential data leaks in federated learning scenarios.

A new framework called Selective Subnetwork Distillation (SSD) has been proposed to enhance continual learning in sparse neural systems, specifically addressing the limitations of Sparse Distributed Memory Multi-Layer Perceptrons (SDMLP). SSD enables the identification and distillation of knowledge from high-activation neurons without relying on task labels or replay, thus preserving modularity while allowing for structural realignment.

A systematic study has been conducted on the privacy-utility relationship in post-training quantization (PTQ) of deep neural networks, focusing on three algorithms: AdaRound, BRECQ, and OBC. The research reveals that low-precision PTQs, specifically at 4-bit, 2-bit, and 1.58-bit levels, can significantly reduce privacy leakage while maintaining model performance across datasets like CIFAR-10, CIFAR-100, and TinyImageNet.

A new study presents REAL (Representation Enhanced Analytic Learning), a method designed to improve exemplar-free class-incremental learning (EFCIL) by addressing issues of representation and knowledge utilization in existing analytic continual learning frameworks. REAL employs a dual-stream pretraining approach followed by a representation-enhancing distillation process to create a more effective classifier during class-incremental learning.

A new one-cycle structured pruning framework has been proposed, integrating pre-training, pruning, and fine-tuning into a single training cycle, which aims to enhance efficiency while maintaining accuracy. This method identifies an optimal sub-network early in the training process, utilizing norm-based group saliency criteria and structured sparsity regularization to improve performance.