Bits for Privacy: Evaluating Post-Training Quantization via Membership Inference

A new gradient-based membership inference attack for federated learning has been introduced, leveraging the temporal evolution of last-layer gradients across multiple federated rounds. This method does not require access to private datasets and is designed to address both semi-honest and malicious adversaries, expanding the scope of potential data leaks in federated learning scenarios.

A new framework called Selective Subnetwork Distillation (SSD) has been proposed to enhance continual learning in sparse neural systems, specifically addressing the limitations of Sparse Distributed Memory Multi-Layer Perceptrons (SDMLP). SSD enables the identification and distillation of knowledge from high-activation neurons without relying on task labels or replay, thus preserving modularity while allowing for structural realignment.

A new study presents REAL (Representation Enhanced Analytic Learning), a method designed to improve exemplar-free class-incremental learning (EFCIL) by addressing issues of representation and knowledge utilization in existing analytic continual learning frameworks. REAL employs a dual-stream pretraining approach followed by a representation-enhancing distillation process to create a more effective classifier during class-incremental learning.

A new framework called Arithmetic-Intensity-Aware Quantization (AIQ) has been introduced to optimize the performance of neural networks by selecting per-layer bit-widths that enhance arithmetic intensity while minimizing accuracy loss. This method has shown a significant increase in throughput and efficiency on models like ResNet-20 and MobileNetV2, outperforming traditional quantization techniques.

A new one-cycle structured pruning framework has been proposed, integrating pre-training, pruning, and fine-tuning into a single training cycle, which aims to enhance efficiency while maintaining accuracy. This method identifies an optimal sub-network early in the training process, utilizing norm-based group saliency criteria and structured sparsity regularization to improve performance.