One-Cycle Structured Pruning via Stability-Driven Subnetwork Search

A new study has introduced GSE ResNeXt, a deep learning architecture that enhances underwater acoustic target classification by integrating learnable Gabor convolutional layers with a ResNeXt backbone and squeeze-and-excitation attention mechanisms. This innovation addresses the challenges posed by complex underwater noise and limited datasets, improving the model's ability to extract discriminative features.

A new gradient-based membership inference attack for federated learning has been introduced, leveraging the temporal evolution of last-layer gradients across multiple federated rounds. This method does not require access to private datasets and is designed to address both semi-honest and malicious adversaries, expanding the scope of potential data leaks in federated learning scenarios.

A new framework called Selective Subnetwork Distillation (SSD) has been proposed to enhance continual learning in sparse neural systems, specifically addressing the limitations of Sparse Distributed Memory Multi-Layer Perceptrons (SDMLP). SSD enables the identification and distillation of knowledge from high-activation neurons without relying on task labels or replay, thus preserving modularity while allowing for structural realignment.

A systematic study has been conducted on the privacy-utility relationship in post-training quantization (PTQ) of deep neural networks, focusing on three algorithms: AdaRound, BRECQ, and OBC. The research reveals that low-precision PTQs, specifically at 4-bit, 2-bit, and 1.58-bit levels, can significantly reduce privacy leakage while maintaining model performance across datasets like CIFAR-10, CIFAR-100, and TinyImageNet.

A new framework called Solution Flow Models (SoFlow) has been introduced, enabling one-step generative modeling from scratch. This approach addresses the inefficiencies associated with multi-step denoising processes in diffusion and Flow Matching models by proposing a Flow Matching loss and a solution consistency loss that enhance training performance without requiring complex calculations like the Jacobian-vector product.

A new study presents REAL (Representation Enhanced Analytic Learning), a method designed to improve exemplar-free class-incremental learning (EFCIL) by addressing issues of representation and knowledge utilization in existing analytic continual learning frameworks. REAL employs a dual-stream pretraining approach followed by a representation-enhancing distillation process to create a more effective classifier during class-incremental learning.

A new framework called Arithmetic-Intensity-Aware Quantization (AIQ) has been introduced to optimize the performance of neural networks by selecting per-layer bit-widths that enhance arithmetic intensity while minimizing accuracy loss. This method has shown a significant increase in throughput and efficiency on models like ResNet-20 and MobileNetV2, outperforming traditional quantization techniques.