Arithmetic-Intensity-Aware Quantization

A new study has introduced GSE ResNeXt, a deep learning architecture that enhances underwater acoustic target classification by integrating learnable Gabor convolutional layers with a ResNeXt backbone and squeeze-and-excitation attention mechanisms. This innovation addresses the challenges posed by complex underwater noise and limited datasets, improving the model's ability to extract discriminative features.

A new framework called Selective Subnetwork Distillation (SSD) has been proposed to enhance continual learning in sparse neural systems, specifically addressing the limitations of Sparse Distributed Memory Multi-Layer Perceptrons (SDMLP). SSD enables the identification and distillation of knowledge from high-activation neurons without relying on task labels or replay, thus preserving modularity while allowing for structural realignment.

A systematic study has been conducted on the privacy-utility relationship in post-training quantization (PTQ) of deep neural networks, focusing on three algorithms: AdaRound, BRECQ, and OBC. The research reveals that low-precision PTQs, specifically at 4-bit, 2-bit, and 1.58-bit levels, can significantly reduce privacy leakage while maintaining model performance across datasets like CIFAR-10, CIFAR-100, and TinyImageNet.

A new one-cycle structured pruning framework has been proposed, integrating pre-training, pruning, and fine-tuning into a single training cycle, which aims to enhance efficiency while maintaining accuracy. This method identifies an optimal sub-network early in the training process, utilizing norm-based group saliency criteria and structured sparsity regularization to improve performance.