Oscillations Make Neural Networks Robust to Quantization

The introduction of DAASH, a meta-attack framework, marks a significant advancement in generating effective and perceptually aligned adversarial examples, addressing the limitations of traditional Lp-norm constrained methods. This framework strategically composes existing attack methods in a multi-stage process, enhancing the perceptual alignment of adversarial examples.

A recent study explores the potential of pruning at initialization (PaI) by drawing inspiration from iterative pruning methods, aiming to enhance performance in deep learning models. The research highlights the significance of identifying surviving subnetworks based on initial features, which could lead to more efficient pruning strategies and reduced training costs, especially as neural networks grow in size.

A recent study has introduced a method for fully decentralized certified unlearning in machine learning, focusing on the removal of specific data influences from trained models without a central coordinator. This approach, termed RR-DU, employs a random-walk procedure to enhance privacy and mitigate data poisoning risks, providing convergence guarantees in convex scenarios and stationarity in nonconvex cases.

The paper introduces Arc Gradient Descent (ArcGD), a new optimizer that reformulates traditional gradient descent methods to incorporate phase-aware and user-controlled step dynamics. The evaluation of ArcGD shows it outperforming the Adam optimizer on a non-convex benchmark and a real-world ML dataset, particularly in challenging scenarios like the Rosenbrock function and CIFAR-10 image classification.

A new study presents a hybrid generative classification approach aimed at enhancing adversarial robustness in deep learning models. The proposed deep ensemble model integrates a pre-trained discriminative network for feature extraction with a generative classification network, achieving high accuracy and robustness against adversarial attacks without the need for adversarial training. Extensive experiments on CIFAR-10 and CIFAR-100 validate its effectiveness.

Recent research has identified an 'Inductive Bottleneck' in Vision Transformers (ViTs), where these models exhibit a U-shaped entropy profile, compressing information in middle layers before expanding it for final classification. This phenomenon is linked to the semantic abstraction required by specific tasks and is not merely an architectural flaw but a data-dependent adaptation observed across various datasets such as UC Merced, Tiny ImageNet, and CIFAR-100.

A recent study highlights the vulnerabilities of backdoor defenses in neural networks when subjected to post-training quantization, revealing that INT8 quantization leads to a 0% detection rate for all evaluated defenses while attack success rates remain above 99%. This raises concerns about the effectiveness of existing security measures in machine learning systems.

A new study proposes a structured initialization method for Vision Transformers (ViTs), aiming to integrate the strong inductive biases of Convolutional Neural Networks (CNNs) without altering the architecture. This approach is designed to enhance performance on small datasets while maintaining scalability as data increases.