World PulseNow
Newsletter

Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networks

arXiv — stat.ML•Wednesday, November 12, 2025 at 5:00:00 AM
PositiveArtificial Intelligence
The paper titled 'Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networks' offers four key theoretical contributions aimed at enhancing the usability of risk certificates in neural networks. It derives the tightest explicit bounds on the true risk of classifiers by utilizing KL divergence between Bernoulli distributions. Furthermore, it introduces an efficient optimization methodology based on implicit differentiation, allowing the integration of PAC-Bayesian risk certificate optimization into the loss function used for model training. A significant highlight is the development of a method to optimize bounds on non-differentiable objectives, such as the 0-1 loss. The empirical evaluation on the MNIST and CIFAR-10 datasets demonstrates the practical implications of these theoretical advancements, marking the first non-vacuous generalization bounds on CIFAR-10 for neural networks. The availability of the code on GitHub facilitates further researc…
— via World Pulse Now AI Editorial System
Read Original

Was this article worth reading? Share it

Recommended Readings
Networks with Finite VC Dimension: Pro and Contra
arXiv — stat.MLa day ago
Networks with Finite VC Dimension: Pro and Contra
NeutralArtificial Intelligence
The article discusses the approximation and learning capabilities of neural networks concerning high-dimensional geometry and statistical learning theory. It examines the impact of the VC dimension on the networks' ability to approximate functions and learn from data samples. While a finite VC dimension is beneficial for uniform convergence of empirical errors, it may hinder function approximation from probability distributions relevant to specific applications. The study highlights the deterministic behavior of approximation and empirical errors in networks with finite VC dimensions.
Read full article
via arXiv — stat.ML
Adaptive Symmetrization of the KL Divergence
arXiv — cs.LG2 days ago
Adaptive Symmetrization of the KL Divergence
PositiveArtificial Intelligence
The article titled 'Adaptive Symmetrization of the KL Divergence' discusses a new approach to minimize the Jeffreys divergence in machine learning. This method aims to improve the learning of probability distributions from finite samples by addressing the limitations of the forward KL divergence, which is commonly used but asymmetrical. The proposed technique utilizes a proxy model to enhance optimization, making it applicable in areas such as density estimation, image generation, and simulation-based inference.
Read full article
via arXiv — cs.LG
AMUN: Adversarial Machine UNlearning
arXiv — cs.LG2 days ago
AMUN: Adversarial Machine UNlearning
PositiveArtificial Intelligence
The paper titled 'AMUN: Adversarial Machine UNlearning' discusses a novel method for machine unlearning, which allows users to delete specific datasets to comply with privacy regulations. Traditional exact unlearning methods require significant computational resources, while approximate methods have not achieved satisfactory accuracy. The proposed Adversarial Machine UNlearning (AMUN) technique enhances model performance by fine-tuning on adversarial examples, effectively reducing model confidence on forgotten samples while maintaining accuracy on test datasets.
Read full article
via arXiv — cs.LG
Orthogonal Soft Pruning for Efficient Class Unlearning
arXiv — cs.LG2 days ago
Orthogonal Soft Pruning for Efficient Class Unlearning
PositiveArtificial Intelligence
The article discusses FedOrtho, a federated unlearning framework designed to enhance data unlearning in federated learning environments. It addresses the challenges of balancing forgetting and retention, particularly in non-IID settings. FedOrtho employs orthogonalized deep convolutional kernels and a one-shot soft pruning mechanism, achieving state-of-the-art performance on datasets like CIFAR-10 and TinyImageNet, with over 98% forgetting quality and 97% retention accuracy.
Read full article
via arXiv — cs.LG
destroR: Attacking Transfer Models with Obfuscous Examples to Discard Perplexity
arXiv — cs.CL2 days ago
destroR: Attacking Transfer Models with Obfuscous Examples to Discard Perplexity
NeutralArtificial Intelligence
The paper titled 'destroR: Attacking Transfer Models with Obfuscous Examples to Discard Perplexity' discusses advancements in machine learning and neural networks, particularly in natural language processing. It highlights the vulnerabilities of machine learning models and proposes a novel adversarial attack strategy that generates ambiguous inputs to confuse these models. The research aims to enhance the robustness of machine learning systems by developing adversarial instances with maximum perplexity.
Read full article
via arXiv — cs.CL
Enhanced Structured Lasso Pruning with Class-wise Information
arXiv — cs.CV2 days ago
Enhanced Structured Lasso Pruning with Class-wise Information
PositiveArtificial Intelligence
The paper titled 'Enhanced Structured Lasso Pruning with Class-wise Information' discusses advancements in neural network pruning methods. Traditional pruning techniques often overlook class-wise information, leading to potential loss of statistical data. This study introduces two new pruning schemes, sparse graph-structured lasso pruning with Information Bottleneck (sGLP-IB) and sparse tree-guided lasso pruning with Information Bottleneck (sTLP-IB), aimed at preserving statistical information while reducing model complexity.
Read full article
via arXiv — cs.CV
On the Necessity of Output Distribution Reweighting for Effective Class Unlearning
arXiv — cs.LG2 days ago
On the Necessity of Output Distribution Reweighting for Effective Class Unlearning
PositiveArtificial Intelligence
The paper titled 'On the Necessity of Output Distribution Reweighting for Effective Class Unlearning' identifies a critical flaw in class unlearning evaluations, specifically the neglect of class geometry, which can lead to privacy breaches. It introduces a membership-inference attack via nearest neighbors (MIA-NN) to identify unlearned samples. The authors propose a new fine-tuning objective that adjusts the model's output distribution to mitigate privacy risks, demonstrating that existing unlearning methods are susceptible to MIA-NN across various datasets.
Read full article
via arXiv — cs.LG
PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks
arXiv — cs.LG2 days ago
PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks
PositiveArtificial Intelligence
The paper introduces PrivDFS, a distributed feature-sharing framework designed for input-private inference in image classification. It addresses vulnerabilities in split inference that allow Data Reconstruction Attacks (DRAs) to reconstruct inputs with high fidelity. By fragmenting the intermediate representation and processing these fragments independently across a majority-honest set of servers, PrivDFS limits the reconstruction capability while maintaining accuracy within 1% of non-private methods.
Read full article
via arXiv — cs.LG