arXiv:2509.23964v2 Announce Type: replace-cross 
Abstract: Label noise in datasets could significantly damage the performance and robustness of deep neural networks (DNNs) trained on these datasets. As the size of modern DNNs grows, there is a growing demand for automated tools for detecting such errors. In this paper, we propose post-hoc, model-agnostic noise detection and rectification methods utilizing the penultimate feature from a DNN. Our idea is based on the observation that the similarity between the penultimate feature of a mislabeled data point and its true class data points is higher than that for data points from other classes, making the probability of label occurrence within a tight, similar cluster informative for detecting and rectifying errors. Through theoretical and empirical analyses, we demonstrate that our approach achieves high detection performance across diverse, realistic noise scenarios and can automatically rectify these errors to improve dataset quality. Our implementation is available at https://anonymous.4open.science/r/noise-detection-and-rectification-AD8E.

تسلط دراسة حديثة الضوء على نهج واعد لمعالجة ضوضاء التسميات في مجموعات البيانات، والتي يمكن أن تعيق أداء الشبكات العصبية العميقة. من خلال تقديم طرق غير مرتبطة بالنموذج لاكتشاف وتصحيح هذه الأخطاء، تتناول الأبحاث حاجة ملحة في مجال التعلم الآلي. لا تعزز هذه الخطوة موثوقية الشبكات العصبية العميقة فحسب، بل تمهد أيضًا الطريق لتطبيقات أكثر قوة في مجالات متنوعة، مما يجعلها مساهمة مهمة في التطور المستمر لتكنولوجيا الذكاء الاصطناعي.

Un estudio reciente destaca un enfoque prometedor para abordar el ruido en las etiquetas de los conjuntos de datos, que puede obstaculizar el rendimiento de las redes neuronales profundas. Al introducir métodos agnósticos al modelo para detectar y corregir estos errores, la investigación aborda una necesidad crítica en el campo del aprendizaje automático. Este avance no solo mejora la fiabilidad de las DNN, sino que también allana el camino para aplicaciones más robustas en diversos dominios, convirtiéndose en una contribución significativa a la evolución continua de la tecnología de IA.

Une étude récente met en avant une approche prometteuse pour lutter contre le bruit des étiquettes dans les ensembles de données, qui peut nuire à la performance des réseaux de neurones profonds. En introduisant des méthodes agnostiques au modèle pour détecter et corriger ces erreurs, la recherche répond à un besoin crucial dans le domaine de l'apprentissage automatique. Cette avancée améliore non seulement la fiabilité des DNN, mais ouvre également la voie à des applications plus robustes dans divers domaines, ce qui en fait une contribution significative à l'évolution continue de la technologie IA.

A recent study highlights a promising approach to tackle label noise in datasets, which can hinder the performance of deep neural networks. By introducing model-agnostic methods for detecting and correcting these errors, the research addresses a critical need in the field of machine learning. This advancement not only enhances the reliability of DNNs but also paves the way for more robust applications in various domains, making it a significant contribution to the ongoing evolution of AI technology.

Detecting and Rectifying Noisy Labels: A Similarity-based Approach

arXiv:2601.08100v1 Announce Type: new 
Abstract: Understanding the generalization behavior of deep neural networks remains a fundamental challenge in modern statistical learning theory. Among existing approaches, PAC-Bayesian norm-based bounds have demonstrated particular promise due to their data-dependent nature and their ability to capture algorithmic and geometric properties of learned models. However, most existing results rely on isotropic Gaussian posteriors, heavy use of spectral-norm concentration for weight perturbations, and largely architecture-agnostic analyses, which together limit both the tightness and practical relevance of the resulting bounds. To address these limitations, in this work, we propose a unified framework for PAC-Bayesian norm-based generalization by reformulating the derivation of generalization bounds as a stochastic optimization problem over anisotropic Gaussian posteriors. The key to our approach is a sensitivity matrix that quantifies the network outputs with respect to structured weight perturbations, enabling the explicit incorporation of heterogeneous parameter sensitivities and architectural structures. By imposing different structural assumptions on this sensitivity matrix, we derive a family of generalization bounds that recover several existing PAC-Bayesian results as special cases, while yielding bounds that are comparable to or tighter than state-of-the-art approaches. Such a unified framework provides a principled and flexible way for geometry-/structure-aware and interpretable generalization analysis in deep learning.

تشير دراسة جديدة إلى إطار موحد لـ PAC-Bayes للحدود العامة المعتمدة على المعايير، حيث تتناول التحديات المتعلقة بفهم سلوك التعميم في الشبكات العصبية العميقة. تعيد هذه الدراسة صياغة اشتقاق هذه الحدود كمشكلة تحسين عشوائي على توزيعات غاوسية غير متساوية، بهدف تعزيز الصلة العملية للنتائج.

Un nuevo estudio propone un marco PAC-Bayes unificado para los límites de generalización basados en normas, abordando los desafíos de comprender el comportamiento de generalización de las redes neuronales profundas. La investigación reformula la derivación de estos límites como un problema de optimización estocástica sobre posteriors gaussianos anisotrópicos, con el objetivo de mejorar la relevancia práctica de los resultados.

Une nouvelle étude propose un cadre PAC-Bayésien unifié pour les bornes de généralisation basées sur les normes, abordant les défis de la compréhension du comportement de généralisation des réseaux de neurones profonds. La recherche reformule la dérivation de ces bornes comme un problème d'optimisation stochastique sur des postérieurs gaussiens anisotropes, visant à améliorer la pertinence pratique des résultats.

A new study proposes a unified PAC-Bayesian framework for norm-based generalization bounds, addressing the challenges of understanding deep neural networks' generalization behavior. The research reformulates the derivation of these bounds as a stochastic optimization problem over anisotropic Gaussian posteriors, aiming to enhance the practical relevance of the results.

Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds

arXiv:2601.07944v1 Announce Type: new 
Abstract: Since the turn of the century, approximate Bayesian inference has steadily evolved as new computational techniques have been incorporated to handle increasingly complex and large-scale predictive problems. The recent success of deep neural networks and foundation models has now given rise to a new paradigm in statistical modeling, in which Bayesian inference can be amortized through large-scale learned predictors. In amortized inference, substantial computation is invested upfront to train a neural network that can subsequently produce approximate posterior or predictions at negligible marginal cost across a wide range of tasks. At deployment, amortized inference offers substantial computational savings compared with traditional Bayesian procedures, which generally require repeated likelihood evaluations or Monte Carlo simulations for predictions for each new dataset.
  Despite the growing popularity of amortized inference, its statistical interpretation and its role within Bayesian inference remain poorly understood. This paper presents statistical perspectives on the working principles of several major neural architectures, including feedforward networks, Deep Sets, and Transformers, and examines how these architectures naturally support amortized Bayesian inference. We discuss how these models perform structured approximation and probabilistic reasoning in ways that yield controlled generalization error across a wide range of deployment scenarios, and how these properties can be harnessed for Bayesian computation. Through simulation studies, we evaluate the accuracy, robustness, and uncertainty quantification of amortized inference under varying signal-to-noise ratios and distributional shifts, highlighting both its strengths and its limitations.

أظهرت دراسة حديثة فعالية الاستدلال المدعوم في الإحصاءات البايزية، خاصة تحت تغيرات نسبة الإشارة إلى الضوضاء وتحولات التوزيع. تستخدم هذه الطريقة الشبكات العصبية العميقة لتبسيط عملية الاستدلال، مما يسمح بتوفير كبير في الحوسبة مقارنة بالأساليب البايزية التقليدية التي تتطلب تقييمات واسعة لاحتمالية.

Un estudio reciente ha evaluado la efectividad de la inferencia amortizada en estadísticas bayesianas, particularmente bajo variaciones en la relación señal-ruido y cambios en la distribución. Este método aprovecha redes neuronales profundas para agilizar el proceso de inferencia, permitiendo ahorros computacionales significativos en comparación con los enfoques bayesianos tradicionales que requieren evaluaciones extensas de verosimilitud.

Une étude récente a évalué l'efficacité de l'inférence amortie dans les statistiques bayésiennes, en particulier sous des variations de rapport signal-bruit et des changements de distribution. Cette méthode utilise des réseaux de neurones profonds pour rationaliser le processus d'inférence, permettant des économies computationnelles significatives par rapport aux approches bayésiennes traditionnelles qui nécessitent des évaluations de vraisemblance étendues.

A recent study has assessed the effectiveness of amortized inference in Bayesian statistics, particularly under varying signal-to-noise ratios and distribution shifts. This method leverages deep neural networks to streamline the inference process, allowing for significant computational savings compared to traditional Bayesian approaches that require extensive likelihood evaluations.

A Statistical Assessment of Amortized Inference Under Signal-to-Noise Variation and Distribution Shift

One More Thing in AI – Your Shortcut to AI Mastery

Detecting and Rectifying Noisy Labels: A Similarity-based Approach

Was this article worth reading? Share it

One More Thing in AI

WasItAI

Airparser

Humanize AI

LucidQuery AI

Attentive AI

Ready to build your own newsroom?