arXiv:2511.01724v2 Announce Type: replace-cross 
Abstract: Deep learning models are notoriously vulnerable to imperceptible perturbations. Most existing research centers on adversarial robustness (AR), which evaluates models under worst-case scenarios by examining the existence of deterministic adversarial examples (AEs). In contrast, probabilistic robustness (PR) adopts a statistical perspective, measuring the probability that predictions remain correct under stochastic perturbations. While PR is widely regarded as a practical complement to AR, dedicated training methods for improving PR are still relatively underexplored, albeit with emerging progress. Among the few PR-targeted training methods, we identify three limitations: i non-comparable evaluation protocols; ii limited comparisons to strong AT baselines despite anecdotal PR gains from AT; and iii no unified framework to compare the generalization of these methods. Thus, we introduce PRBench, the first benchmark dedicated to evaluating improvements in PR achieved by different robustness training methods. PRBench empirically compares most common AT and PR-targeted training methods using a comprehensive set of metrics, including clean accuracy, PR and AR performance, training efficiency, and generalization error (GE). We also provide theoretical analysis on the GE of PR performance across different training methods. Main findings revealed by PRBench include: AT methods are more versatile than PR-targeted training methods in terms of improving both AR and PR performance across diverse hyperparameter settings, while PR-targeted training methods consistently yield lower GE and higher clean accuracy. A leaderboard comprising 222 trained models across 7 datasets and 10 model architectures is publicly available at https://tmpspace.github.io/PRBenchLeaderboard/.

تم تقديم معيار جديد يسمى PRBench لتحسين تقييم القوة الاحتمالية (PR) في نماذج التعلم العميق، حيث يتناول القيود المفروضة على طرق التدريب الحالية. يهدف هذا المعيار إلى توفير إطار موحد لمقارنة الأساليب المختلفة المستهدفة لـ PR، والتي تم استكشافها بشكل محدود نسبيًا مقارنة بأساليب القوة العدائية (AR).

Se ha introducido un nuevo benchmark llamado PRBench para mejorar la evaluación de la robustez probabilística (PR) en modelos de aprendizaje profundo, abordando las limitaciones de los métodos de entrenamiento existentes. Este benchmark tiene como objetivo proporcionar un marco unificado para comparar diversos enfoques de entrenamiento dirigidos a la PR, que han sido relativamente poco explorados en comparación con los métodos de robustez adversarial (AR).

Un nouveau benchmark appelé PRBench a été introduit pour améliorer l'évaluation de la robustesse probabiliste (PR) dans les modèles d'apprentissage profond, en abordant les limitations des méthodes d'entraînement existantes. Ce benchmark vise à fournir un cadre unifié pour comparer diverses approches d'entraînement ciblées sur la PR, qui ont été relativement peu explorées par rapport aux méthodes de robustesse adversariale (AR).

A new benchmark called PRBench has been introduced to enhance the evaluation of probabilistic robustness (PR) in deep learning models, addressing the limitations of existing training methods. This benchmark aims to provide a unified framework for comparing various PR-targeted training approaches, which have been relatively underexplored compared to adversarial robustness (AR) methods.

Probabilistic Robustness for Free? Revisiting Training via a Benchmark

One More Thing in AI – Your Shortcut to AI Mastery

Probabilistic Robustness for Free? Revisiting Training via a Benchmark

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Humanize AI

Portfolio Backtest

AskTuring

Dyad

Ready to build your own newsroom?