arXiv:2504.10833v3 Announce Type: replace 
Abstract: Deep vision models perform input-output computations that are hard to interpret. Concept-based explanation methods (CBEMs) increase interpretability by re-expressing parts of the model with human-understandable semantic units, or concepts. Checking if the derived explanations are faithful -- that is, they represent the model's internal computation -- requires a surrogate that combines concepts to compute the output. Simplifications made for interpretability inevitably reduce faithfulness, resulting in a tradeoff between the two. State-of-the-art unsupervised CBEMs (U-CBEMs) have reported increasingly interpretable concepts, while also being more faithful to the model. However, we observe that the reported improvement in faithfulness artificially results from either (1) using overly complex surrogates, which introduces an unmeasured cost to the explanation's interpretability, or (2) relying on deletion-based approaches that, as we demonstrate, do not properly measure faithfulness. We propose Surrogate Faithfulness (SURF), which (1) replaces prior complex surrogates with a simple, linear surrogate that measures faithfulness without changing the explanation's interpretability and (2) introduces well-motivated metrics that assess loss across all output classes, not just the predicted class. We validate SURF with a measure-over-measure study by proposing a simple sanity check -- explanations with random concepts should be less faithful -- which prior surrogates fail. SURF enables the first reliable faithfulness benchmark of U-CBEMs, revealing that many visually compelling U-CBEMs are not faithful. Code to be released.

تعتبر نماذج الرؤية العميقة أنظمة معقدة يصعب تفسير حساباتها. تهدف طرق التفسير المعتمدة على المفاهيم (CBEM) إلى تعزيز القابلية للتفسير من خلال استخدام مفاهيم مفهومة للبشر. ومع ذلك، فإن ضمان موثوقية هذه التفسيرات، التي تمثل العمليات الداخلية للنموذج، يتطلب تنازلات. تشير التطورات الأخيرة في CBEM غير الخاضعة للإشراف (U-CBEM) إلى تحسين كل من القابلية للتفسير والموثوقية، لكن هذه التحسينات قد تنشأ من استخدام بدائل معقدة أو طرق تعتمد على الحذف التي تؤثر على الوضوح.

Los modelos de visión profunda son sistemas complejos cuyas computaciones son difíciles de interpretar. Los métodos de explicación basados en conceptos (CBEM) buscan mejorar la interpretabilidad utilizando conceptos comprensibles para los humanos. Sin embargo, garantizar la fidelidad de estas explicaciones, que representan el funcionamiento interno del modelo, implica compromisos. Los avances recientes en CBEM no supervisados (U-CBEM) afirman mejorar tanto la interpretabilidad como la fidelidad, pero estas mejoras pueden derivarse de surrogados complejos o métodos basados en eliminación que co…

Les modèles de vision profonde sont des systèmes complexes dont les calculs sont difficiles à interpréter. Les méthodes d'explication basées sur des concepts (CBEM) visent à améliorer l'interprétabilité en utilisant des concepts compréhensibles par les humains. Cependant, garantir la fidélité de ces explications, qui représentent le fonctionnement interne du modèle, implique des compromis. Les avancées récentes dans les CBEM non supervisés (U-CBEM) prétendent améliorer à la fois l'interprétabilité et la fidélité, mais ces améliorations peuvent découler de surrogates complexes ou de méthodes ba…

Deep vision models are complex systems that perform computations difficult to interpret. Concept-based explanation methods (CBEMs) aim to enhance interpretability by using human-understandable concepts. However, ensuring the faithfulness of these explanations, which represent the model's internal workings, involves trade-offs. Recent advancements in unsupervised CBEMs (U-CBEMs) claim to improve both interpretability and faithfulness, but these improvements may stem from complex surrogates or deletion-based methods that compromise clarity.

Measuring the (Un)Faithfulness of Concept-Based Explanations

arXiv:2511.15779v1 Announce Type: cross 
Abstract: Beyond the practical goal of improving search and measurement sensitivity through better jet tagging algorithms, there is a deeper question: what are their upper performance limits? Generative surrogate models with learned likelihood functions offer a new approach to this problem, provided the surrogate correctly captures the underlying data distribution. In this work, we introduce the SUrrogate ReFerence (SURF) method, a new approach to validating generative models. This framework enables exact Neyman-Pearson tests by training the target model on samples from another tractable surrogate, which is itself trained on real data. We argue that the EPiC-FM generative model is a valid surrogate reference for JetClass jets and apply SURF to show that modern jet taggers may already be operating close to the true statistical limit. By contrast, we find that autoregressive GPT models unphysically exaggerate top vs. QCD separation power encoded in the surrogate reference, implying that they are giving a misleading picture of the fundamental limit.

يتناول المقال طريقة SURF، وهي نهج جديد للتحقق من صحة النماذج التوليدية في تصنيف الجتات. يبرز أهمية فهم حدود الأداء العليا لخوارزميات تصنيف الجتات. من خلال استخدام نماذج توليدية بديلة، تتيح طريقة SURF إجراء اختبارات Neyman-Pearson الدقيقة، مما يدل على أن مصنفي الجتات الحديثة قد تعمل بالفعل بالقرب من حدودها الإحصائية الحقيقية. تطبق الدراسة نموذج EPiC-FM التوليدي كمرجع بديل صالح لجتات JetClass.

El artículo discute el método SURF, un nuevo enfoque para validar modelos generativos en el etiquetado de jets. Destaca la importancia de comprender los límites de rendimiento de los algoritmos de etiquetado de jets. Al utilizar modelos generativos de sustitución, el método SURF permite realizar pruebas de Neyman-Pearson exactas, demostrando que los etiquetadores de jets modernos pueden estar operando cerca de sus límites estadísticos. El estudio aplica específicamente el modelo generativo EPiC-FM como referencia válida para jets JetClass.

L'article traite de la méthode SURF, une nouvelle approche pour valider les modèles génératifs dans le tagging de jets. Il souligne l'importance de comprendre les limites de performance des algorithmes de tagging de jets. En utilisant des modèles génératifs de substitution, la méthode SURF permet des tests de Neyman-Pearson exacts, démontrant que les taggers de jets modernes peuvent fonctionner près de leurs limites statistiques. L'étude applique spécifiquement le modèle génératif EPiC-FM comme référence valide pour les jets JetClass.

The article discusses the SURF method, a new approach to validating generative models in jet tagging. It highlights the importance of understanding the upper performance limits of jet tagging algorithms. By using generative surrogate models, the SURF method enables exact Neyman-Pearson tests, demonstrating that modern jet taggers may be operating near their statistical limits. The study specifically applies the EPiC-FM generative model as a valid surrogate reference for JetClass jets.

Measuring the (Un)Faithfulness of Concept-Based Explanations

Was this article worth reading? Share it