arXiv:2602.16608v2 Announce Type: replace-cross 
Abstract: Transformer models achieve state-of-the-art performance across domains and tasks, yet their deeply layered representations make their predictions difficult to interpret. Existing explainability methods rely on final-layer attributions, capture either local token-level attributions or global attention patterns without unification, and lack context-awareness of inter-token dependencies and structural components. They also fail to capture how relevance evolves across layers and how structural components shape decision-making. To address these limitations, we proposed the \textbf{Context-Aware Layer-wise Integrated Gradients (CA-LIG) Framework}, a unified hierarchical attribution framework that computes layer-wise Integrated Gradients within each Transformer block and fuses these token-level attributions with class-specific attention gradients. This integration yields signed, context-sensitive attribution maps that capture supportive and opposing evidence while tracing the hierarchical flow of relevance through the Transformer layers. We evaluate the CA-LIG Framework across diverse tasks, domains, and transformer model families, including sentiment analysis and long and multi-class document classification with BERT, hate speech detection in a low-resource language setting with XLM-R and AfroLM, and image classification with Masked Autoencoder vision Transformer model. Across all tasks and architectures, CA-LIG provides more faithful attributions, shows stronger sensitivity to contextual dependencies, and produces clearer, more semantically coherent visualizations than established explainability methods. These results indicate that CA-LIG provides a more comprehensive, context-aware, and reliable explanation of Transformer decision-making, advancing both the practical interpretability and conceptual understanding of deep neural models.

قدم الباحثون إطار عمل Gradients المدمجة على مستوى الطبقات الواعية بالسياق (CA-LIG)، وهو نهج جديد لتحسين قابلية تفسير نماذج Transformer. يحسب هذا الإطار Gradients المدمجة على مستوى الطبقات داخل كل كتلة Transformer، مما يعالج قيود الأساليب الحالية التي تفشل في توحيد التخصيصات المحلية والعالمية وتفتقر إلى الوعي بالسياق.

Investigadores han presentado el marco Context-Aware Layer-wise Integrated Gradients (CA-LIG), un enfoque novedoso para mejorar la explicabilidad de los modelos Transformer. Este marco calcula los Gradientes Integrados capa por capa dentro de cada bloque Transformer, abordando las limitaciones de los métodos existentes que no logran unificar las atribuciones locales y globales y carecen de conciencia contextual.

Des chercheurs ont introduit le cadre Context-Aware Layer-wise Integrated Gradients (CA-LIG), une nouvelle approche visant à améliorer l'explicabilité des modèles Transformer. Ce cadre calcule les gradients intégrés couche par couche au sein de chaque bloc Transformer, répondant aux limitations des méthodes existantes qui échouent à unifier les attributions locales et globales et manquent de conscience contextuelle.

Researchers have introduced the Context-Aware Layer-wise Integrated Gradients (CA-LIG) Framework, a novel approach to enhance the explainability of Transformer models. This framework computes layer-wise Integrated Gradients within each Transformer block, addressing the limitations of existing methods that fail to unify local and global attributions and lack context-awareness.