arXiv:2512.05556v1 Announce Type: new 
Abstract: With the increasing complexity of black-box machine learning models and their adoption in high-stakes areas, it is critical to provide explanations for their predictions. Local Interpretable Model-agnostic Explanation (LIME) is a widely used technique that explains the prediction of any classifier by learning an interpretable model locally around the predicted instance. However, it assumes that the local decision boundary is linear and fails to capture the non-linear relationships, leading to incorrect explanations. In this paper, we propose a novel method that can generate high-fidelity explanations. Multivariate adaptive regression splines (MARS) is used to model non-linear local boundaries that effectively captures the underlying behavior of the reference model, thereby enhancing the local fidelity of the explanation. Additionally, we utilize the N-ball sampling technique, which samples directly from the desired distribution instead of reweighting samples as done in LIME, further improving the faithfulness score. We evaluate our method on three UCI datasets across different classifiers and varying kernel widths. Experimental results show that our method yields more faithful explanations compared to baselines, achieving an average reduction of 37% in root mean square error, significantly improving local fidelity.

تم اقتراح طريقة جديدة لتحسين دقة التفسيرات التي تنتجها تقنية التفسير القابل للتفسير بشكل محلي (LIME) في التعلم الآلي. تستخدم هذه الطريقة الانحدار المتكيف متعدد المتغيرات (MARS) لنمذجة الحدود المحلية غير الخطية، مما يعالج قيود الافتراض الخطي لـ LIME. تهدف هذه الاستراتيجية إلى تقديم تفسيرات أكثر دقة للتنبؤات التي تقوم بها النماذج المعقدة ذات الصندوق الأسود.

Se ha propuesto un nuevo método para mejorar la fidelidad de las explicaciones generadas por la Explicación de Modelo Interpretable de Manera Agnóstica Local (LIME) en el aprendizaje automático. Este método utiliza Splines de Regresión Adaptativa Multivariada (MARS) para modelar fronteras de decisión locales no lineales, abordando las limitaciones de la suposición lineal de LIME. La estrategia busca proporcionar interpretaciones más precisas de las predicciones realizadas por modelos complejos de caja negra.

Une nouvelle méthode a été proposée pour améliorer la fidélité des explications générées par l'explication de modèle interprétable localement (LIME) dans l'apprentissage automatique. Cette méthode utilise des splines de régression adaptative multivariée (MARS) pour modéliser des frontières de décision locales non linéaires, abordant ainsi les limites de l'hypothèse linéaire de LIME. L'approche vise à fournir des interprétations plus précises des prédictions faites par des modèles complexes en boîte noire.

A novel method has been proposed to enhance the fidelity of explanations generated by Local Interpretable Model-agnostic Explanation (LIME) in machine learning. This method utilizes Multivariate Adaptive Regression Splines (MARS) to model non-linear local decision boundaries, addressing the limitations of LIME's linear assumption. The approach aims to provide more accurate interpretations of predictions made by complex black-box models.

Improving Local Fidelity Through Sampling and Modeling Nonlinearity

arXiv:2512.07522v1 Announce Type: new 
Abstract: Pre-training decoder-only language models relies on vast amounts of high-quality data, yet the availability of such data is increasingly reaching its limits. While metadata is commonly used to create and curate these datasets, its potential as a direct training signal remains under-explored. We challenge this status quo and propose LIME (Linguistic Metadata Embeddings), a method that enriches token embeddings with metadata capturing syntax, semantics, and contextual properties. LIME substantially improves pre-training efficiency. Specifically, it adapts up to 56% faster to the training data distribution, while introducing only 0.01% additional parameters at negligible compute overhead. Beyond efficiency, LIME improves tokenization, leading to remarkably stronger language modeling capabilities and generative task performance. These benefits persist across model scales (500M to 2B). In addition, we develop a variant with shifted metadata, LIME+1, that can guide token generation. Given prior metadata for the next token, LIME+1 improves reasoning performance by up to 38% and arithmetic accuracy by up to 35%.

تم تقديم طريقة جديدة تُدعى LIME (تضمينات البيانات اللغوية) لتعزيز كفاءة التدريب المسبق لنماذج اللغة التي تعتمد فقط على فك التشفير من خلال دمج البيانات اللغوية في تضمينات الرموز. تتيح هذه الطريقة للنماذج التكيف بسرعة تصل إلى 56% مع بيانات التدريب مع إضافة حد أدنى من الحمل الحاسوبي والمعلمات.

Se ha introducido un nuevo método llamado LIME (Linguistic Metadata Embeddings) para mejorar la eficiencia del preentrenamiento de modelos de lenguaje solo de decodificación al integrar metadatos lingüísticos en las incrustaciones de tokens. Este enfoque permite que los modelos se adapten hasta un 56 % más rápido a los datos de entrenamiento, añadiendo un costo computacional y parámetros mínimos.

Une nouvelle méthode appelée LIME (Linguistic Metadata Embeddings) a été introduite pour améliorer l'efficacité de la pré-formation des modèles de langage uniquement décodeurs en intégrant des métadonnées linguistiques dans les embeddings de tokens. Cette approche permet aux modèles de s'adapter jusqu'à 56 % plus rapidement aux données d'entraînement tout en ajoutant un coût computationnel et des paramètres minimes.

A new method called LIME (Linguistic Metadata Embeddings) has been introduced to enhance the efficiency of pre-training decoder-only language models by integrating linguistic metadata into token embeddings. This approach allows models to adapt up to 56% faster to training data while adding minimal computational overhead and parameters.

Improving Local Fidelity Through Sampling and Modeling Nonlinearity

Was this article worth reading? Share it

LucidQuery AI

The Visualizer

FastML