arXiv:2511.15487v1 Announce Type: new 
Abstract: Implicit Neural Representations (INRs) parameterize continuous signals via multilayer perceptrons (MLPs), enabling compact, resolution-independent modeling for tasks like image, audio, and 3D reconstruction. However, fitting high-resolution signals demands optimizing over millions of coordinates, incurring prohibitive computational costs. To address it, we propose NTK-Guided Implicit Neural Teaching (NINT), which accelerates training by dynamically selecting coordinates that maximize global functional updates. Leveraging the Neural Tangent Kernel (NTK), NINT scores examples by the norm of their NTK-augmented loss gradients, capturing both fitting errors and heterogeneous leverage (self-influence and cross-coordinate coupling). This dual consideration enables faster convergence compared to existing methods. Through extensive experiments, we demonstrate that NINT significantly reduces training time by nearly half while maintaining or improving representation quality, establishing state-of-the-art acceleration among recent sampling-based strategies.

تعتبر طريقة التعليم العصبي الضمني الموجه بواسطة NTK (NINT) مقترحًا لتحسين تدريب التمثيلات العصبية الضمنية (INRs) من خلال تحسين اختيار الإحداثيات لتعزيز الكفاءة الحاسوبية. من خلال استخدام النواة المماسية العصبية (NTK)، تقوم NINT بتقييم الأمثلة بناءً على تدرجات خسائرها، مما يسمح بتقارب أسرع في التدريب. تشير النتائج التجريبية إلى أن NINT يمكن أن تقلل من وقت التدريب بمقدار النصف تقريبًا مع الحفاظ على الأداء.

La enseñanza neural implícita guiada por NTK (NINT) es un método propuesto que mejora el entrenamiento de las Representaciones Neurales Implícitas (INRs) al optimizar la selección de coordenadas para mejorar la eficiencia computacional. Al utilizar el Núcleo Tangente Neural (NTK), NINT puntúa ejemplos según sus gradientes de pérdida, lo que permite una convergencia más rápida en el entrenamiento. Los resultados experimentales indican que NINT puede reducir el tiempo de entrenamiento casi a la mitad mientras mantiene el rendimiento.

L'enseignement neural implicite guidé par NTK (NINT) est une méthode proposée qui améliore l'entraînement des représentations neuronales implicites (INRs) en optimisant la sélection des coordonnées pour améliorer l'efficacité computationnelle. En utilisant le noyau tangent neural (NTK), NINT évalue les exemples en fonction de leurs gradients de perte, permettant une convergence plus rapide lors de l'entraînement. Les résultats expérimentaux montrent que NINT peut réduire le temps d'entraînement de près de moitié tout en maintenant les performances.

NTK-Guided Implicit Neural Teaching (NINT) is a proposed method that enhances the training of Implicit Neural Representations (INRs) by optimizing coordinate selection to improve computational efficiency. By utilizing the Neural Tangent Kernel (NTK), NINT scores examples based on their loss gradients, allowing for faster convergence in training. Experimental results indicate that NINT can reduce training time by nearly half while maintaining performance.

NTK-Guided Implicit Neural Teaching

arXiv:2505.17260v2 Announce Type: replace 
Abstract: Over time, a growing wave of large language models from various series has been introduced to the community. Researchers are striving to maximize the performance of language models with constrained parameter sizes. However, from a microscopic perspective, there has been limited research on how to better store knowledge in model parameters, particularly within MLPs, to enable more effective utilization of this knowledge by the model. In this work, we analyze twenty publicly available open-source large language models to investigate the relationship between their strong performance and the way knowledge is stored in their corresponding MLP parameters. Our findings reveal that as language models become more advanced and demonstrate stronger knowledge capabilities, their parameters exhibit increased specialization. Specifically, parameters in the MLPs tend to be more focused on encoding similar types of knowledge. We experimentally validate that this specialized distribution of knowledge contributes to improving the efficiency of knowledge utilization in these models. Furthermore, by conducting causal training experiments, we confirm that this specialized knowledge distribution plays a critical role in improving the model's efficiency in leveraging stored knowledge.

أظهرت دراسة حديثة تحليل عشرين نموذجًا مفتوح المصدر للغة (LLMs) لاستكشاف كيفية تخزين المعرفة في معلمات MLP الخاصة بها، كاشفةً أنه مع تقدم النماذج، تصبح معلماتها أكثر تخصصًا في ترميز أنواع مشابهة من المعرفة. تسلط هذه الدراسة الضوء على اتجاه متزايد نحو تخصص المعلمات لتخزين المعرفة بشكل فعال في LLMs.

Un estudio reciente ha analizado veinte modelos de lenguaje de código abierto (LLMs) para explorar cómo se almacenan los conocimientos en sus parámetros MLP, revelando que a medida que los modelos avanzan, sus parámetros se especializan cada vez más en codificar tipos similares de conocimiento. Esta investigación destaca una tendencia creciente hacia la especialización de parámetros para un almacenamiento efectivo del conocimiento en los LLMs.

Une étude récente a analysé vingt modèles de langage open-source (LLMs) pour explorer comment les connaissances sont stockées dans leurs paramètres MLP, révélant qu'à mesure que les modèles avancent, leurs paramètres deviennent de plus en plus spécialisés dans l'encodage de types de connaissances similaires. Cette recherche met en lumière une tendance croissante à la spécialisation des paramètres pour un stockage efficace des connaissances dans les LLMs.

A recent study has analyzed twenty open-source large language models (LLMs) to explore how knowledge is stored in their MLP parameters, revealing that as models advance, their parameters become increasingly specialized in encoding similar types of knowledge. This research highlights a growing trend in parameter specialization for effective knowledge storage in LLMs.

The Rise of Parameter Specialization for Knowledge Storage in Large Language Models

arXiv:2506.08274v5 Announce Type: replace-cross 
Abstract: This research addresses the critical lack of comprehensive studies on feature scaling by systematically evaluating 12 scaling techniques - including several less common transformations - across 14 different Machine Learning algorithms and 16 datasets for classification and regression tasks. We meticulously analyzed impacts on predictive performance (using metrics such as accuracy, MAE, MSE, and $R^2$) and computational costs (training time, inference time, and memory usage). Key findings reveal that while ensemble methods (such as Random Forest and gradient boosting models like XGBoost, CatBoost and LightGBM) demonstrate robust performance largely independent of scaling, other widely used models such as Logistic Regression, SVMs, TabNet, and MLPs show significant performance variations highly dependent on the chosen scaler. This extensive empirical analysis, with all source code, experimental results, and model parameters made publicly available to ensure complete transparency and reproducibility, offers model-specific crucial guidance to practitioners on the need for an optimal selection of feature scaling techniques.

دراسة حديثة نُشرت على arXiv قامت بتقييم منهجي لـ 12 تقنية لتقييس الميزات عبر 14 خوارزمية تعلم آلي و16 مجموعة بيانات، كاشفة عن تباينات كبيرة في الأداء في نماذج مثل الانحدار اللوجستي وSVM، بينما أظهرت طرق التجميع قوة مستقلة عن التقييس.

Un estudio reciente publicado en arXiv evaluó sistemáticamente 12 técnicas de escalado de características a través de 14 algoritmos de aprendizaje automático y 16 conjuntos de datos, revelando variaciones significativas en el rendimiento de modelos como la regresión logística y los SVM, mientras que los métodos de ensamblaje mostraron robustez independientemente del escalado.

Une étude récente publiée sur arXiv a évalué systématiquement 12 techniques de mise à l'échelle des caractéristiques à travers 14 algorithmes d'apprentissage automatique et 16 ensembles de données, révélant des variations de performance significatives dans des modèles tels que la régression logistique et les SVM, tandis que les méthodes d'ensemble ont montré une robustesse indépendamment de la mise à l'échelle.

A recent study published on arXiv systematically evaluated 12 feature scaling techniques across 14 machine learning algorithms and 16 datasets, revealing significant performance variations in models like Logistic Regression and SVMs, while ensemble methods showed robustness regardless of scaling.

NTK-Guided Implicit Neural Teaching

Was this article worth reading? Share it

MicroEstimates

GPTHuman

Https