arXiv:2605.27078v2 Announce Type: replace 
Abstract: Training loss and accuracy are the standard signals used to monitor generalization during deep neural network training. Two well-documented phenomena complicate this picture: in grokking, train loss falls rapidly while test performance improves abruptly only after a long delay; in epoch-wise double descent, train loss decreases monotonically while test loss or error rises and falls. Existing accounts are often task-specific, and a task-agnostic analysis framework for diagnosing and explaining these phenomena across realistic tasks and architectures is missing. We address this challenge by analyzing two competing processes that underlie learning dynamics: representation learning in the encoder and readout calibration in the final classifier. Using tools from representational geometry, neural tangent kernels, and linear probing, we show that both processes are active throughout training, with the fluctuations of their relative speed giving rise to seemingly anomalous generalization dynamics. Applying the representation-readout decomposition to grokking across a wide range of tasks and architectures, we find that the readout is train-biased before grokking onset, and representation learning is gradual but not absent, contrary to the lazy-to-rich account. The framework further provides diagnostic signatures distinguishing spurious from genuine generalization: in a previously reported MNIST grokking example and an epoch-wise double descent example, apparent delayed or non-monotone generalization is shown to arise from representation degradation and readout misalignment induced by non-standard training recipes. Together, these results establish the representation-readout decomposition as a top-down framework for understanding learning dynamics and revealing underlying algorithms for interpretability research.

دراسة حديثة نُشرت على arXiv تستكشف ديناميات التعلم في الشبكات العصبية العميقة، مع التركيز على ظاهرتين: الغروك، حيث تنخفض خسارة التدريب بسرعة لكن أداء الاختبار يتحسن فقط بعد تأخير، والانحدار المزدوج حسب الحقبة، الذي يتميز بعلاقة غير أحادية بين خسارة التدريب وخسارة الاختبار. تقدم البحث إطارًا غير محدد للمهام لتحليل هذه السلوكيات من خلال تعلم التمثيل ومعايرة القراءة.

Un estudio reciente publicado en arXiv explora la dinámica del aprendizaje en redes neuronales profundas, centrándose en dos fenómenos: el grokking, donde la pérdida de entrenamiento disminuye rápidamente pero el rendimiento de prueba mejora solo después de un retraso, y la doble descenso por época, caracterizada por una relación no monótona entre la pérdida de entrenamiento y la pérdida de prueba. La investigación introduce un marco agnóstico a la tarea para analizar estos comportamientos a través del aprendizaje de representación y la calibración de lectura.

Une étude récente publiée sur arXiv examine la dynamique de l'apprentissage dans les réseaux de neurones profonds, en se concentrant sur deux phénomènes : le grokking, où la perte d'entraînement diminue rapidement mais la performance de test s'améliore seulement après un délai, et la double descente par époque, caractérisée par une relation non monotone entre la perte d'entraînement et la perte de test. La recherche introduit un cadre agnostique aux tâches pour analyser ces comportements à travers l'apprentissage de représentation et la calibration de lecture.

A recent study published on arXiv explores the dynamics of learning in deep neural networks, focusing on two phenomena: grokking, where training loss decreases rapidly but test performance improves only after a delay, and epoch-wise double descent, characterized by a non-monotonic relationship between training and test loss. The research introduces a task-agnostic framework to analyze these behaviors through representation learning and readout calibration.