arXiv:2505.11638v4 Announce Type: replace-cross 
Abstract: Natural Gradient Descent (NGD) has emerged as a promising optimization algorithm for training neural network-based solvers for partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs). However, its practical use is often limited by the high computational cost of solving linear systems involving the Gramian matrix. While matrix-free NGD methods based on the conjugate gradient (CG) method avoid explicit matrix inversion, the ill-conditioning of the Gramian significantly slows the convergence of CG. In this work, we extend matrix-free NGD to broader classes of problems than previously considered and propose the use of Randomized Numerical Linear Algebra (RandNLA) techniques for efficient preconditioning of the inner CG solver. The resulting algorithm demonstrates substantial performance improvements over existing NGD-based methods and other state-of-the-art optimizers on a range of PDE problems discretized using neural networks.

قدمت دراسة جديدة طريقة محسنة لأسلوب الانحدار الطبيعي (NGD) المخصص للشبكات العصبية المستندة إلى الفيزياء (PINNs)، باستخدام تقنيات الجبر الخطي العددي العشوائي (RandNLA) لتحسين الكفاءة الحسابية. تعالج هذه التطورات التحديات التي تطرحها الحالة السيئة لمصفوفة غرام، والتي كانت تعيق سابقًا تقارب خوارزميات التحسين.

Un nuevo estudio ha presentado un método mejorado para el Descenso de Gradiente Natural (NGD) adaptado a las Redes Neuronales Informadas por la Física (PINNs), utilizando técnicas de Álgebra Lineal Numérica Aleatorizada (RandNLA) para mejorar la eficiencia computacional. Este avance aborda los desafíos planteados por la mala condición de la matriz de Gram, que anteriormente obstaculizaba la convergencia de los algoritmos de optimización.

Une nouvelle étude a introduit une méthode améliorée pour la descente de gradient naturel (NGD) adaptée aux réseaux de neurones informés par la physique (PINNs), utilisant des techniques d'algèbre linéaire numérique randomisée (RandNLA) pour améliorer l'efficacité computationnelle. Cette avancée répond aux défis posés par la mauvaise condition du matrice de Gram, qui entravait auparavant la convergence des algorithmes d'optimisation.

A new study has introduced an enhanced method for Natural Gradient Descent (NGD) tailored for Physics-Informed Neural Networks (PINNs), utilizing Randomized Numerical Linear Algebra (RandNLA) techniques to improve computational efficiency. This advancement addresses the challenges posed by the Gramian matrix's ill-conditioning, which previously hindered the convergence of optimization algorithms.