arXiv:2511.16340v1 Announce Type: cross 
Abstract: Scalable Gaussian process (GP) inference is essential for sequential decision-making tasks, yet improving GP scalability remains a challenging problem with many open avenues of research. This paper focuses on iterative GPs, where iterative linear solvers, such as conjugate gradients, stochastic gradient descent or alternative projections, are used to approximate the GP posterior. We propose a new method which improves solver convergence of a large linear system by leveraging the known solution to a smaller system contained within. This is significant for tasks with incremental data additions, and we show that our technique achieves speed-ups when solving to tolerance, as well as improved Bayesian optimisation performance under a fixed compute budget.

تناقش هذه الورقة التحدي المتمثل في استنتاج العمليات الغاوسية القابلة للتوسع (GP) لمهام اتخاذ القرار المتسلسل. تقدم طريقة تعزز من تقارب الحلول الخطية التكرارية من خلال الاستفادة من الحلول المعروفة من أنظمة أصغر. هذه الطريقة مفيدة بشكل خاص للمهام التي تتضمن إضافات بيانات تدريجية، حيث تظهر تحسينات في السرعة وأداء التحسين البايزي ضمن ميزانية حسابية ثابتة.

Este artículo aborda el desafío de la inferencia escalable de procesos gaussianos (GP) para tareas de toma de decisiones secuenciales. Introduce un método que mejora la convergencia de los solucionadores lineales iterativos al utilizar soluciones conocidas de sistemas más pequeños. Este enfoque es particularmente beneficioso para tareas que implican adiciones de datos incrementales, mostrando mejoras en la velocidad y el rendimiento de la optimización bayesiana dentro de un presupuesto computacional fijo.

Cet article aborde le défi de l'inférence scalable des processus gaussiens (GP) pour les tâches de prise de décision séquentielle. Il introduit une méthode qui améliore la convergence des solveurs linéaires itératifs en utilisant des solutions connues provenant de systèmes plus petits. Cette approche est particulièrement bénéfique pour les tâches impliquant des données incrémentales, montrant une amélioration de la vitesse et des performances d'optimisation bayésienne dans un budget computationnel fixe.

This paper addresses the challenge of scalable Gaussian process (GP) inference for sequential decision-making tasks. It introduces a method that enhances the convergence of iterative linear solvers by utilizing known solutions from smaller systems. This approach is particularly beneficial for tasks involving incremental data, demonstrating improved speed and Bayesian optimization performance within a fixed computational budget.

Improving Iterative Gaussian Processes via Warm Starting Sequential Posteriors

arXiv:2505.11355v2 Announce Type: replace 
Abstract: Gaussian processes (GPs) have gained popularity as flexible machine learning models for regression and function approximation with an in-built method for uncertainty quantification. However, GPs suffer when the amount of training data is large or when the underlying function contains multi-scale features that are difficult to represent by a stationary kernel. To address the former, training of GPs with large-scale data is often performed through inducing point approximations, also known as sparse GP regression (GPR), where the size of the covariance matrices in GPR is reduced considerably through a greedy search on the data set. To aid the latter, deep GPs have gained traction as hierarchical models that resolve multi-scale features by combining multiple GPs. Posterior inference in deep GPs requires a sampling or, more usual, a variational approximation. Variational approximations lead to large-scale stochastic, non-convex optimisation problems and the resulting approximation tends to represent uncertainty incorrectly. In this work, we combine variational learning with MCMC to develop a particle-based expectation-maximisation method to simultaneously find inducing points within the large-scale data (variationally) and accurately train the deep GPs (sampling-based). The result is a highly efficient and accurate methodology for deep GP training on large-scale data. We test our method on standard benchmark problems.

تم استكشاف تقنيات نادرة للانحدار في العمليات الغاوسية العميقة (GPs) لتحسين قابلية التوسع والكفاءة لهذه النماذج، خاصة عند التعامل مع مجموعات بيانات كبيرة أو وظائف متعددة المقاييس معقدة. تبرز الأبحاث استخدام تقريبات النقاط المحورية في الانحدار GPR النادر ومزايا GPs العميقة للنمذجة الهرمية.

Se han explorado técnicas dispersas para la regresión en procesos gaussianos profundos (GPs) para mejorar la escalabilidad y eficiencia de estos modelos, especialmente al tratar con grandes conjuntos de datos o funciones complejas de múltiples escalas. La investigación destaca el uso de aproximaciones por puntos de inducción en la regresión GPR dispersa y las ventajas de los GPs profundos para la modelización jerárquica.

Des techniques éparses pour la régression dans les processus gaussiens profonds (GPs) ont été explorées pour améliorer la scalabilité et l'efficacité de ces modèles, en particulier lors du traitement de grands ensembles de données ou de fonctions multi-échelles complexes. La recherche met en avant l'utilisation d'approximations par points d'induction dans la régression GPR éparse et les avantages des GPs profonds pour la modélisation hiérarchique.

Sparse techniques for regression in deep Gaussian processes (GPs) have been explored to enhance the scalability and efficiency of these models, particularly when dealing with large datasets or complex multi-scale functions. The research highlights the use of inducing point approximations in sparse GP regression (GPR) and the advantages of deep GPs for hierarchical modeling.

Improving Iterative Gaussian Processes via Warm Starting Sequential Posteriors

Was this article worth reading? Share it

GPACal

Augmeta

Supavec