arXiv:2508.03679v3 Announce Type: replace-cross 
Abstract: Gaussian Processes (GPs), as a nonparametric learning method, offer flexible modeling capabilities and calibrated uncertainty quantification for function approximations. Additionally, GPs support online learning by efficiently incorporating new data with polynomial-time computation, making them well-suited for safety-critical dynamical systems that require rapid adaptation. However, the inference and online updates of exact GPs, when processing streaming data, incur cubic computation time and quadratic storage memory complexity, limiting their scalability to large datasets in real-time settings. In this paper, we propose a streaming kernel-induced progressively generated expert framework of Gaussian processes (SkyGP) that addresses both computational and memory constraints by maintaining a bounded set of experts, while inheriting the learning performance guarantees from exact Gaussian processes. Furthermore, two SkyGP variants are introduced, each tailored to a specific objective, either maximizing prediction accuracy (SkyGP-Dense) or improving computational efficiency (SkyGP-Fast). The effectiveness of SkyGP is validated through extensive benchmarks and real-time control experiments demonstrating its superior performance compared to state-of-the-art approaches.

تقدم الورقة إطارًا جديدًا يسمى SkyGP، الذي يستخدم العمليات الغاوسية (GP) للتعلم عبر الإنترنت والتحكم. تُعرف العمليات الغاوسية بمرونتها وقدرتها على قياس عدم اليقين في تقريب الدوال. ومع ذلك، تواجه العمليات الغاوسية التقليدية تحديات في قابلية التوسع بسبب متطلبات الحوسبة والذاكرة العالية عند معالجة البيانات المتدفقة. يتناول SkyGP هذه المشكلات من خلال الحفاظ على مجموعة محدودة من الخبراء، مما يضمن تحديثات فعالة عبر الإنترنت مع الحفاظ على فوائد الأداء للعمليات الغاوسية الدقيقة.

El artículo presenta un nuevo marco llamado SkyGP, que utiliza Procesos Gaussianos (GP) para el aprendizaje en línea y el control. Los GP son conocidos por su flexibilidad y capacidad para cuantificar la incertidumbre en las aproximaciones de funciones. Sin embargo, los GP tradicionales enfrentan desafíos en escalabilidad debido a los altos requisitos computacionales y de memoria al procesar datos en streaming. SkyGP aborda estos problemas al mantener un conjunto limitado de expertos, garantizando actualizaciones en línea eficientes mientras preserva los beneficios de rendimiento de los GP exa…

Cet article présente un nouveau cadre appelé SkyGP, qui utilise les processus gaussiens (GP) pour l'apprentissage en ligne et le contrôle. Les GP sont connus pour leur flexibilité et leur capacité à quantifier l'incertitude dans les approximations de fonctions. Cependant, les GP traditionnels rencontrent des défis en matière d'évolutivité en raison de leurs exigences élevées en matière de calcul et de mémoire lors du traitement de données en streaming. SkyGP aborde ces problèmes en maintenant un ensemble limité d'experts, garantissant des mises à jour en ligne efficaces tout en préservant les …

The paper presents a novel framework called SkyGP, which utilizes Gaussian Processes (GPs) for online learning and control. GPs are known for their flexibility and ability to quantify uncertainty in function approximations. However, traditional GPs face challenges in scalability due to high computational and memory requirements when processing streaming data. SkyGP addresses these issues by maintaining a limited set of experts, ensuring efficient online updates while preserving the performance benefits of exact GPs.

Streaming Generated Gaussian Process Experts for Online Learning and Control: Extended Version

arXiv:2601.08172v1 Announce Type: new 
Abstract: Many real-world tasks require optimizing expensive black-box functions accessible only through noisy evaluations, a setting commonly addressed with Bayesian optimization (BO). While Bayesian neural networks (BNNs) have recently emerged as scalable alternatives to Gaussian Processes (GPs), traditional BNN-BO frameworks remain burdened by expensive posterior sampling and acquisition function optimization. In this work, we propose {VBO-MI} (Variational Bayesian Optimization with Mutual Information), a fully gradient-based BO framework that leverages recent advances in variational mutual information estimation. To enable end-to-end gradient flow, we employ an actor-critic architecture consisting of an {action-net} to navigate the input space and a {variational critic} to estimate information gain. This formulation effectively eliminates the traditional inner-loop acquisition optimization bottleneck, achieving up to a {$10^2 \times$ reduction in FLOPs} compared to BNN-BO baselines. We evaluate our method on a diverse suite of benchmarks, including high-dimensional synthetic functions and complex real-world tasks such as PDE optimization, the Lunar Lander control problem, and categorical Pest Control. Our experiments demonstrate that VBO-MI consistently provides the same or superior optimization performance and computational scalability over the baselines.

تم تقديم إطار عمل جديد يسمى VBO-MI (التحسين البايزي المتغير باستخدام تقدير المعلومات المتبادلة) الذي يستخدم نهجًا قائمًا بالكامل على التدرجات لتحسين بايزي، مما يعالج التحديات المتعلقة بأخذ عينات ما بعد التوزيع المكلفة وتحسين دالة الاكتساب في الشبكات العصبية البايزية التقليدية.

Se ha presentado un nuevo marco llamado VBO-MI (Optimización Bayesiana Variacional con Estimación de Información Mutua), que utiliza un enfoque completamente basado en gradientes para la optimización bayesiana, abordando los desafíos del muestreo posterior costoso y la optimización de la función de adquisición en redes neuronales bayesianas tradicionales.

Un nouveau cadre nommé VBO-MI (Optimisation Bayésienne Variationnelle avec Estimation de l'Information Mutuelle) a été introduit, utilisant une approche entièrement basée sur le gradient pour l'optimisation bayésienne, répondant aux défis de l'échantillonnage postérieur coûteux et de l'optimisation de la fonction d'acquisition dans les réseaux de neurones bayésiens traditionnels.

A new framework named VBO-MI (Variational Bayesian Optimization with Mutual Information) has been introduced, which utilizes a fully gradient-based approach to Bayesian optimization, addressing the challenges of expensive posterior sampling and acquisition function optimization in traditional Bayesian neural networks.

VBO-MI: A Fully Gradient-Based Bayesian Optimization Framework Using Variational Mutual Information Estimation

arXiv:2503.02259v2 Announce Type: replace 
Abstract: Gaussian Processes (GPs) are flexible, nonparametric Bayesian models widely used for regression and classification because of their ability to capture complex data patterns and quantify predictive uncertainty. However, the O(n^3) computational cost of kernel matrix operations poses a major obstacle to applying GPs at scale. HiGP is a high-performance Python package designed to overcome these scalability limitations through advanced numerical linear algebra and hierarchical kernel representations. It integrates H^2 matrices to achieve near-linear complexity in both storage and computation for spatial datasets, supports on-the-fly kernel evaluation to avoid explicit storage in large-scale problems, and incorporates a robust Adaptive Factorized Nystr\"om (AFN) preconditioner that accelerates convergence of iterative solvers across a broad range of kernel spectra. These computational kernels are implemented in C++ for maximum performance and exposed through Python interfaces, enabling seamless integration with modern machine learning workflows. HiGP also includes analytically derived gradient computations for efficient hyperparameter optimization, avoiding the inefficiencies of automatic differentiation in iterative solvers. By serving as a reusable numerical engine, HiGP complements existing GP frameworks such as GPJax, KeOps, and GaussianProcesses.jl, providing a reliable and scalable computational backbone for large-scale Gaussian Process regression and classification.

HiGP هو حزمة بايثون عالية الأداء تم تقديمها حديثًا تهدف إلى تحسين قابلية التوسع لعمليات Gaussian (GP)، والتي تُستخدم على نطاق واسع في مهام الانحدار والتصنيف نظرًا لمرونتها وقدرتها على قياس عدم اليقين. تتناول الحزمة التحديات الحسابية الكبيرة المرتبطة بعمليات مصفوفة النواة، والتي تقليديًا لها تعقيد مكعب.

HiGP es un nuevo paquete de Python de alto rendimiento diseñado para mejorar la escalabilidad de los Procesos Gaussianos (GP), que son ampliamente utilizados para tareas de regresión y clasificación debido a su flexibilidad y capacidad para cuantificar la incertidumbre. El paquete aborda los desafíos computacionales significativos asociados con las operaciones de matriz de núcleo, que tradicionalmente tienen una complejidad cúbica.

HiGP est un nouveau package Python haute performance conçu pour améliorer l'évolutivité des processus gaussiens (GP), largement utilisés pour des tâches de régression et de classification en raison de leur flexibilité et de leur capacité à quantifier l'incertitude. Le package s'attaque aux défis computationnels significatifs associés aux opérations de matrice noyau, qui ont traditionnellement une complexité cubique.

HiGP is a newly introduced high-performance Python package designed to enhance the scalability of Gaussian Processes (GPs), which are widely utilized for regression and classification tasks due to their flexibility and ability to quantify uncertainty. The package addresses the significant computational challenges associated with kernel matrix operations, which traditionally have a cubic complexity.

Streaming Generated Gaussian Process Experts for Online Learning and Control: Extended Version

Was this article worth reading? Share it

LucidQuery AI

GPTHumanizer

Aqaba.ai

Gonogo by Trampoline

NoFilterGPT

Graphite Note

Ready to build your own newsroom?