arXiv:2505.15557v3 Announce Type: replace-cross 
Abstract: Gaussian processes (GPs) furnish accurate nonlinear predictions with well-calibrated uncertainty. However, the typical GP setup has a built-in stationarity assumption, making it ill-suited for modeling data from processes with sudden changes, or "jumps" in the output variable. The "jump GP" (JGP) was developed for modeling data from such processes, combining local GPs and latent "level" variables under a joint inferential framework. But joint modeling can be fraught with difficulty. We aim to simplify by suggesting a more modular setup, eschewing joint inference but retaining the main JGP themes: (a) learning optimal neighborhood sizes that locally respect manifolds of discontinuity; and (b) a new cluster-based (latent) feature to capture regions of distinct output levels on both sides of the manifold. We show that each of (a) and (b) separately leads to dramatic improvements when modeling processes with jumps. In tandem (but without requiring joint inference) that benefit is compounded, as illustrated on real and synthetic benchmark examples from the recent literature.

تقدم الورقة نهجًا معياريًا لعمليات غاوس (GP) لنمذجة البيانات ذات التغيرات المفاجئة، المعروفة باسم 'القفزات'. تقترح إطارًا مبسطًا يحتفظ بالميزات الرئيسية لجهاز القفز GP (JGP)، مع التركيز على تعلم أحجام الجوار المثلى وإدخال ميزة كامنة قائمة على التجمعات لالتقاط مستويات الإخراج المتميزة. يهدف هذا النهج إلى تحسين نمذجة العمليات التي تتميز بالانتقالات المفاجئة.

El artículo presenta un enfoque modular para los procesos gaussianos (GP) en la modelización de datos con cambios repentinos, conocidos como 'saltos'. Propone un marco simplificado que conserva características clave del jump GP (JGP), centrándose en el aprendizaje de tamaños óptimos de vecindario e introduciendo una característica latente basada en clústeres para capturar niveles de salida distintos. Este enfoque busca mejorar la modelización de procesos caracterizados por transiciones abruptas.

Cet article présente une approche modulaire des processus gaussiens (GP) pour modéliser des données avec des changements soudains, appelés 'sauts'. Il propose un cadre simplifié qui conserve les caractéristiques clés du jump GP (JGP), en se concentrant sur l'apprentissage de tailles de voisinage optimales et en introduisant une caractéristique latente basée sur des clusters pour capturer des niveaux de sortie distincts. Cette approche vise à améliorer la modélisation des processus caractérisés par des transitions abruptes.

The paper introduces a modular approach to Gaussian processes (GPs) for modeling data with sudden changes, known as 'jumps'. It proposes a simplified framework that retains key features of the jump GP (JGP), focusing on learning optimal neighborhood sizes and introducing a cluster-based latent feature to capture distinct output levels. This approach aims to enhance the modeling of processes characterized by abrupt transitions.

Modular Jump Gaussian Processes

arXiv:2505.11355v2 Announce Type: replace 
Abstract: Gaussian processes (GPs) have gained popularity as flexible machine learning models for regression and function approximation with an in-built method for uncertainty quantification. However, GPs suffer when the amount of training data is large or when the underlying function contains multi-scale features that are difficult to represent by a stationary kernel. To address the former, training of GPs with large-scale data is often performed through inducing point approximations, also known as sparse GP regression (GPR), where the size of the covariance matrices in GPR is reduced considerably through a greedy search on the data set. To aid the latter, deep GPs have gained traction as hierarchical models that resolve multi-scale features by combining multiple GPs. Posterior inference in deep GPs requires a sampling or, more usual, a variational approximation. Variational approximations lead to large-scale stochastic, non-convex optimisation problems and the resulting approximation tends to represent uncertainty incorrectly. In this work, we combine variational learning with MCMC to develop a particle-based expectation-maximisation method to simultaneously find inducing points within the large-scale data (variationally) and accurately train the deep GPs (sampling-based). The result is a highly efficient and accurate methodology for deep GP training on large-scale data. We test our method on standard benchmark problems.

تم استكشاف تقنيات نادرة للانحدار في العمليات الغاوسية العميقة (GPs) لتحسين قابلية التوسع والكفاءة لهذه النماذج، خاصة عند التعامل مع مجموعات بيانات كبيرة أو وظائف متعددة المقاييس معقدة. تبرز الأبحاث استخدام تقريبات النقاط المحورية في الانحدار GPR النادر ومزايا GPs العميقة للنمذجة الهرمية.

Se han explorado técnicas dispersas para la regresión en procesos gaussianos profundos (GPs) para mejorar la escalabilidad y eficiencia de estos modelos, especialmente al tratar con grandes conjuntos de datos o funciones complejas de múltiples escalas. La investigación destaca el uso de aproximaciones por puntos de inducción en la regresión GPR dispersa y las ventajas de los GPs profundos para la modelización jerárquica.

Des techniques éparses pour la régression dans les processus gaussiens profonds (GPs) ont été explorées pour améliorer la scalabilité et l'efficacité de ces modèles, en particulier lors du traitement de grands ensembles de données ou de fonctions multi-échelles complexes. La recherche met en avant l'utilisation d'approximations par points d'induction dans la régression GPR éparse et les avantages des GPs profonds pour la modélisation hiérarchique.

Sparse techniques for regression in deep Gaussian processes (GPs) have been explored to enhance the scalability and efficiency of these models, particularly when dealing with large datasets or complex multi-scale functions. The research highlights the use of inducing point approximations in sparse GP regression (GPR) and the advantages of deep GPs for hierarchical modeling.

Modular Jump Gaussian Processes

Was this article worth reading? Share it

ChatOne

Growthonomics

Gonogo by Trampoline