arXiv:2405.15376v3 Announce Type: replace 
Abstract: Restricted Boltzmann Machines (RBMs) are powerful tools for modeling complex systems and extracting insights from data, but their training is hindered by the slow mixing of Markov Chain Monte Carlo (MCMC) processes, especially with highly structured datasets. In this study, we build on recent theoretical advances in RBM training and focus on the stepwise encoding of data patterns into singular vectors of the coupling matrix, significantly reducing the cost of generating new samples and evaluating the quality of the model, as well as the training cost in highly clustered datasets. The learning process is analogous to the thermodynamic continuous phase transitions observed in ferromagnetic models, where new modes in the probability measure emerge in a continuous manner. We leverage the continuous transitions in the training process to define a smooth annealing trajectory that enables reliable and computationally efficient log-likelihood estimates. This approach enables online assessment during training and introduces a novel sampling strategy called Parallel Trajectory Tempering (PTT) that outperforms previously optimized MCMC methods. To mitigate the critical slowdown effect in the early stages of training, we propose a pre-training phase. In this phase, the principal components are encoded into a low-rank RBM through a convex optimization process, facilitating efficient static Monte Carlo sampling and accurate computation of the partition function. Our results demonstrate that this pre-training strategy allows RBMs to efficiently handle highly structured datasets where conventional methods fail. Additionally, our log-likelihood estimation outperforms computationally intensive approaches in controlled scenarios, while the PTT algorithm significantly accelerates MCMC processes compared to conventional methods.

قدمت دراسة نهجًا جديدًا لتدريب آلات بولتزمان المقيدة (RBMs)، حيث تعالج المشكلات المتعلقة بالخلط البطيء المرتبطة بأساليب مونت كارلو بواسطة سلاسل ماركوف (MCMC). من خلال ترميز أنماط البيانات في متجهات فردية لمصفوفة الربط، تقلل الأبحاث بشكل كبير من التكلفة الحسابية لتوليد عينات جديدة وتقييم جودة النموذج، خاصة في مجموعات البيانات المجمعة بشكل كبير.

Un estudio ha presentado un enfoque novedoso para el entrenamiento de Máquinas de Boltzmann Restringidas (RBM), abordando los problemas de mezcla lenta asociados con los métodos de Monte Carlo por Cadenas de Markov (MCMC). Al codificar patrones de datos en vectores singulares de la matriz de acoplamiento, la investigación reduce significativamente el costo computacional de generar nuevas muestras y evaluar la calidad del modelo, especialmente en conjuntos de datos altamente agrupados.

Une étude a introduit une nouvelle approche pour l'entraînement des Machines de Boltzmann Restreintes (RBM), abordant les problèmes de mélange lent associés aux méthodes de Monte Carlo par chaînes de Markov (MCMC). En encodant les motifs de données dans des vecteurs singuliers de la matrice de couplage, la recherche réduit considérablement le coût computationnel de génération de nouveaux échantillons et d'évaluation de la qualité du modèle, en particulier dans des ensembles de données fortement groupés.

A study has introduced a novel approach to training Restricted Boltzmann Machines (RBMs), addressing the slow mixing issues associated with Markov Chain Monte Carlo (MCMC) methods. By encoding data patterns into singular vectors of the coupling matrix, the research significantly reduces the computational cost of generating new samples and evaluating model quality, particularly in highly clustered datasets.

Fast training and sampling of Restricted Boltzmann Machines

arXiv:2407.20432v2 Announce Type: replace 
Abstract: Bayesian inference methods such as Markov Chain Monte Carlo (MCMC) typically require repeated computations of the likelihood function, but in some scenarios this is infeasible and alternative methods are needed. Simulation-based inference (SBI) methods address this problem by using machine learning to amortize computations. In this work, we highlight a particular synergy between the SBI method of neural likelihood estimation and the classic MCMC method of Hamiltonian Monte Carlo. We show that approximating the likelihood function with a neural network model can provide three distinct advantages: (1) amortizing the computations for MCMC; (2) providing gradients for Hamiltonian Monte Carlo, and (3) smoothing over noisy simulations resulting from numerical instabilities. We provide practical guidelines for defining a prior, sampling a training set, and evaluating convergence. The method is demonstrated in an application modeling the heliospheric transport of galactic cosmic rays, where it enables efficient inference of latent parameters in the Parker equation.

تقدم دراسة جديدة نموذج هاملتونيان مونت كارلو (HMC) البديل العصبي، الذي يستفيد من الاحتمالات العصبية لتحسين طرق الاستدلال البايزي، وخاصة سلسلة ماركوف مونت كارلو (MCMC). يتناول هذا النهج التحديات الحسابية المرتبطة بتقييمات دالة الاحتمالية من خلال استخدام تقنيات التعلم الآلي لتبسيط العملية. تظهر الطريقة مزايا كبيرة، بما في ذلك تحسين الكفاءة والموثوقية في المحاكاة.

Un nuevo estudio presenta el Hamiltoniano Monte Carlo (HMC) de sustitución neuronal, que aprovecha las probabilidades neuronales para mejorar los métodos de inferencia bayesiana, en particular la cadena de Markov Monte Carlo (MCMC). Este enfoque aborda los desafíos computacionales asociados con las evaluaciones de funciones de verosimilitud mediante técnicas de aprendizaje automático para optimizar el proceso. La metodología muestra ventajas significativas, incluida una mayor eficiencia y robustez en las simulaciones.

Une nouvelle étude présente le Hamiltonien Monte Carlo (HMC) par substitution neuronale, qui exploite les probabilités neuronales pour améliorer les méthodes d'inférence bayésienne, en particulier la chaîne de Markov Monte Carlo (MCMC). Cette approche répond aux défis computationnels associés aux évaluations de fonctions de vraisemblance en utilisant des techniques d'apprentissage automatique pour rationaliser le processus. La méthode montre des avantages significatifs, notamment une efficacité et une robustesse accrues dans les simulations.

A new study introduces Neural Surrogate Hamiltonian Monte Carlo (HMC), which leverages neural likelihoods to enhance Bayesian inference methods, particularly Markov Chain Monte Carlo (MCMC). This approach addresses the computational challenges associated with likelihood function evaluations by employing machine learning techniques to streamline the process. The method demonstrates significant advantages, including improved efficiency and robustness in simulations.

Fast training and sampling of Restricted Boltzmann Machines

Was this article worth reading? Share it

LucidQuery AI

Chattermate

Betron.io