arXiv:2405.16441v4 Announce Type: replace-cross 
Abstract: We introduce Statistical Flow Matching (SFM), a novel and mathematically rigorous flow-matching framework on the manifold of parameterized probability measures inspired by the results from information geometry. We demonstrate the effectiveness of our method on the discrete generation problem by instantiating SFM on the manifold of categorical distributions whose geometric properties remain unexplored in previous discrete generative models. Utilizing the Fisher information metric, we equip the manifold with a Riemannian structure whose intrinsic geometries are effectively leveraged by following the shortest paths of geodesics. We develop an efficient training and sampling algorithm that overcomes numerical stability issues with a diffeomorphism between manifolds. Our distinctive geometric perspective of statistical manifolds allows us to apply optimal transport during training and interpret SFM as following the steepest direction of the natural gradient. Unlike previous models that rely on variational bounds for likelihood estimation, SFM enjoys the exact likelihood calculation for arbitrary probability measures. We manifest that SFM can learn more complex patterns on the statistical manifold where existing models often fail due to strong prior assumptions. Comprehensive experiments on real-world generative tasks ranging from image, text to biological domains further demonstrate that SFM achieves higher sampling quality and likelihood than other discrete diffusion or flow-based models.

تم تقديم إطار عمل جديد يسمى Statistical Flow Matching (SFM) يركز على مطابقة التدفقات ضمن مجموعة من مقاييس الاحتمال المعلمة. تستفيد هذه الطريقة من مقياس معلومات فيشر لإنشاء هيكل ريمان، مما يتيح خوارزميات تدريب وعينة فعالة تتعامل مع مشكلات الاستقرار العددي في النماذج التوليدية المنفصلة.

Se ha introducido un nuevo marco llamado Statistical Flow Matching (SFM), que se centra en el emparejamiento de flujos dentro de la variedad de medidas de probabilidad parametrizadas. Este método aprovecha la métrica de información de Fisher para crear una estructura riemanniana, lo que permite algoritmos de entrenamiento y muestreo eficientes que abordan problemas de estabilidad numérica en modelos generativos discretos.

Un nouveau cadre appelé Statistical Flow Matching (SFM) a été introduit, se concentrant sur l'appariement de flux au sein de la variété des mesures de probabilité paramétrées. Cette méthode exploite la métrique d'information de Fisher pour créer une structure riemannienne, permettant des algorithmes d'entraînement et d'échantillonnage efficaces qui traitent les problèmes de stabilité numérique dans les modèles génératifs discrets.

A new framework called Statistical Flow Matching (SFM) has been introduced, focusing on flow matching within the manifold of parameterized probability measures. This method leverages the Fisher information metric to create a Riemannian structure, enabling efficient training and sampling algorithms that address numerical stability issues in discrete generative models.

Categorical Flow Matching on Statistical Manifolds

Was this article worth reading? Share it

Fluig AI

Spark AI Dating

Stockfill