arXiv:2510.01621v2 Announce Type: replace 
Abstract: We investigate the phenomenon of posterior collapse in variational autoencoders (VAEs) from the perspective of statistical physics, and reveal that it constitutes a phase transition governed jointly by data structure and model hyper-parameters. By analyzing the stability of the trivial solution associated with posterior collapse, we identify a critical hyper-parameter threshold. In particular, we derive an explicit criterion for the onset of collapse: posterior collapse occurs when the decoder variance exceeds the largest eigenvalue of the data covariance matrix. This critical boundary, separating meaningful latent inference from collapse, is characterized by a discontinuity in the KL divergence between the approximate posterior and the prior distribution, where the KL divergence and its derivatives exhibit clear non-analytic behavior. We validate this critical behavior on both synthetic and real-world datasets, confirming the existence of a phase transition. The experimental results align well with our theoretical predictions, demonstrating the robustness of our collapse criterion across various VAE architectures. Our stability-based analysis demonstrate that posterior collapse is not merely an optimization failure, but rather an emerging phase transition arising from the interplay between data structure and variational constraints. This perspective offers new insights into the trainability and representational capacity of deep generative models.

قام الباحثون بدراسة ظاهرة الانهيار الخلفي في أجهزة التشفير التلقائي المتغيرة (VAEs) من منظور الفيزياء الإحصائية، وكشفوا أنها تشكل انتقالًا في الطور يتأثر بهيكل البيانات ومعلمات النموذج. وقد حددوا عتبة حرجة للمعلمات، موضحين أن الانهيار الخلفي يحدث عندما تتجاوز تباين المفكك أكبر قيمة ذاتية لمصفوفة التباين البياني.

Investigadores han analizado el fenómeno del colapso posterior en autoencoders variacionales (VAEs) desde la perspectiva de la física estadística, revelando que constituye una transición de fase influenciada por la estructura de los datos y los hiperparámetros del modelo. Identificaron un umbral crítico para los hiperparámetros, estableciendo que el colapso posterior ocurre cuando la varianza del decodificador supera el mayor valor propio de la matriz de covarianza de los datos.

Des chercheurs ont examiné le phénomène de l'effondrement postérieur dans les autoencodeurs variationnels (VAEs) à travers le prisme de la physique statistique, révélant qu'il s'agit d'une transition de phase influencée par la structure des données et les hyper-paramètres du modèle. Ils ont identifié un seuil critique pour les hyper-paramètres, établissant que l'effondrement postérieur se produit lorsque la variance du décodeur dépasse la plus grande valeur propre de la matrice de covariance des données.

Researchers have investigated posterior collapse in variational autoencoders (VAEs) through the lens of statistical physics, revealing it as a phase transition influenced by data structure and model hyper-parameters. They identified a critical threshold for hyper-parameters, establishing that posterior collapse occurs when decoder variance surpasses the largest eigenvalue of the data covariance matrix.

Posterior Collapse as a Phase Transition in Variational Autoencoders

Was this article worth reading? Share it

LucidQuery AI

The Visualizer

Videotok

Videolulu

Omnifact

Dynamiq

Ready to build your own newsroom?