MMbeddings: Parameter-Efficient, Low-Overfitting Probabilistic Embeddings Inspired by Nonlinear Mixed Models

MMbeddings is a newly proposed probabilistic embedding technique that integrates classical statistical methods with contemporary deep learning frameworks. Specifically, it conceptualizes embeddings as latent random effects within a variational autoencoder, drawing inspiration from nonlinear mixed models. This methodological innovation leads to a significant reduction in the number of parameters required, enhancing parameter efficiency. Additionally, MMbeddings demonstrates a lower tendency to overfit compared to traditional embedding approaches, addressing a common challenge in model generalization. These advantages position MMbeddings as a promising advancement in embedding techniques within machine learning. The approach's foundation in well-established statistical concepts combined with modern variational inference underscores its potential for robust and efficient representation learning.