Muon$^p$: Muon with Fractional Spectral Powers

A new optimizer called Muon$^p$ has been introduced, which enhances the Muon optimizer by utilizing fractional spectral-power updates instead of fully flattening the singular spectrum. This approach allows for better adaptation by preserving singular-value information, making it a significant advancement in optimization techniques.

The development of Muon$^p$ is particularly important for applications requiring fine-tuning, as it maximizes linear improvement in loss under the Schatten $q$-norm, potentially leading to more efficient and effective machine learning models.