arXiv:2605.20533v1 Announce Type: new 
Abstract: Optimization algorithms are core methods by which machine learning models iteratively minimize loss functions, update parameters, learn from data, and improve performance. Momentum SGD and AdamW represent two important optimization paradigms. AdamW produces stable updates and usually has strong robustness across training scenarios, but its generalization performance is sometimes weaker than that of momentum methods. Momentum SGD can often obtain better generalization after careful tuning, but it is more sensitive to gradient-scale variation and hyperparameter settings. To balance the strengths and weaknesses of the two paradigms, this paper proposes Ada2MS, an optimization algorithm that achieves a smooth transition between AdamW-like behavior and momentum-SGD-like behavior through continuous exponential interpolation between elementwise second-moment estimates and global second-moment estimates. On the visual tasks evaluated in this study, Ada2MS obtains competitive results under a unified optimizer-comparison protocol. The code will be released at https://github.com/mengzhu0308/Ada2MS

يقدم المقال Ada2MS، وهو خوارزمية تحسين هجينة تجمع بين نقاط القوة في AdamW وMomentum SGD من خلال استخدام التداخل الأسي المستمر بين تقديرات اللحظة الثانية العنصرية والعالمية. تهدف هذه الطريقة إلى تحسين أداء نماذج التعلم الآلي من خلال توفير تحديثات مستقرة مع تحسين قدرات التعميم.

El artículo presenta Ada2MS, un algoritmo de optimización híbrido que combina las fortalezas de AdamW y Momentum SGD mediante la interpolación exponencial continua entre estimaciones de segundo momento elementales y globales. Este enfoque busca mejorar el rendimiento de los modelos de aprendizaje automático al proporcionar actualizaciones estables y mejorar las capacidades de generalización.

L'article présente Ada2MS, un algorithme d'optimisation hybride qui combine les forces d'AdamW et de Momentum SGD en utilisant une interpolation exponentielle continue entre les estimations de second moment élément par élément et globales. Cette approche vise à améliorer les performances des modèles d'apprentissage automatique en fournissant des mises à jour stables tout en améliorant les capacités de généralisation.

The paper introduces Ada2MS, a hybrid optimization algorithm that combines the strengths of AdamW and Momentum SGD by employing continuous exponential interpolation between elementwise and global second-moment estimates. This approach aims to enhance the performance of machine learning models by providing stable updates while improving generalization capabilities.