Multiplicative Reweighting for Robust Neural Network Optimization

The introduction of Multiplicative Reweighting (MW) for neural network optimization addresses a critical challenge: the degradation of model performance due to noisy labels during training. By leveraging principles from learning with expert advice, MW updates have been theoretically established to converge when used with gradient descent, proving advantageous in one-dimensional cases. Empirical validation on datasets such as CIFAR-10, CIFAR-100, and Clothing1M reveals that MW significantly enhances the accuracy of neural networks in the presence of label noise. Furthermore, the method shows promise in improving adversarial robustness, which is increasingly vital in the context of machine learning applications facing real-world data challenges. This advancement not only contributes to the field of artificial intelligence but also sets a foundation for more resilient neural network architectures in various applications.