CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignment

The introduction of CONGRAD marks a pivotal advancement in the field of multilingual preference alignment for large language models (LLMs). Traditional joint training methods often suffer from negative interference due to conflicting objectives, which can degrade performance. CONGRAD addresses this issue by employing a filtering technique that selects high-quality preference samples, thereby minimizing gradient conflicts across languages. This innovative approach utilizes gradient surgery and a sublinear gradient compression strategy to enhance efficiency during training. Evaluated on the LLaMA3-8B and Gemma2-2B models across 10 languages, the results demonstrate that CONGRAD consistently outperforms strong baselines, indicating its effectiveness in improving multilingual training outcomes. The implications of this research are significant, as it opens new avenues for enhancing the performance of multilingual models, which are increasingly vital in our globalized world.