arXiv:2509.16959v4 Announce Type: replace-cross 
Abstract: When different objectives conflict with each other in multi-task learning, gradients begin to interfere and slow convergence, thereby potentially reducing the final model's performance. To address this, we introduce SON-GOKU, a scheduler that computes gradient interference, constructs an interference graph, and then applies greedy graph-coloring to partition tasks into groups that align well with each other. At each training step, only one group (color class) of tasks are activated, and the grouping partition is constantly recomputed as task relationships evolve throughout training. By ensuring that each mini-batch contains only tasks that pull the model in the same direction, our method improves the effectiveness of any underlying multi-task learning optimizer without additional tuning. Since tasks within these groups will update in compatible directions, multi-task learning will improve model performance rather than impede it. Empirical results on six different datasets show that this interference-aware graph-coloring approach consistently outperforms baselines and state-of-the-art multi-task optimizers. We provide extensive theory showing why grouping and sequential updates improve multi-task learning, with guarantees on descent, convergence, and accurately identifying what tasks conflict or align.

تم تقديم طريقة جديدة تُدعى SON-GOKU لتحسين التعلم متعدد المهام من خلال معالجة تداخل التدرجات الذي يمكن أن يعيق أداء النموذج. يقوم هذا المجدول بحساب تداخل التدرجات، وبناء رسم بياني للتداخل، واستخدام تلوين الرسوم البيانية الجشع لتجميع المهام التي تتماشى بشكل جيد، حيث يتم تفعيل مجموعة واحدة فقط في كل خطوة تدريب.

Se ha introducido un nuevo método llamado SON-GOKU para mejorar el aprendizaje multitarea al abordar la interferencia de gradientes que puede obstaculizar el rendimiento del modelo. Este programador calcula la interferencia de gradientes, construye un grafo de interferencia y utiliza el coloreado de grafos codicioso para agrupar tareas que se alinean bien, activando solo un grupo en cada paso de entrenamiento.

Une nouvelle méthode appelée SON-GOKU a été introduite pour améliorer l'apprentissage multi-tâches en s'attaquant à l'interférence des gradients qui peut nuire à la performance des modèles. Ce planificateur calcule l'interférence des gradients, construit un graphe d'interférence et utilise le coloriage de graphe glouton pour regrouper les tâches qui s'alignent bien, n'activant qu'un seul groupe à chaque étape d'entraînement.

A new method called SON-GOKU has been introduced to enhance multi-task learning by addressing gradient interference that can hinder model performance. This scheduler computes gradient interference, constructs an interference graph, and utilizes greedy graph-coloring to group tasks that align well, activating only one group at each training step.

Graph Coloring for Multi-Task Learning

Was this article worth reading? Share it

ChatOne

Genfoo

SET.DO