arXiv:2605.09665v1 Announce Type: new 
Abstract: Data selection is a key component of efficient instruction tuning for large language models, as recent work has shown that data quality often matters more than data quantity. Accordingly, prior studies have introduced various multi-dimensional heuristics to evaluate and filter instruction data. However, most existing methods rely on static task-agnostic and model-agnostic weighting schemes, which overlook the varying requirements of specific downstream tasks and the differing pre-existing capabilities of models. In this paper, we propose a framework for learning multi-indicator weights that jointly adapts data selection to both the downstream task and the specific model. Our method identifies optimal weight configurations without full-scale fine-tuning by utilizing in-context learning (ICL) signals on compact tiny-validation sets. These signals serve as efficient performance proxies that ensure high-fidelity evaluation at minimal computational cost. Experiments across multiple benchmarks and model families, including Mistral, Qwen, and Llama, show that the approach achieves performance comparable to or exceeding full-dataset tuning while using only 30\% of the training samples on GSM8K. Furthermore, our analysis reveals a trade-off between semantic diversity and logical complexity in reasoning tasks, highlighting the necessity of joint task-model adaptation.

تم اقتراح إطار عمل جديد لتعلم أوزان متعددة المؤشرات لتحسين اختيار البيانات لنماذج اللغة الكبيرة (LLMs)، مع التركيز على تكييف اختيار البيانات مع المهام والنماذج المحددة. تستخدم هذه الطريقة إشارات التعلم في السياق على مجموعات التحقق المدمجة لتحديد تكوينات الوزن المثلى دون الحاجة إلى ضبط دقيق موسع.

Se ha propuesto un nuevo marco para aprender pesos multiindicadores que mejoren la selección de datos para grandes modelos de lenguaje (LLMs), centrándose en adaptar la selección de datos a tareas y modelos específicos. Este enfoque utiliza señales de aprendizaje en contexto en conjuntos de validación compactos para identificar configuraciones de peso óptimas sin necesidad de un ajuste fino extenso.

Un nouveau cadre a été proposé pour apprendre des poids multi-indicateurs afin d'améliorer la sélection de données pour les grands modèles de langage (LLMs), en mettant l'accent sur l'adaptation de la sélection de données aux tâches et modèles spécifiques en aval. Cette approche utilise des signaux d'apprentissage en contexte sur des ensembles de validation compacts pour identifier des configurations de poids optimales sans ajustement fin étendu.

A new framework has been proposed for learning multi-indicator weights to enhance data selection for large language models (LLMs), focusing on adapting data selection to specific downstream tasks and models. This approach utilizes in-context learning signals on compact validation sets to identify optimal weight configurations without extensive fine-tuning.