arXiv:2510.25744v2 Announce Type: replace 
Abstract: Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems, where human goals are often underspecified and evolve. We argue for a shift from building and assessing task completion agents to developing collaborative agents, assessed not only by the quality of their final outputs but by how well they engage with and enhance human effort throughout the problem-solving process. To support this shift, we introduce collaborative effort scaling, a framework that captures how an agent's utility grows with increasing user involvement. Through case studies and simulated evaluations, we show that state-of-the-art agents often underperform in multi-turn, real-world scenarios, revealing a missing ingredient in agent design: the ability to sustain engagement and scaffold user understanding. Collaborative effort scaling offers a lens for diagnosing agent behavior and guiding development toward more effective interactions.

تسلط ورقة بحثية حديثة الضوء على الحاجة إلى الانتقال من تقييم الوكلاء بناءً على إكمال المهام فقط إلى التركيز على قدراتهم التعاونية. هذا التغيير مهم لأن العديد من المشكلات في العالم الحقيقي تتطلب تفاعلًا مستمرًا وتكيفًا، مما يعكس الطبيعة الديناميكية للأهداف البشرية. من خلال إعطاء الأولوية للتعاون، يمكننا تطوير وكلاء لا ينتجون نتائج أفضل فحسب، بل يعززون أيضًا التفاعل البشري، مما يجعلهم أكثر فعالية في البيئات المعقدة.

Un artículo reciente destaca la necesidad de cambiar la evaluación de los agentes, que actualmente se centra en la finalización de tareas, hacia un enfoque en sus habilidades colaborativas. Este cambio es crucial porque muchos problemas del mundo real requieren interacción continua y adaptación, reflejando la naturaleza dinámica de los objetivos humanos. Al priorizar la colaboración, podemos desarrollar agentes que no solo produzcan mejores resultados, sino que también mejoren el compromiso humano, haciéndolos más efectivos en entornos complejos.

Un article récent souligne la nécessité de passer de l'évaluation des agents uniquement sur la réalisation des tâches à la concentration sur leurs capacités collaboratives. Ce changement est crucial car de nombreux problèmes du monde réel nécessitent une interaction continue et une adaptation, reflétant la nature dynamique des objectifs humains. En privilégiant la collaboration, nous pouvons développer des agents qui non seulement produisent de meilleurs résultats, mais améliorent également l'engagement humain, les rendant plus efficaces dans des environnements complexes.

A recent paper highlights the need to shift from evaluating agents solely on task completion to focusing on their collaborative abilities. This change is crucial because many real-world problems require ongoing interaction and adaptation, reflecting the dynamic nature of human goals. By prioritizing collaboration, we can develop agents that not only produce better outcomes but also enhance human engagement, making them more effective in complex environments.

Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents

arXiv:2411.00365v2 Announce Type: replace 
Abstract: In the paradigm of decentralized learning, a group of agents collaborate to learn a global model using a distributed dataset without a central server; nevertheless, it is severely challenged by the heterogeneity of the data distribution across the agents. For example, the data may be distributed non-independently and identically, and even be noised or poisoned. To address these data challenges, we propose ROSS, a novel robust decentralized stochastic learning algorithm based on Shapley values, in this paper. Specifically, in each round, each agent aggregates the cross-gradient information from its neighbors, i.e., the derivatives of its local model with respect to the datasets of its neighbors, to update its local model in a momentum like manner, while we innovate in weighting the derivatives according to their contributions measured by Shapley values. We perform solid theoretical analysis to reveal the linear convergence speedup of our ROSS algorithm. We also verify the efficacy of our algorithm through extensive experiments on public datasets. Our results demonstrate that, in face of the above variety of data challenges, our ROSS algorithm has significant advantages over existing state-of-the-art proposals in terms of both convergence and prediction accuracy.

تم اقتراح خوارزمية جديدة للتعلم اللامركزي تُدعى ROSS، والتي تستخدم قيم شابلي لتعزيز قوة التعلم العشوائي بين الوكلاء. تتناول هذه الطريقة التحديات التي تطرحها توزيعات البيانات غير المتجانسة، مما يسمح للوكلاء بتعلم نموذج عالمي بشكل تعاوني دون الحاجة إلى خادم مركزي. يقوم كل وكيل بتحديث نموذجه من خلال تجميع معلومات التدرج المتقاطع من الوكلاء المجاورين، مع وزنها وفقًا لمساهماتهم.

Se ha propuesto un nuevo algoritmo de aprendizaje descentralizado llamado ROSS, que utiliza valores de Shapley para mejorar la robustez del aprendizaje estocástico entre agentes. Este enfoque aborda los desafíos planteados por las distribuciones de datos heterogéneas, permitiendo que los agentes aprendan de manera colaborativa un modelo global sin un servidor central. Cada agente actualiza su modelo agregando información de gradiente cruzado de los agentes vecinos, ponderada por sus contribuciones.

Un nouvel algorithme d'apprentissage décentralisé nommé ROSS a été proposé, utilisant les valeurs de Shapley pour améliorer la robustesse de l'apprentissage stochastique parmi les agents. Cette approche répond aux défis posés par les distributions de données hétérogènes, permettant aux agents d'apprendre de manière collaborative un modèle global sans serveur central. Chaque agent met à jour son modèle en agrégeant des informations de gradient croisé provenant des agents voisins, pondérées par leurs contributions.

A new decentralized learning algorithm named ROSS has been proposed, which utilizes Shapley values to enhance the robustness of stochastic learning among agents. This approach addresses challenges posed by heterogeneous data distributions, allowing agents to collaboratively learn a global model without a central server. Each agent updates its model by aggregating cross-gradient information from neighboring agents, weighted by their contributions.

ROSS: RObust decentralized Stochastic learning based on Shapley values

One More Thing in AI – Your Shortcut to AI Mastery

Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents

Was this article worth reading? Share it

One More Thing in AI

KushoAI

Earmark

Teammately

Questflow

Envelope

Ready to build your own newsroom?