arXiv:2606.05622v1 Announce Type: new 
Abstract: Planning for real-world problems by language models often involves both world and user constraints, which may not be fully specified upfront and are progressively disclosed through interaction. However, existing benchmarks still underexplore adaptive planning under such progressively revealed dual constraints. To address this gap, we introduce AdaPlanBench, a dynamic interactive benchmark for evaluating whether Large Language Model (LLM) agents can adaptively plan and re-plan under progressively revealed world and user constraints. AdaPlanBench is built on 307 household tasks, with a scalable constraint construction pipeline that augments each task with dual constraints. At runtime, agents interact with the environment in a multi-turn protocol where hidden constraints are revealed only when the agent proposes a plan that violates them, requiring iterative plan revision under accumulating feedback. This makes planning challenging, as agents must infer and track constraints from feedback while re-planning effectively. Experiments on ten leading LLMs show that adaptive planning under dual constraints remains challenging, with the best model reaching only 67.75% accuracy. We further observe that performance degrades as more constraints accumulate, with user constraints posing a particularly large challenge and failures often stemming from weaker physical grounding and reduced effectiveness. These results establish AdaPlanBench as a testbed for dual-constrained interactive planning and highlight the challenge of reliable adaptation to dynamically revealed constraints in LLM agents.

يمثل تقديم AdaPlanBench تقدمًا كبيرًا في تقييم قدرات التخطيط التكيفي لوكلاء نماذج اللغة الكبيرة (LLM)، حيث يركز على قدرتهم على إدارة القيود المفروضة من العالم والمستخدم التي يتم الكشف عنها تدريجيًا من خلال بروتوكولات تفاعلية. تم بناء هذا المعيار على 307 مهام منزلية، مما يتطلب من الوكلاء مراجعة الخطط بشكل تكراري بناءً على التغذية الراجعة حول القيود المخفية.

La introducción de AdaPlanBench marca un avance significativo en la evaluación de las capacidades de planificación adaptativa de los agentes de Modelos de Lenguaje Grande (LLM), centrándose en su capacidad para gestionar restricciones del mundo y del usuario que se revelan progresivamente a través de protocolos interactivos. Este benchmark se basa en 307 tareas domésticas, requiriendo que los agentes revisen planes de manera iterativa en función de la retroalimentación sobre restricciones ocultas.

L'introduction d'AdaPlanBench marque une avancée significative dans l'évaluation des capacités de planification adaptative des agents de Modèle de Langage de Grande Taille (LLM), en se concentrant sur leur capacité à gérer des contraintes du monde et de l'utilisateur révélées progressivement à travers des protocoles interactifs. Ce benchmark est construit sur 307 tâches ménagères, nécessitant des agents de réviser leurs plans de manière itérative en fonction des retours sur des contraintes cachées.

The introduction of AdaPlanBench marks a significant advancement in evaluating adaptive planning capabilities of Large Language Model (LLM) agents, focusing on their ability to manage progressively revealed world and user constraints through interactive protocols. This benchmark is built on 307 household tasks, requiring agents to revise plans iteratively based on feedback from hidden constraints.