Structured Uncertainty guided Clarification for LLM Agents

The recent introduction of the SAGE-Agent marks a significant advancement in the capabilities of large language model (LLM) agents, particularly in handling ambiguous user instructions. By employing a structured uncertainty approach, the SAGE-Agent effectively increases task coverage by 7-39% while simultaneously reducing the number of clarification questions by 1.5-2.7 times compared to existing methods. This improvement is crucial as it enhances task success rates and overall interaction efficiency across various domains, including document editing, vehicle control, and travel booking. Additionally, the development of ClarifyBench, a benchmark for multi-turn tool-augmented disambiguation, provides a realistic simulation environment for evaluating LLM-based user interactions. The structured uncertainty framework also boosts the accuracy of the When2Call model significantly, demonstrating its potential for effective reinforcement learning applications. Overall, these advancements highl…