Prompt-Based Value Steering of Large Language Models

A new reference-free metric called ConCISE has been introduced to evaluate the conciseness of responses generated by large language models (LLMs). This metric addresses the issue of verbosity in LLM outputs, which often contain unnecessary details that can hinder clarity and user satisfaction. ConCISE calculates conciseness through various compression ratios and word removal techniques without relying on standard reference responses.

ToolHaystack has been introduced as a benchmark for evaluating the long-term interaction capabilities of large language models (LLMs) in realistic contexts, highlighting their performance in maintaining context and handling disruptions during extended conversations. This benchmark reveals significant gaps in the robustness of current models, which perform well in standard multi-turn settings but struggle under the conditions set by ToolHaystack.