Steering Evaluation-Aware Language Models to Act Like They Are Deployed

The rapid expansion of large language models' (LLMs) capabilities—including web search, code execution, data analysis, and hypothesis generation—is outpacing critical reflection on their role in academic research. This raises questions about the implications of LLMs in various fields and the need for a more structured approach to their integration into research methodologies.

The paper introduces LLMInit, a scalable framework that integrates pre-trained large language model (LLM) embeddings into collaborative filtering (CF) models to address cold-start and data-sparse issues in recommendation systems. By employing selective initialization strategies and efficient sampling methods, LLMInit aims to enhance the performance of CF models while mitigating embedding collapse challenges associated with large LLMs.

AMS-KV introduces an adaptive Key and Value (KV) caching mechanism for multi-scale visual autoregressive transformers, addressing the challenges of excessive memory growth in next-scale predictions. The study reveals that local scale token attention enhances generation quality, while allocating minimal memory for coarsest scales stabilizes image generation. The findings emphasize the importance of cache-efficient layers in maintaining strong KV similarity across finer scales.

The article introduces LIARS' BENCH, a comprehensive testbed designed to evaluate lie detection techniques in large language models (LLMs). It consists of 72,863 examples of lies and honest responses generated by four open-weight models across seven datasets. The study reveals that existing lie detection methods often fail to identify certain types of lies, particularly when the model's deception cannot be discerned from the transcript alone, highlighting limitations in current techniques.

The paper presents SDA (Steering-Driven Distribution Alignment), a model-agnostic framework aimed at aligning large language models (LLMs) with human intent without the need for fine-tuning. As LLMs are increasingly deployed in various applications, ensuring their responses meet user expectations is crucial. SDA dynamically adjusts model output probabilities based on user-defined instructions, addressing the challenge of alignment during inference efficiently and cost-effectively.

PepThink-R1 is a generative framework designed for optimizing therapeutic cyclic peptides by integrating large language models (LLMs) with chain-of-thought (CoT) supervised fine-tuning and reinforcement learning (RL). This approach allows for interpretable design choices and enhances multiple pharmacological properties, such as lipophilicity and stability, by autonomously exploring diverse sequence variants guided by a tailored reward function.

The concept of verbalized algorithms (VAs) is introduced as a method to enhance the reliability of large language models (LLMs) in reasoning tasks. VAs break down complex tasks into simpler operations on natural language strings, allowing LLMs to function effectively within a limited scope. An example provided is verbalized sorting, which utilizes an LLM as a binary comparison oracle within a known sorting algorithm, demonstrating effectiveness in sorting and clustering tasks.

HalluClean is a new framework designed to detect and correct hallucinations in large language models (LLMs). This task-agnostic framework enhances factual reliability by breaking down the process into planning, execution, and revision stages. It utilizes minimal task-routing prompts for zero-shot generalization across various domains, demonstrating significant improvements in factual consistency across multiple tasks such as question answering and dialogue.