arXiv — cs.LG

The introduction of TF-Engram marks a significant advancement in the field of large language models (LLMs), providing a train-free Engram system that enhances memory storage and retrieval efficiency. This system utilizes SSD-backed memory and Early-Exit Guided Predictive Prefetching to improve performance during autoregressive decoding, resulting in a notable increase in downstream task scores for the Qwen3-0.6B model.

The introduction of Continuous-Query Limited Memory Language Models (CO-LMLM) marks a significant advancement in the field of artificial intelligence, allowing models to externalize factual knowledge to a knowledge base instead of relying solely on internal weights. This innovative approach enables the generation of flexible vector queries, integrating human-readable knowledge into outputs while overcoming limitations of traditional language models.

A new pruning method called PALS (Percentile-Aware Layerwise Sparsity) has been introduced, which adjusts the sparsity ratio for each layer of a transformer model based on the 99th percentile of activation magnitudes, achieving improved performance on LLaMA-2-7B compared to uniform methods like Wanda.

A recent study has introduced the CowCorpus dataset, which comprises 400 real-user web navigation trajectories, revealing four distinct patterns of human interaction with web agents: hands-off supervision, hands-on oversight, collaborative task-solving, and full user takeover. This research aims to model human intervention to enhance collaborative web task execution.

A recent review on Large Language Models (LLMs) highlights their evolving reasoning capabilities, emphasizing the need for adaptive strategies that range from fast, intuitive responses to slow, deliberate reasoning and tool-augmented thinking. This taxonomy draws from cognitive psychology to categorize LLM reasoning methods based on internal and external knowledge boundaries.

A novel data-driven pipeline has been proposed to monitor the emergence of transformative technologies by leveraging Large Language Models (LLMs) to extract semantic triples from unstructured text, thereby constructing a graph of technology-related entities and their relations. This method aims to address the challenges faced by traditional expert-based forecasting methods in rapidly evolving fields like Information and Communication Technologies (ICTs).

A recent study on multi-teacher on-policy distillation highlights the importance of behavior leverage imbalance in training agentic language models, which must learn to effectively utilize tools and provide direct responses. The research indicates that while generalized knowledge distillation can enhance tool-call recall, it may also lead to over-calling, where models incorrectly opt for tool usage instead of direct answers.

An empirical study has been conducted to evaluate the relevance of various Retrieval-Augmented Generation (RAG) metrics, utilizing a question-answering dataset derived from business data. The experiment involved scoring generated responses and retrieved spans using metrics from four libraries, comparing these scores to those from human evaluators and standard metrics like recall. Limitations of the methodology were also discussed, alongside suggestions for future research directions.

A novel framework called Dual Path Attribution (DPA) has been introduced to enhance the efficiency of attribution for SwiGLU Transformers, enabling effective information flow tracing through a single forward and backward pass without counterfactual examples. This method achieves O(1) time complexity, significantly improving computational efficiency in large language models (LLMs).

SynthAVE has been introduced as a scalable synthetic labeling framework for e-commerce, utilizing large language models (LLMs) to generate human-validated labels for attribute value extraction across 12,726 products in four languages. This approach addresses the high costs associated with human labeling by implementing a multi-LLM arena framework for quality control.

A recent study introduces EvoSOP, a framework designed to enhance the capabilities of Large Language Model (LLM) agents by transforming granular atomic actions into reusable Standard Operating Procedures (SOPs). This approach aims to reduce the reasoning overhead and failure rates associated with static toolsets, enabling agents to self-evolve through a systematic lifecycle of tool optimization.

A recent study titled 'The Blind Curator' reveals that self-evolving agents, which typically retire ineffective skills by observing failures, are hindered when a biased judge is involved. This bias prevents accurate skill retirement, leading to a deterioration in the agent's performance as it fails to recognize its shortcomings.