To Err Is Human: Systematic Quantification of Errors in Published AI Papers via LLM Analysis

A recent study evaluated the effectiveness of deep learning models and large language models (LLMs) for vulnerability detection, focusing on models like ReVeal and LineVul across four datasets: Juliet, Devign, BigVul, and ICVul. The research highlights the gap between benchmark performance and real-world applicability, emphasizing the need for systematic evaluation in practical scenarios.

A new study introduces the DeepNews Framework, which aims to overcome the limitations of large language models (LLMs) in long-form text generation by addressing the 'Statistical Smoothing Trap.' This framework incorporates cognitive processes similar to those of expert financial journalists, enhancing the quality of generated content.

A recent study has examined the vulnerability of Large Language Model (LLM)-based scientific reviewers to indirect prompt injection, focusing on the potential to alter peer review decisions from 'Reject' to 'Accept'. This research introduces a new metric, the Weighted Adversarial Vulnerability Score (WAVS), and evaluates 15 attack strategies across 13 LLMs, including GPT-5 and DeepSeek, using a dataset of 200 scientific papers.

TheMCPCompany has introduced a benchmark for evaluating tool-calling agents that utilize the Model Context Protocol (MCP) to interact with various real-world services, significantly expanding the tool sets available for Large Language Models (LLMs). This initiative aims to enhance the performance and cost-effectiveness of these agents by leveraging over 18,000 tools through REST APIs.