MCP-RiskCue: Can LLM Infer Risk Information From MCP Server System Logs?

The MCP-RiskCue study addresses significant security concerns associated with the Model Context Protocol (MCP) server systems, particularly when they are compromised. By generating 1,800 synthetic system logs and analyzing 2,421 chat histories, the research evaluates the ability of various large language models (LLMs) to detect risks. The results reveal that smaller models often fail to identify risky logs, resulting in high false negatives, while models trained with Reinforcement Learning from Verifiable Reward demonstrate a better balance between precision and recall. This research is vital as it sheds light on the vulnerabilities in LLM-MCP interactions, emphasizing the need for robust detection mechanisms to safeguard against potential threats.