Self-HarmLLM: Can Large Language Model Harm Itself?

The study 'Self-HarmLLM' investigates a novel scenario where Large Language Models (LLMs) might inadvertently harm themselves by generating Mitigated Harmful Queries (MHQs). These ambiguous queries maintain their original intent while concealing their harmful nature. Experiments conducted on models such as GPT-3.5-turbo and LLaMA3-8B-instruct revealed alarming success rates: up to 65% for transformation and 41% for jailbreak in Few-shot conditions. The findings highlight a critical gap in current defenses, which typically assume external threats rather than internal vulnerabilities. Automated evaluations overestimated jailbreak success by an average of 52%, indicating a need for more robust assessment methods. This research underscores the importance of addressing the self-harming potential of LLMs, as it poses significant implications for AI safety and the reliability of automated systems.