Cost-Minimized Label-Flipping Poisoning Attack to LLM Alignment

The publication titled 'Cost-Minimized Label-Flipping Poisoning Attack to LLM Alignment' presents a theoretical analysis of vulnerabilities in large language models (LLMs), particularly focusing on the risks associated with reinforcement learning from human feedback (RLHF) and data poisoning optimization (DPO). By formulating a minimum-cost poisoning attack as a convex optimization problem, the study provides insights into how attackers can manipulate LLM policies with minimal resource expenditure. Empirical results demonstrate that existing label-flipping attacks can be enhanced through a cost-minimization post-processing method, significantly reducing the number of label flips needed while maintaining the attack's effectiveness. This research underscores fundamental vulnerabilities in current AI alignment strategies, calling for urgent attention to enhance the security of LLMs against potential data poisoning threats.