arXiv:2506.09562v3 Announce Type: replace-cross 
Abstract: Deep reinforcement learning (DRL) has achieved remarkable success in a wide range of sequential decision-making applications, including robotics, healthcare, smart grids, and finance. Recent studies reveal that adversaries can implant backdoors into DRL agents during the training phase. These backdoors can later be activated by specific triggers during deployment, compelling the agent to execute targeted actions and potentially leading to severe consequences, such as drone crashes or vehicle collisions. However, existing backdoor attacks utilize simplistic and heuristic trigger configurations, overlooking the critical impact of trigger design on attack effectiveness. To address this gap, we introduce TooBadRL, the first framework to systematically optimize DRL backdoor triggers across three critical aspects: injection timing, trigger dimension, and manipulation magnitude. Specifically, we first introduce a performance-aware adaptive freezing mechanism to determine the injection timing during training. Then, we formulate trigger selection as an influence attribution problem and apply Shapley value analysis to identify the most influential trigger dimension for injection. Furthermore, we propose an adversarial input synthesis method to optimize the manipulation magnitude under environmental constraints. Extensive evaluations on three DRL algorithms and nine benchmark tasks demonstrate that TooBadRL outperforms five baseline methods in terms of attack success rate while only slightly affecting normal task performance. We further evaluate potential defense strategies from detection and mitigation perspectives. We open-source our code to facilitate reproducibility and further research.

يتناول المقال TooBadRL، وهو إطار جديد مصمم لتحسين تكوينات الزناد لهجمات الباب الخلفي على وكلاء التعلم العميق المعزز (DRL). يمكن زرع هذه الأبواب الخلفية خلال مرحلة التدريب وتفعيلها لاحقًا، مما قد يؤدي إلى عواقب ضارة. يركز الإطار على ثلاثة جوانب رئيسية: توقيت الحقن، أبعاد الزناد، ودرجة التلاعب، بهدف تحسين فعالية هذه الهجمات، التي لها تداعيات كبيرة في تطبيقات متنوعة مثل الروبوتات والتمويل.

El artículo discute TooBadRL, un nuevo marco diseñado para optimizar las configuraciones de desencadenadores para ataques de puerta trasera en agentes de aprendizaje por refuerzo profundo (DRL). Estas puertas traseras pueden ser implantadas durante la fase de entrenamiento y activadas posteriormente, lo que puede llevar a consecuencias potencialmente dañinas. El marco se centra en tres aspectos clave: el momento de inyección, la dimensión del desencadenador y la magnitud de la manipulación, con el objetivo de mejorar la efectividad de dichos ataques, que tienen implicaciones significativas en …

L'article traite de TooBadRL, un nouveau cadre conçu pour optimiser les configurations de déclencheurs pour les attaques par porte dérobée sur les agents d'apprentissage par renforcement profond (DRL). Ces portes dérobées peuvent être implantées pendant l'entraînement et activées plus tard, entraînant des conséquences potentiellement nuisibles. Le cadre se concentre sur trois aspects clés : le moment de l'injection, la dimension du déclencheur et l'ampleur de la manipulation, visant à améliorer l'efficacité de ces attaques, qui ont des implications significatives dans divers domaines tels que …

The article discusses TooBadRL, a novel framework designed to optimize trigger configurations for backdoor attacks on deep reinforcement learning (DRL) agents. These backdoors can be implanted during training and activated later, leading to potentially harmful outcomes. The framework focuses on three key aspects: injection timing, trigger dimension, and manipulation magnitude, aiming to enhance the effectiveness of such attacks, which have significant implications in various applications like robotics and finance.

TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning

Was this article worth reading? Share it

Tombot Spark

Guidejar-4eb95b

AskTuring