arXiv:2603.11331v3 Announce Type: replace 
Abstract: Adversarial attacks can reliably steer safety-aligned large language models toward unsafe behavior. Empirically, we find that adversarial prompt-injection attacks can amplify attack success rate from the slow polynomial growth observed without injection to exponential growth with the number of inference-time samples. We first identify a minimal statistical mechanism for these two regimes by giving a small set of assumptions on the distribution of safe generation across contexts under which both scaling laws follow. To explain this phenomenon further, we propose a theoretical generative model of proxy language in terms of a spin-glass system operating in a replica-symmetry-breaking regime, where generations are drawn from the associated Gibbs measure and a subset of low-energy, size-biased clusters is designated unsafe. We analytically show how this model naturally realizes the minimal assumptions. Short injected prompts correspond to a weak magnetic field aligned towards unsafe cluster centers and yield a power-law scaling of attack success rate with the number of inference-time samples, while long injected prompts, i.e., strong magnetic field, yield exponential scaling. We observe qualitatively consistent behavior across a broad range of large language models, spanning parameter scales from 3B to 70B. In particular, the main trends remain stable across multiple attack methods, such as GCG and AutoDAN, as well as across benchmark datasets such as AdvBench and HarmBench.

دراسة حديثة نُشرت على arXiv تسلط الضوء على ضعف النماذج اللغوية الكبيرة (LLMs) أمام الهجمات العدائية، وخاصة من خلال تقنيات حقن المطالبات التي يمكن أن تزيد بشكل كبير من احتمالية النتائج غير الآمنة. تحدد الأبحاث تقاطعًا متعدد الحدود-أسي في قوانين القياس، مما يشير إلى أن فعالية هذه الهجمات تنمو بشكل أسي مع عدد عينات وقت الاستدلال.

Un estudio reciente publicado en arXiv destaca las vulnerabilidades de los grandes modelos de lenguaje (LLMs) ante ataques adversariales, específicamente a través de técnicas de inyección de prompts que pueden aumentar significativamente la probabilidad de resultados inseguros. La investigación identifica un cruce polinómico-exponencial en las leyes de escalado, sugiriendo que la efectividad de estos ataques crece exponencialmente con el número de muestras en tiempo de inferencia.

Une étude récente publiée sur arXiv met en lumière les vulnérabilités des grands modèles de langage (LLMs) face aux attaques adversariales, en particulier par le biais de techniques d'injection de prompts qui peuvent augmenter considérablement la probabilité de résultats non sécurisés. La recherche identifie un croisement polynomial-exponentiel dans les lois de mise à l'échelle, suggérant que l'efficacité de ces attaques croît de manière exponentielle avec le nombre d'échantillons en temps d'inférence.

A recent study published on arXiv highlights the vulnerabilities of large language models (LLMs) to adversarial attacks, specifically through prompt-injection techniques that can significantly increase the likelihood of unsafe outputs. The research identifies a polynomial-exponential crossover in scaling laws, suggesting that the effectiveness of these attacks grows exponentially with the number of inference-time samples.