arXiv:2606.10487v1 Announce Type: new 
Abstract: Deploying large language models in user-facing systems requires efficient output safety filtering. Existing approaches typically rely on a separate moderation model applied after generation, which doubles inference cost and only detects violations after generation completes. We observe that the signal needed for moderation is already present in the model hidden states. Based on this, we train lightweight token-level probes that operate directly on internal activations, producing per-token safety scores that can be aggregated for both offline evaluation and online intervention. The probe reuses activations from the generator and requires no additional forward pass, enabling sub millisecond per-token safety checks inside the decoding loop. A probe applied to a single mid layer recovers most decisions of a strong guard model, acting as a low cost surrogate optimized for latency rather than accuracy. In streaming settings, it can halt or modify unsafe outputs before they are fully generated, replacing end of sequence moderation with continuous token level monitoring. Compared to post hoc and streaming guard models, our method achieves orders of magnitude lower compute overhead with minimal latency cost. We also provide a practical deployment recipe, including layer selection, aggregation strategy, probing frequency, and triggering thresholds. Finally, we show that the probe linear component corresponds to a direction in residual space, enabling both detection and activation steering at negligible cost.

دراسة حديثة نُشرت على arXiv تقدم طريقة لتصفية أمان المخرجات بشكل فعال في نماذج اللغة الكبيرة (LLMs) باستخدام مجسات الحالة المخفية. تتيح هذه الطريقة الاعتدال في الوقت الفعلي للمخرجات من خلال توليد درجات أمان لكل رمز مباشرة من التنشيطات الداخلية للنموذج، مما يقلل بشكل كبير من تكاليف الاستدلال ويحسن أوقات الاستجابة.

Un estudio reciente publicado en arXiv presenta un método para el filtrado de seguridad de salidas eficiente en grandes modelos de lenguaje (LLMs) mediante el uso de sondas de estado oculto. Este enfoque permite la moderación en tiempo real de las salidas al generar puntuaciones de seguridad por token directamente a partir de las activaciones internas del modelo, reduciendo significativamente los costos de inferencia y mejorando los tiempos de respuesta.

Une étude récente publiée sur arXiv présente une méthode de filtrage de sécurité des sorties efficace pour les grands modèles de langage (LLMs) en utilisant des sondes d'état caché. Cette approche permet une modération en temps réel des sorties en générant des scores de sécurité par jeton directement à partir des activations internes du modèle, réduisant ainsi considérablement les coûts d'inférence et améliorant les temps de réponse.

A recent study published on arXiv introduces a method for efficient output safety filtering in large language models (LLMs) by utilizing hidden-state probes. This approach allows for real-time moderation of outputs by generating per-token safety scores directly from the model's internal activations, significantly reducing inference costs and improving response times.