arXiv:2606.08236v1 Announce Type: cross 
Abstract: As large language models are increasingly deployed in high-stakes settings, there is a growing need for tools that audit not only model outputs but also the internal computations that produce them. Circuit analysis is a central approach in mechanistic interpretability, but it is typically target-conditioned, explaining a single prompt paired with a chosen completion. This target-conditioned setup can obscure heterogeneity across a model's continuation distribution. We introduce distribution-level unsupervised feature discovery, which clusters sampled continuations using both semantic content and sequence-level mechanistic attributions, without manually specifying target outputs. Our method represents each continuation with a semantic embedding and a prefix-to-continuation attribution signature, then optimizes a rate-distortion objective that trades off semantic coherence, mechanistic consistency, and cluster granularity. Across clustering and steering analyses, the discovered clusters expose continuation modes that single-view baselines miss and provide interventional evidence that cluster signatures correspond to actionable mechanistic factors. Overall, our approach complements circuit analysis and behavioral evaluation by providing a scalable audit of the mechanisms underlying a model's continuation distribution.

تقدم دراسة جديدة طريقة لاكتشاف الميزات غير الخاضعة للإشراف في نماذج اللغة الكبيرة (LLMs) من خلال محاذاة المحتوى الدلالي مع التخصيصات الآلية، مما يعزز فهم مخرجات النموذج والعمليات الداخلية. تتجمع هذه الطريقة بين الاستمراريات المأخوذة عشوائياً دون الحاجة إلى مخرجات مستهدفة، مع تحسين التناسق الدلالي والاتساق الآلي.

Un nuevo estudio presenta un método para el descubrimiento de características no supervisado en grandes modelos de lenguaje (LLMs) al alinear el contenido semántico con las atribuciones mecanísticas, mejorando la comprensión de las salidas del modelo y los cálculos internos. Este enfoque agrupa continuaciones muestreadas sin necesidad de salidas objetivo, optimizando la coherencia semántica y la consistencia mecanística.

Une nouvelle étude présente une méthode de découverte de caractéristiques non supervisée dans les grands modèles de langage (LLMs) en alignant le contenu sémantique avec les attributions mécanistes, améliorant ainsi la compréhension des sorties du modèle et des calculs internes. Cette approche regroupe les continuations échantillonnées sans nécessiter de sorties cibles, optimisant la cohérence sémantique et la consistance mécaniste.

A new study introduces a method for unsupervised feature discovery in large language models (LLMs) by aligning semantic content with mechanistic attributions, enhancing the understanding of model outputs and internal computations. This approach clusters sampled continuations without the need for target outputs, optimizing for semantic coherence and mechanistic consistency.