arXiv:2511.16309v1 Announce Type: cross 
Abstract: Sparse autoencoders (SAEs) are used to analyze embeddings, but their role and practical value are debated. We propose a new perspective on SAEs by demonstrating that they can be naturally understood as topic models. We extend Latent Dirichlet Allocation to embedding spaces and derive the SAE objective as a maximum a posteriori estimator under this model. This view implies SAE features are thematic components rather than steerable directions. Based on this, we introduce SAE-TM, a topic modeling framework that: (1) trains an SAE to learn reusable topic atoms, (2) interprets them as word distributions on downstream data, and (3) merges them into any number of topics without retraining. SAE-TM yields more coherent topics than strong baselines on text and image datasets while maintaining diversity. Finally, we analyze thematic structure in image datasets and trace topic changes over time in Japanese woodblock prints. Our work positions SAEs as effective tools for large-scale thematic analysis across modalities. Code and data will be released upon publication.

تم اقتراح مشفرات التلقائي النادرة (SAEs) كوجهة نظر جديدة حول نمذجة الموضوعات. من خلال توسيع تخصيص ديريشليت الكامن إلى فضاءات التضمين، يتم اشتقاق هدف SAE كـ تقدير أقصى لاحق. يسمح إدخال SAE-TM بتدريب SAEs لتعلم ذرات الموضوع القابلة لإعادة الاستخدام، وتفسيرها كتواريخ كلمات، ودمجها في مواضيع دون إعادة تدريب. تنتج هذه الطريقة مواضيع متماسكة مع الحفاظ على التنوع، مع تطبيقات في تحليل الهياكل الموضوعية في مجموعات بيانات الصور وتتبع تغييرات الموضوعات بمرور الوقت.

Los autoencoders dispersos (SAE) se proponen como una nueva perspectiva sobre la modelización de temas. Al extender la Asignación de Dirichlet Latente a espacios de embeddings, se deriva el objetivo SAE como un estimador máximo a posteriori. La introducción de SAE-TM permite entrenar SAE para aprender átomos de tema reutilizables, interpretándolos como distribuciones de palabras y fusionándolos en temas sin necesidad de reentrenamiento. Este método produce temas coherentes mientras mantiene la diversidad, con aplicaciones en el análisis de estructuras temáticas en conjuntos de datos de imágene…

Les autoencodeurs épars (SAE) sont proposés comme une nouvelle perspective sur la modélisation des sujets. En étendant l'allocation de Dirichlet latente aux espaces d'embeddings, l'objectif SAE est dérivé comme un estimateur maximum a posteriori. L'introduction de SAE-TM permet de former des SAE pour apprendre des atomes de sujet réutilisables, les interprétant comme des distributions de mots, et de les fusionner en sujets sans réentraînement. Cette méthode produit des sujets cohérents tout en maintenant la diversité, avec des applications dans l'analyse des structures thématiques dans les ens…

Sparse autoencoders (SAEs) are proposed as a new perspective on topic modeling. By extending Latent Dirichlet Allocation to embedding spaces, the SAE objective is derived as a maximum a posteriori estimator. The introduction of SAE-TM allows for training SAEs to learn reusable topic atoms, interpreting them as word distributions, and merging them into topics without retraining. This method yields coherent topics while maintaining diversity, with applications in analyzing thematic structures in image datasets and tracking topic changes over time.

Sparse Autoencoders are Topic Models

arXiv:2512.00768v1 Announce Type: new 
Abstract: The fast growth of digital health systems has led to a need to better comprehend how they interpret and represent patient-reported symptoms. Chatbots have been used in healthcare to provide clinical support and enhance the user experience, making it possible to provide meaningful clinical patterns from text-based data through chatbots. The proposed research utilises several different natural language processing methods to study the occurrences of symptom descriptions in medicine as well as analyse the patterns that emerge through these conversations within medical bots. Through the use of the Medical Conversations to Disease Dataset which contains 960 multi-turn dialogues divided into 24 Clinical Conditions, a standardised representation of conversations between patient and bot is created for further analysis by computational means. The multi-method approach uses a variety of tools, including Latent Dirichlet Allocation (LDA) to identify latent symptom themes, K-Means to group symptom descriptions by similarity, Transformer-based Named Entity Recognition (NER) to extract medical concepts, and the Apriori algorithm to discover frequent symptom pairs. Findings from the analysis indicate a coherent structure of clinically relevant topics, moderate levels of clustering cohesiveness and several high confidence rates on the relationships between symptoms like fever headache and rash itchiness. The results support the notion that conversational medical data can be a valuable diagnostic signal for early symptom interpretation, assist in strengthening decision support and improve how users interact with tele-health technology. By demonstrating a method for converting unstructured free-flowing dialogue into actionable knowledge regarding symptoms this work provides an extensible framework to further enhance future performance, dependability and clinical utility of selecting medical chatbots.

أظهرت دراسة حديثة تحليل أنماط الأعراض في محادثات الدردشة الطبية باستخدام طرق مختلفة لمعالجة اللغة الطبيعية. استخدمت الدراسة مجموعة بيانات المحادثات الطبية للأمراض، التي تتكون من 960 حوارًا عبر 24 حالة سريرية، لإنشاء تمثيل موحد للتفاعلات بين المرضى والدردشة لأغراض التحليل الحسابي.

Un estudio reciente ha analizado los patrones de síntomas en las conversaciones con chatbots médicos utilizando varios métodos de procesamiento de lenguaje natural. La investigación utilizó el Medical Conversations to Disease Dataset, que comprende 960 diálogos en 24 condiciones clínicas, para crear una representación estandarizada de las interacciones entre pacientes y chatbots para un análisis computacional posterior.

Une étude récente a analysé les schémas de symptômes dans les conversations avec des chatbots médicaux en utilisant diverses méthodes de traitement du langage naturel. La recherche a utilisé le Medical Conversations to Disease Dataset, qui comprend 960 dialogues répartis sur 24 conditions cliniques, pour créer une représentation standardisée des interactions entre patients et chatbots pour une analyse computationnelle ultérieure.

A recent study has analyzed symptom patterns in medical chatbot conversations using various natural language processing methods. The research utilized the Medical Conversations to Disease Dataset, which comprises 960 dialogues across 24 clinical conditions, to create a standardized representation of interactions between patients and chatbots for further computational analysis.

Text Mining Analysis of Symptom Patterns in Medical Chatbot Conversations

arXiv:2512.02004v1 Announce Type: new 
Abstract: Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidden activations into more fine-grained, interpretable features, they often struggle to reliably align these features with human-defined concepts, resulting in entangled and distributed feature representations. To address this, we introduce AlignSAE, a method that aligns SAE features with a defined ontology through a "pre-train, then post-train" curriculum. After an initial unsupervised training phase, we apply supervised post-training to bind specific concepts to dedicated latent slots while preserving the remaining capacity for general reconstruction. This separation creates an interpretable interface where specific relations can be inspected and controlled without interference from unrelated features. Empirical results demonstrate that AlignSAE enables precise causal interventions, such as reliable "concept swaps", by targeting single, semantically aligned slots.

إن تقديم AlignSAE، وهي طريقة مصممة لمواءمة ميزات Autoencoders النادرة مع أنطولوجيا محددة، يمثل تقدمًا كبيرًا في قابلية تفسير نماذج اللغة الكبيرة (LLMs). تستخدم هذه الطريقة عملية تدريب من مرحلتين، تجمع بين التدريب غير الخاضع للإشراف والتدريب الخاضع للإشراف لتحسين مواءمة الميزات مع المفاهيم المحددة من قبل البشر.

La introducción de AlignSAE, un método diseñado para alinear las características de los Autoencoders Escasos con una ontología definida, marca un avance significativo en la interpretabilidad de los Modelos de Lenguaje Grande (LLMs). Este enfoque emplea un proceso de entrenamiento en dos fases, combinando un preentrenamiento no supervisado con un postentrenamiento supervisado para mejorar la alineación de las características con conceptos definidos por humanos.

L'introduction d'AlignSAE, une méthode conçue pour aligner les caractéristiques des Sparse Autoencoders avec une ontologie définie, marque une avancée significative dans l'interprétabilité des grands modèles de langage (LLMs). Cette approche utilise un processus d'entraînement en deux phases, combinant un pré-entraînement non supervisé avec un post-entraînement supervisé pour améliorer l'alignement des caractéristiques avec des concepts définis par l'homme.

The introduction of AlignSAE, a method designed to align Sparse Autoencoder features with a defined ontology, marks a significant advancement in the interpretability of Large Language Models (LLMs). This approach employs a two-phase training process, combining unsupervised pre-training with supervised post-training to enhance the alignment of features with human-defined concepts.

Sparse Autoencoders are Topic Models

Was this article worth reading? Share it

LucidQuery AI

TypeThinkAI

Airparser