arXiv:2406.03442v3 Announce Type: replace 
Abstract: Do norms of rationality apply to machine learning models, in particular language models? In this paper we investigate this question by focusing on a special subset of rational norms: coherence norms. We consider both logical coherence norms as well as coherence norms tied to the strength of belief. To make sense of the latter, we introduce the Minimal Assent Connection (MAC) and propose a new account of credence, which captures the strength of belief in language models. This proposal uniformly assigns strength of belief simply on the basis of model internal next token probabilities. We argue that rational norms tied to coherence do apply to some language models, but not to others. This issue is significant since rationality is closely tied to predicting and explaining behavior, and thus it is connected to considerations about AI safety and alignment, as well as understanding model behavior more generally.

تستكشف الورقة المعنونة 'هل نماذج اللغة عقلانية؟ حالة معايير التماسك وإعادة مراجعة المعتقدات' تطبيق معايير العقلانية، وخاصة معايير التماسك، على نماذج اللغة. تميز بين معايير التماسك المنطقي وتلك المرتبطة بقوة المعتقد. يقدم المؤلفون اتصال الموافقة الأدنى (MAC)، وهو إطار جديد لفهم المصداقية في نماذج اللغة استنادًا إلى احتمالات الرموز الداخلية. تشير النتائج إلى أنه بينما تلتزم بعض نماذج اللغة بهذه المعايير العقلانية، فإن البعض الآخر لا يفعل، مما يثير تساؤلات مهمة حول سلوك الذكاء الاصطناعي وسلامته.

El artículo titulado '¿Son racionales los modelos de lenguaje? El caso de las normas de coherencia y la revisión de creencias' investiga la aplicación de normas de racionalidad, específicamente las normas de coherencia, a los modelos de lenguaje. Se distingue entre normas de coherencia lógica y aquellas relacionadas con la fuerza de la creencia. Los autores introducen la Conexión de Consentimiento Mínimo (MAC), un nuevo marco para entender la credibilidad en los modelos de lenguaje basado en las probabilidades internas de los tokens. Los hallazgos sugieren que, mientras algunos modelos de lenguaje se adhieren a estas normas racionales, otros no lo hacen, lo que plantea preguntas importantes sobre el comportamiento y la seguridad de la IA.

L'article intitulé 'Les modèles linguistiques sont-ils rationnels ? Le cas des normes de cohérence et de révision des croyances' examine l'application des normes de rationalité, en particulier les normes de cohérence, aux modèles linguistiques. Il distingue entre les normes de cohérence logique et celles liées à la force de la croyance. Les auteurs introduisent la connexion de consentement minimal (MAC), un nouveau cadre pour comprendre la crédence dans les modèles linguistiques basé sur les probabilités internes des tokens. Les résultats suggèrent que certains modèles linguistiques respectent ces normes rationnelles, tandis que d'autres ne le font pas, soulevant des questions importantes sur le comportement et la sécurité de l'IA.

The paper titled 'Are language models rational? The case of coherence norms and belief revision' explores the application of rationality norms, specifically coherence norms, to language models. It distinguishes between logical coherence norms and those related to the strength of belief. The authors introduce the Minimal Assent Connection (MAC), a new framework for understanding credence in language models based on internal token probabilities. The findings suggest that while some language models adhere to these rational norms, others do not, raising important questions about AI behavior and safety.

Are language models rational? The case of coherence norms and belief revision

Here's everything you need to know about the latest smart ring by Oura, based on a wearable expert's real-world usage.

Is the $500 Oura Ring 4 Ceramic worth it? I wore one for a month, and here's my advice

Samsung's QN90F QLED delivers great streaming and gaming performance, making it a strong holiday value.

Why I recommend this Samsung QLED TV over pricier OLED models in 2025 - and don't regret it

<h1>
 
 
 AI Amnesia: Erasing Knowledge Without a Trace
</h1>

Imagine your AI model accidentally learned something it shouldn't have – sensitive customer data, for example. Current methods for deleting this information often require retraining the entire model, an expensive and time-consuming process. What if we could surgically remove that knowledge without starting from scratch?

The key lies in a novel approach: creating artificial "forgetting cues." We're teaching the model to unlearn specific data patterns by exposing it to carefully crafted synthetic examples. These examples are designed to strongly contradict the information we want the model to forget, effectively overwriting the problematic associations in its memory. This works even if you don’t have access to the original data you need to erase.

Think of it like this: you're trying to forget a bad song stuck in your head. Instead of trying to actively suppress it (which rarely works), you blast an even more catchy song. The new song overwrites the old one, effectively erasing it from your mental playlist.

Benefits of Selective Forgetting:

<ul>
<li> Enhanced Data Privacy: Remove sensitive data without compromising the overall model's performance.</li>
<li> Reduced Retraining Costs: Avoid full model retraining, saving significant time and resources.</li>
<li> Improved Model Security: Eliminate vulnerabilities introduced by unintentionally learned patterns.</li>
<li> Adaptable Learning: Enables continuous refinement of AI models based on evolving data landscapes.</li>
<li> Compliance Ready: Supports compliance with data privacy regulations like GDPR.</li>
<li> Scalable Solutions: Works efficiently even with limited access to training data.</li>
</ul>

Practical Tip: One challenge is ensuring the synthetic data accurately targets the information you want to remove without negatively impacting the model's ability to generalize. Rigorous testing and validation with holdout datasets are crucial.

The promise of AI that can truly 'forget' opens exciting possibilities for responsible AI development. By enabling precise data deletion, we pave the way for more secure, compliant, and adaptable machine learning systems. Imagine AI models that can adapt to changing ethical guidelines or quickly unlearn incorrect information, all without massive retraining efforts. This is a crucial step towards trustworthy and responsible AI that respects data privacy and aligns with societal values. Future exploration could include extending this to different data modalities and model architectures.

Related Keywords: machine unlearning, data privacy, few-shot learning, zero-shot learning, synthetic data, model editing, catastrophic forgetting, incremental learning, continual learning, deep learning, neural networks, data security, algorithmic fairness, responsible ai, ethical ai, federated unlearning, privacy-preserving ai, model retraining, AI governance, data deletion, GDPR compliance

يتناول المقال نهجًا جديدًا لمعالجة مشكلة نسيان الذكاء الاصطناعي، حيث قد تحتفظ نماذج الذكاء الاصطناعي عن غير قصد بمعلومات حساسة. تتطلب الطرق التقليدية لحذف هذه البيانات غالبًا إعادة تدريب كاملة للنموذج، وهو ما يعد مكلفًا ويستغرق وقتًا. تتضمن التقنية الجديدة إنشاء 'إشارات نسيان' اصطناعية تساعد النموذج على نسيان أنماط بيانات معينة من خلال تعريضه لأمثلة صناعية تتناقض مع المعلومات غير المرغوب فيها، مما يسمح بإزالة المعرفة المستهدفة دون الحاجة للوصول إلى البيانات الأصلية.

El artículo aborda un nuevo enfoque para tratar el problema de la amnesia de la IA, donde los modelos de IA pueden retener inadvertidamente información sensible. Los métodos tradicionales para eliminar estos datos a menudo requieren un reentrenamiento completo del modelo, lo que es costoso y lleva tiempo. La nueva técnica implica crear 'señales de olvido' artificiales que ayudan al modelo a desaprender patrones de datos específicos al exponerlo a ejemplos sintéticos que contradicen la información no deseada, permitiendo así la eliminación selectiva de conocimientos sin necesidad de acceder a l…

L'article traite d'une nouvelle approche pour résoudre le problème de l'amnésie de l'IA, où les modèles d'IA peuvent involontairement conserver des informations sensibles. Les méthodes traditionnelles pour supprimer ces données nécessitent souvent un réentraînement complet du modèle, ce qui est coûteux et long. La nouvelle technique consiste à créer des 'cues d'oubli' artificiels qui aident le modèle à désapprendre des schémas de données spécifiques en lui présentant des exemples synthétiques qui contredisent les informations indésirables, permettant ainsi une suppression ciblée des connaissan…

The article discusses a novel approach to address the issue of AI amnesia, where AI models may inadvertently retain sensitive information. Traditional methods for deleting such data often require complete retraining of the model, which is costly and time-consuming. The new technique involves creating artificial 'forgetting cues' that help the model unlearn specific data patterns by presenting it with synthetic examples that contradict the unwanted information, allowing for targeted knowledge removal without needing access to the original data.

AI Amnesia: Erasing Knowledge Without a Trace

Baseus' Enercore CG11 travel adapter is one of the better designed ones I've tested, although its best feature isn't immediately apparent.

Traveling soon? Why this one charger is the only one you'll ever need to pack

Turning materials like wood chips, crop residues and municipal solid waste into fuels and chemicals is important for our country's energy independence.

طور الباحثون نماذج حاسوبية متقدمة تهدف إلى تحسين التنبؤات لعمليات طحن الكتلة الحيوية. تركز هذه الابتكارات على تحويل مواد مثل رقائق الخشب، بقايا المحاصيل، والنفايات الصلبة البلدية إلى وقود ومواد كيميائية قيمة، وهو أمر مهم لتعزيز استقلالية الطاقة في البلاد. من المتوقع أن تعمل النماذج على تحسين كفاءة معالجة الكتلة الحيوية، مما يساهم في حلول الطاقة المستدامة.

Investigadores han desarrollado modelos informáticos avanzados destinados a mejorar las predicciones para los procesos de molienda de biomasa. Esta innovación se centra en convertir materiales como astillas de madera, residuos de cultivos y residuos sólidos municipales en combustibles y productos químicos valiosos, lo que es crucial para mejorar la independencia energética del país. Se espera que los modelos optimicen la eficiencia del procesamiento de biomasa, contribuyendo así a soluciones energéticas sostenibles.

Des chercheurs ont développé des modèles informatiques avancés visant à améliorer les prévisions pour les processus de broyage de la biomasse. Cette innovation se concentre sur la conversion de matériaux tels que les copeaux de bois, les résidus de culture et les déchets solides municipaux en combustibles et produits chimiques précieux, ce qui est crucial pour renforcer l'indépendance énergétique du pays. Les modèles devraient optimiser l'efficacité du traitement de la biomasse, contribuant ainsi à des solutions énergétiques durables.

Researchers have developed advanced computer models aimed at improving predictions for biomass milling processes. This innovation focuses on converting materials such as wood chips, crop residues, and municipal solid waste into valuable fuels and chemicals, which is crucial for enhancing energy independence in the country. The models are expected to optimize the efficiency of biomass processing, thereby contributing to sustainable energy solutions.

Researchers develop computer models for better biomass milling predictions

The most interesting aspect of the latest Even Realities glasses may be their limitations.

I've tried several AI smart glasses (including Meta Ray-Bans) in 2025 - these are the most comfortable

arXiv:2511.11966v1 Announce Type: cross 
Abstract: We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increasing (and text quality decreasing) as generations grow longer. This error accumulation is a fundamental problem in autoregressive models, and the standard solution is to truncate the distribution, which improves text quality at the cost of diversity. In this paper, we ask: is miscalibration likely to improve with scale, and is it theoretically possible to calibrate without tradeoffs? To build intuition, we first study a simplified theoretical setting to characterize the scaling behavior of miscalibration with respect to dataset size. We find that the scaling behavior depends on the power law exponent of the data distribution -- in particular, for a power law exponent close to 1, the scaling exponent is close to 0, meaning that miscalibration improves very slowly with scale. Next, we measure miscalibration empirically in language models ranging from 0.5B to 70B parameters. We find that the observed scaling behavior is similar to what is predicted by the simplified setting: our fitted scaling exponents for text are close to 0, meaning that larger models accumulate error at a similar rate as smaller ones. This scaling (or, lack thereof) provides one explanation for why we sample from larger models with similar amounts of truncation as smaller models, even though the larger models are of higher quality. However, truncation is not a satisfying solution because it comes at the cost of increased log loss. In theory, is it even possible to reduce entropy while preserving log loss? We prove that it is possible, if we assume access to a black box which can fit models to predict the future entropy of text.

تدرس الورقة مشكلة معايرة الإنتروبيا في نماذج اللغة، مع التركيز على ما إذا كانت إنتروبيا النموذج تتماشى مع خسارة اللوغاريتم على النصوص البشرية. وجدت الدراسات السابقة أن إنتروبيا كل خطوة تزداد (وتنخفض جودة النص) مع زيادة طول الأجيال، مما يبرز مشكلة أساسية في النماذج التلقائية. تسأل الورقة: هل من المحتمل أن تتحسن المعايرة الخاطئة مع زيادة الحجم، وهل من الممكن نظريًا المعايرة دون تنازلات؟ لبناء الفهم، تدرس الورقة أولاً إعدادًا نظريًا مبسطًا لتوصيف سلوك المعايرة الخاطئة بالنسبة لحجم مجموعة البيانات.

El artículo examina la calibración de la entropía en los modelos de lenguaje, centrándose en si la entropía de un modelo se alinea con la pérdida logarítmica en texto humano. Estudios anteriores indicaron que a medida que la longitud de la generación de texto aumenta, la entropía también aumenta mientras que la calidad del texto disminuye, destacando un problema fundamental en los modelos autorregresivos. Los autores investigan si la mala calibración puede mejorar con la escala y si es teóricamente posible calibrar sin compromisos, analizando el comportamiento de escalado en relación con el ta…

Cet article examine la calibration de l'entropie dans les modèles de langage, en se concentrant sur la question de savoir si leur entropie est alignée avec la perte logarithmique sur le texte humain. Des études antérieures ont indiqué qu'à mesure que la longueur de génération de texte augmente, l'entropie augmente tandis que la qualité du texte diminue, soulignant un problème fondamental dans les modèles autorégressifs. Les auteurs se demandent si la mauvaise calibration peut s'améliorer avec l'échelle et si une calibration sans compromis est théoriquement possible, en analysant le comportemen…

The paper examines entropy calibration in language models, focusing on whether their entropy aligns with log loss on human text. Previous studies indicated that as text generation lengthens, entropy increases while text quality declines, highlighting a fundamental issue in autoregressive models. The authors investigate whether miscalibration can improve with scale and if calibration without tradeoffs is theoretically feasible, analyzing the scaling behavior concerning dataset size and power law exponents.

On the Entropy Calibration of Language Models

arXiv:2511.11389v1 Announce Type: new 
Abstract: According to Futrell and Mahowald [arXiv:2501.17047], both infants and language models (LMs) find attested languages easier to learn than impossible languages that have unnatural structures. We review the literature and show that LMs often learn attested and many impossible languages equally well. Difficult to learn impossible languages are simply more complex (or random). LMs are missing human inductive biases that support language acquisition.

تتناول دراسة نُشرت على arXiv قدرات التعلم لدى الرضع ونماذج اللغة (LM) فيما يتعلق باللغات المعتمدة مقابل اللغات المستحيلة. تشير الأبحاث إلى أن كلا المجموعتين تجدان أن اللغات المعتمدة أسهل في التعلم من تلك التي تحتوي على هياكل غير طبيعية. ومع ذلك، تكشف النتائج أن نماذج اللغة يمكن أن تتعلم العديد من اللغات المستحيلة بنفس فعالية اللغات المعتمدة. تقترح الدراسة أن تعقيد هذه اللغات، بدلاً من استحالتها، يساهم في التحديات التي تواجهها نماذج اللغة، التي تفتقر إلى التحيزات الاستقرائية البشرية الأساسية اللازمة لاكتساب اللغة.

Un estudio publicado en arXiv examina las capacidades de aprendizaje de los bebés y los modelos de lenguaje (LM) en relación con los idiomas attestados frente a los imposibles. La investigación indica que ambos grupos encuentran más fácil aprender idiomas attestados que aquellos con estructuras no naturales. Sin embargo, los hallazgos revelan que los LM pueden aprender muchos idiomas imposibles tan eficazmente como los attestados. El estudio sugiere que la complejidad de estos idiomas, en lugar de su imposibilidad, contribuye a los desafíos que enfrentan los LM, que carecen de los sesgos inductivos humanos esenciales para la adquisición del lenguaje.

Une étude publiée sur arXiv examine les capacités d'apprentissage des nourrissons et des modèles linguistiques (LM) concernant les langues attestées par rapport aux langues impossibles. La recherche indique que les deux groupes trouvent les langues attestées plus faciles à apprendre que celles avec des structures non naturelles. Cependant, les résultats révèlent que les LM peuvent apprendre de nombreuses langues impossibles aussi efficacement que les langues attestées. L'étude suggère que la complexité de ces langues, plutôt que leur impossibilité, contribue aux défis rencontrés par les LM, qui manquent des biais inductifs humains essentiels à l'acquisition du langage.

A study published on arXiv examines the learning capabilities of infants and language models (LMs) regarding attested versus impossible languages. The research indicates that both groups find attested languages easier to learn than those with unnatural structures. However, the findings reveal that LMs can learn many impossible languages as effectively as attested ones. The study suggests that the complexity of these languages, rather than their impossibility, contributes to the challenges faced by LMs, which lack the human inductive biases essential for language acquisition.

Studies with impossible languages falsify LMs as models of human language

arXiv:2510.24021v2 Announce Type: replace 
Abstract: Knowledge distillation (KD) is a standard route to compress Large Language Models (LLMs) into compact students, yet most pipelines uniformly apply token-wise loss regardless of teacher confidence. This indiscriminate supervision amplifies noisy, high-entropy signals and is especially harmful under large teacher-student capacity gaps. We introduce SelecTKD, a plug-and-play Selective Token-Weighted distillation framework that shifts the focus from "how to measure divergence" to "where to apply learning". At each step, the student proposes tokens that are verified by the teacher through a robust propose-and-verify procedure with two variants: greedy Top-k and non-greedy Spec-k. Accepted tokens receive full loss, while rejected tokens are masked or down-weighted. This objective-agnostic design works with on- and off-policy data, induces an implicit curriculum quantified by Token Acceptance Rate (TAR), and stabilizes optimization. Across instruction following, mathematical reasoning, code generation, and a VLM setting, SelecTKD consistently improves strong baselines and achieves state-of-the-art results for small models without architectural changes or extra reference models.

SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs

arXiv:2511.13368v1 Announce Type: new 
Abstract: Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages and their combinations remains poorly understood. We conduct a controlled PEFT/LoRA study across multiple open-weight LLM families and sizes, treating task and language as transfer axes while conditioning on model family and size; we fine-tune each model on a single task-language source and measure transfer as the percentage-point change versus its baseline score when evaluated on all other task-language target pairs. We decompose transfer into (i) Matched-Task (Cross-Language), (ii) Matched-Language (Cross-Task), and (iii) Cross-Task (Cross-Language) regimes. We uncover two consistent general patterns. First, a pronounced on-task vs. off-task asymmetry: Matched-Task (Cross-Language) transfer is reliably positive, whereas off-task transfer often incurs collateral degradation. Second, a stable donor-recipient structure across languages and tasks (hub donors vs. brittle recipients). We outline implications for risk-aware fine-tuning and model specialisation.

Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning

arXiv:2511.11878v1 Announce Type: new 
Abstract: While large language models (LLMs) show transformative potential in healthcare, their development remains focused on high-resource languages, creating a critical barrier for others as simple translation fails to capture unique clinical and cultural nuances, such as endemic diseases. To address this, we introduce MedPT, the first large-scale, real-world corpus for Brazilian Portuguese, comprising 384,095 authentic question-answer pairs from patient-doctor interactions. The dataset underwent a meticulous multi-stage curation protocol, using a hybrid quantitative-qualitative analysis to filter noise and contextually enrich thousands of ambiguous queries. We further augmented the corpus via LLM-driven annotation, classifying questions into seven semantic types to capture user intent. Our analysis reveals its thematic breadth (3,200 topics) and unique linguistic properties, like the natural asymmetry in patient-doctor communication. To validate its utility, we benchmark a medical specialty routing task: fine-tuning a 1.7B parameter model achieves an outstanding 94\% F1-score on a 20-class setup. Furthermore, our qualitative error analysis shows misclassifications are not random but reflect genuine clinical ambiguities (e.g., between comorbid conditions), proving the dataset's deep semantic richness. We publicly release MedPT to foster the development of more equitable, accurate, and culturally-aware medical technologies for the Portuguese-speaking world.

Are language models rational? The case of coherence norms and belief revision

Was this article worth reading? Share it