arXiv:2511.11927v1 Announce Type: new 
Abstract: We study the high-dimensional inference of a rank-one signal corrupted by sparse noise. The noise is modelled as the adjacency matrix of a weighted undirected graph with finite average connectivity in the large size limit. Using the replica method from statistical physics, we analytically compute the typical value of the top eigenvalue, the top eigenvector component density, and the overlap between the signal vector and the top eigenvector. The solution is given in terms of recursive distributional equations for auxiliary probability density functions which can be efficiently solved using a population dynamics algorithm. Specialising the noise matrix to Poissonian and Random Regular degree distributions, the critical signal strength is analytically identified at which a transition happens for the recovery of the signal via the top eigenvector, thus generalising the celebrated BBP transition to the sparse noise case. In the large-connectivity limit, known results for dense noise are recovered. Analytical results are in agreement with numerical diagonalisation of large matrices.

تدرس هذه الدراسة استنتاج الإشارات ذات الرتبة الواحدة في الأبعاد العالية المتأثرة بالضوضاء المتناثرة، والتي تم نمذجتها كمصفوفة ارتباط لرسم بياني غير موجه ذو أوزان. باستخدام طريقة النسخ من الفيزياء الإحصائية، تحسب الدراسة القيم النموذجية لأعلى قيمة ذاتية وكثافة مكونات أعلى متجه ذاتي. يتم تحديد القوة الحرجة للإشارة لاستعادتها عبر أعلى متجه ذاتي، مما يعمم انتقال BBP إلى السيناريوهات ذات الضوضاء المتناثرة.

Este estudio investiga la inferencia de alta dimensión de una señal de rango uno afectada por ruido disperso, modelado como la matriz de adyacencia de un grafo no dirigido ponderado. Utilizando el método de réplicas de la física estadística, la investigación calcula los valores típicos del valor propio principal y la densidad de componentes del vector propio. Se identifica la fuerza crítica de la señal para la recuperación a través del vector propio principal, generalizando la transición BBP a escenarios con ruido disperso.

Cette étude examine l'inférence de signaux de rang un en haute dimension, affectés par un bruit épars, modélisé comme la matrice d'adjacence d'un graphe non dirigé pondéré. En utilisant la méthode des répliques de la physique statistique, la recherche calcule les valeurs typiques de la plus grande valeur propre et de la densité des composantes du vecteur propre. La force critique du signal pour la récupération via le vecteur propre principal est identifiée, généralisant la transition BBP aux scénarios de bruit épars.

This study investigates the high-dimensional inference of a rank-one signal affected by sparse noise, modeled as the adjacency matrix of a weighted undirected graph. Utilizing the replica method from statistical physics, the research computes the typical values of the top eigenvalue and eigenvector component density. The critical signal strength for recovery via the top eigenvector is identified, generalizing the BBP transition to scenarios with sparse noise.

PCA recovery thresholds in low-rank matrix inference with sparse noise

arXiv:2601.08331v1 Announce Type: new 
Abstract: Understanding and controlling the behavior of large language models (LLMs) is an increasingly important topic in multilingual NLP. Beyond prompting or fine-tuning, , i.e.,~manipulating internal representations during inference, has emerged as a more efficient and interpretable technique for adapting models to a target language. Yet, no dedicated benchmarks or evaluation protocols exist to quantify the effectiveness of steering techniques. We introduce CLaS-Bench, a lightweight parallel-question benchmark for evaluating language-forcing behavior in LLMs across 32 languages, enabling systematic evaluation of multilingual steering methods. We evaluate a broad array of steering techniques, including residual-stream DiffMean interventions, probe-derived directions, language-specific neurons, PCA/LDA vectors, Sparse Autoencoders, and prompting baselines. Steering performance is measured along two axes: language control and semantic relevance, combined into a single harmonic-mean steering score. We find that across languages simple residual-based DiffMean method consistently outperforms all other methods. Moreover, a layer-wise analysis reveals that language-specific structure emerges predominantly in later layers and steering directions cluster based on language family. CLaS-Bench is the first standardized benchmark for multilingual steering, enabling both rigorous scientific analysis of language representations and practical evaluation of steering as a low-cost adaptation alternative.

يمثل تقديم CLaS-Bench تقدمًا كبيرًا في تقييم نماذج اللغة الكبيرة (LLMs)، حيث يوفر معيارًا للأسئلة الموازية لتقييم تقنيات التوجيه متعددة اللغات عبر 32 لغة. يهدف هذا المعيار إلى قياس فعالية مجموعة متنوعة من أساليب التوجيه، بما في ذلك تدخلات DiffMean في التدفق المتبقي والعصبونات الخاصة باللغة.

La introducción de CLaS-Bench marca un avance significativo en la evaluación de los grandes modelos de lenguaje (LLMs), proporcionando un benchmark de preguntas paralelas para evaluar técnicas de dirección multilingües en 32 idiomas. Este benchmark tiene como objetivo cuantificar la efectividad de varios métodos de dirección, incluyendo intervenciones DiffMean de flujo residual y neuronas específicas de idioma.

L'introduction de CLaS-Bench représente une avancée significative dans l'évaluation des grands modèles de langage (LLMs), fournissant un benchmark de questions parallèles pour évaluer les techniques de pilotage multilingues dans 32 langues. Ce benchmark vise à quantifier l'efficacité de diverses méthodes de pilotage, y compris les interventions DiffMean en flux résiduel et les neurones spécifiques à la langue.

The introduction of CLaS-Bench marks a significant advancement in the evaluation of large language models (LLMs), providing a parallel-question benchmark for assessing multilingual steering techniques across 32 languages. This benchmark aims to quantify the effectiveness of various steering methods, including residual-stream DiffMean interventions and language-specific neurons.

PCA recovery thresholds in low-rank matrix inference with sparse noise

Was this article worth reading? Share it

LucidQuery AI

Spicy Ranks AI

Graphite Note

ChartPixel

SnapChip

GPTHumanizer

Ready to build your own newsroom?