arXiv:2505.11085v2 Announce Type: replace-cross 
Abstract: Kernel-based conditional independence (KCI) testing is a powerful nonparametric method commonly employed in causal discovery tasks. Despite its flexibility and statistical reliability, cubic computational complexity limits its application to large datasets. To address this computational bottleneck, we propose \textit{FastKCI}, a scalable and parallelizable kernel-based conditional independence test that utilizes a mixture-of-experts approach inspired by embarrassingly parallel inference techniques for Gaussian processes. By partitioning the dataset based on a Gaussian mixture model over the conditioning variables, FastKCI conducts local KCI tests in parallel, aggregating the results using an importance-weighted sampling scheme. Experiments on synthetic datasets and benchmarks on real-world production data validate that FastKCI maintains the statistical power of the original KCI test while achieving substantial computational speedups. FastKCI thus represents a practical and efficient solution for conditional independence testing in causal inference on large-scale data.

تم تقديم طريقة جديدة تُدعى FastKCI لتحسين اختبارات الاستقلال الشرطي المعتمدة على النوى، وهي أداة حيوية في اكتشاف الأسباب. تتناول هذه الطريقة القيود الحسابية للاختبارات التقليدية من خلال استخدام استراتيجية مزيج من الخبراء، مما يسمح بمعالجة البيانات بشكل متوازي باستخدام نماذج المزيج الغاوسي.

Se ha introducido un nuevo método llamado FastKCI para mejorar las pruebas de independencia condicional basadas en núcleos, una herramienta crítica en el descubrimiento causal. Este enfoque aborda las limitaciones computacionales de las pruebas KCI tradicionales mediante el uso de una estrategia de mezcla de expertos, permitiendo el procesamiento paralelo de datos utilizando modelos de mezcla gaussiana.

Une nouvelle méthode appelée FastKCI a été introduite pour améliorer les tests d'indépendance conditionnelle basés sur les noyaux, un outil essentiel dans la découverte causale. Cette approche répond aux limitations computationnelles des tests KCI traditionnels en utilisant une stratégie de mélange d'experts, permettant un traitement parallèle des données à l'aide de modèles de mélange gaussien.

A new method called FastKCI has been introduced to enhance kernel-based conditional independence testing, a critical tool in causal discovery. This approach addresses the computational limitations of traditional KCI tests by employing a mixture-of-experts strategy, allowing for parallel processing of data using Gaussian mixture models.

A Fast Kernel-based Conditional Independence test with Application to Causal Discovery

One More Thing in AI – Your Shortcut to AI Mastery

A Fast Kernel-based Conditional Independence test with Application to Causal Discovery

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

AINF Membership

KushoAI

Cubic

FETCH HIVE

Ready to build your own newsroom?