arXiv:2602.05649v2 Announce Type: replace 
Abstract: The long-standing dominance of gradient-boosted decision trees for tabular data has recently been challenged by in-context learning tabular foundation models. In-context learning methods fit and predict in one forward pass without parameter updates by leveraging the training data as context for predicting on query test points. While recent tabular foundation models achieve state-of-the-art performance, their transformer architecture based on the attention mechanism has quadratic complexity regarding dataset size, which in turn increases the overhead on training and inference time, and limits the capacity of the models to handle large-scale datasets. In this work, we propose TACO, an end-to-end tabular compression model that compresses the training dataset in a latent space. We test our method on the TabArena benchmark, where our proposed method is up to 94x faster in inference time, while consuming up to 97\% less memory compared to the state-of-the-art tabular transformer architecture, all while retaining performance without significant degradation. Lastly, our method not only scales better with increased dataset sizes, but it also achieves better performance compared to other baselines.

تمثل الإضافة الأخيرة لـ TACO، وهو نموذج ضغط شامل لنماذج الأساس الجدولية، محاولة لمعالجة عدم الكفاءة في الهياكل المعمارية الحالية التي تواجه صعوبات مع مجموعات البيانات الكبيرة بسبب تعقيدها التربيعي. يقوم TACO بضغط مجموعات بيانات التدريب في مساحة كامنة، مما يحسن بشكل كبير من سرعة الاستدلال ويقلل من استخدام الذاكرة.

La reciente introducción de TACO, un modelo de compresión de extremo a extremo para modelos de fundación tabular, busca abordar las ineficiencias de las arquitecturas de transformadores existentes que luchan con grandes conjuntos de datos debido a su complejidad cuadrática. TACO comprime los conjuntos de datos de entrenamiento en un espacio latente, mejorando significativamente la velocidad de inferencia y reduciendo el uso de memoria.

La récente introduction de TACO, un modèle de compression de bout en bout pour les modèles de fondation tabulaire, vise à résoudre les inefficacités des architectures de transformateurs existantes qui peinent avec de grands ensembles de données en raison de leur complexité quadratique. TACO compresse les ensembles de données d'entraînement dans un espace latent, améliorant ainsi considérablement la vitesse d'inférence et réduisant l'utilisation de la mémoire.

The recent introduction of TACO, an end-to-end compression model for tabular foundation models, aims to address the inefficiencies of existing transformer architectures that struggle with large datasets due to their quadratic complexity. TACO compresses training datasets in a latent space, significantly improving inference speed and reducing memory usage.