arXiv:2512.01819v1 Announce Type: new 
Abstract: Decision trees and random forest remain highly competitive for classification on medium-sized, standard datasets due to their robustness, minimal preprocessing requirements, and interpretability. However, a single tree suffers from high estimation variance, while large ensembles reduce this variance at the cost of substantial computational overhead and diminished interpretability. In this paper, we propose Decision Tree Embedding (DTE), a fast and effective method that leverages the leaf partitions of a trained classification tree to construct an interpretable feature representation. By using the sample means within each leaf region as anchor points, DTE maps inputs into an embedding space defined by the tree's partition structure, effectively circumventing the high variance inherent in decision-tree splitting rules. We further introduce an ensemble extension based on additional bootstrap trees, and pair the resulting embedding with linear discriminant analysis for classification. We establish several population-level theoretical properties of DTE, including its preservation of conditional density under mild conditions and a characterization of the resulting classification error. Empirical studies on synthetic and real datasets demonstrate that DTE strikes a strong balance between accuracy and computational efficiency, outperforming or matching random forest and shallow neural networks while requiring only a fraction of their training time in most cases. Overall, the proposed DTE method can be viewed either as a scalable decision tree classifier that improves upon standard split rules, or as a neural network model whose weights are learned from tree-derived anchor points, achieving an intriguing integration of both paradigms.

تم اقتراح طريقة جديدة تُسمى Decision Tree Embedding (DTE)، والتي تستخدم تقسيمات الأوراق لشجرة تصنيف مدربة لإنشاء تمثيل للميزات قابل للتفسير. تهدف هذه الطريقة إلى تقليل التباين العالي في التقديرات المرتبطة عادةً بالأشجار الفردية، مع الحفاظ على القابلية للتفسير والكفاءة في مهام التصنيف.

Se ha propuesto un nuevo método llamado Decision Tree Embedding (DTE), que utiliza las particiones de hojas de un árbol de clasificación entrenado para crear una representación de características interpretable. Este enfoque busca reducir la alta varianza de estimación asociada típicamente con árboles de decisión individuales, manteniendo la interpretabilidad y eficiencia en las tareas de clasificación.

Une nouvelle méthode appelée Decision Tree Embedding (DTE) a été proposée, utilisant les partitions de feuilles d'un arbre de classification entraîné pour créer une représentation des caractéristiques interprétable. Cette approche vise à réduire la forte variance d'estimation généralement associée aux arbres de décision uniques tout en maintenant l'interprétabilité et l'efficacité dans les tâches de classification.

A new method called Decision Tree Embedding (DTE) has been proposed, which utilizes the leaf partitions of a trained classification tree to create an interpretable feature representation. This approach aims to reduce the high estimation variance typically associated with single decision trees while maintaining interpretability and efficiency in classification tasks.

Decision Tree Embedding by Leaf-Means

Was this article worth reading? Share it

The Visualizer

Augmeta

Deptho.ai