arXiv:2602.05786v3 Announce Type: replace 
Abstract: Tree-boosting is a widely used machine learning technique for tabular data. However, its out-of-sample accuracy is critically dependent on multiple hyperparameters. In this article, we empirically compare several popular methods for hyperparameter optimization for tree-boosting including random grid search, the tree-structured Parzen estimator (TPE), Gaussian-process-based Bayesian optimization (GP-BO), Hyperband, the sequential model-based algorithm configuration (SMAC) method, and deterministic full grid search using $59$ regression and binary classification data sets. We find that the SMAC method clearly outperforms all the other considered methods on average, and it gives stable performance across a diverse collection of tabular data sets under a fixed tuning budget, which is relevant for users who cannot afford extensive manual trial-and-error tuning. We further observe that (i) a relatively large number of trials larger than $100$ is typically required for accurate tuning, (ii) using default values for hyperparameters or a full search over a small grid often yields very inaccurate models, (iii) all considered hyperparameters can have a material effect on the accuracy of tree-boosting, i.e., there is no small set of hyperparameters that is more important than others, and (iv) choosing the number of boosting iterations using early stopping yields more accurate results compared to including it in the search space for regression tasks.

دراسة حديثة نُشرت على arXiv تستكشف طرقًا متعددة لتحسين المعلمات في تقنية tree-boosting، وهي تقنية شائعة في التعلم الآلي للبيانات الجدولية. تقارن الدراسة تجريبيًا طرقًا مثل البحث العشوائي، SMAC، وتحسين بايزي القائم على العمليات الغاوسية عبر 59 مجموعة بيانات، مما يكشف أن SMAC يتفوق باستمرار على الطرق الأخرى ضمن ميزانية ضبط ثابتة.

Un estudio reciente publicado en arXiv explora varios métodos para la optimización de hiperparámetros en el tree-boosting, una técnica de aprendizaje automático ampliamente utilizada para datos tabulares. La investigación compara empíricamente métodos como la búsqueda aleatoria, SMAC y la optimización bayesiana basada en procesos gaussianos en 59 conjuntos de datos, revelando que SMAC supera consistentemente a los demás bajo un presupuesto de ajuste fijo.

Une étude récente publiée sur arXiv examine diverses méthodes d'optimisation des hyperparamètres dans le cadre du tree-boosting, une technique d'apprentissage automatique largement utilisée pour les données tabulaires. La recherche compare empiriquement des méthodes telles que la recherche aléatoire, SMAC et l'optimisation bayésienne basée sur des processus gaussiens sur 59 ensembles de données, révélant que SMAC surpasse systématiquement les autres sous un budget de réglage fixe.

A recent study published on arXiv explores various methods for hyperparameter optimization in tree-boosting, a prevalent machine learning technique for tabular data. The research empirically compares methods such as random grid search, SMAC, and Gaussian-process-based Bayesian optimization across 59 datasets, revealing that SMAC consistently outperforms others under a fixed tuning budget.