arXiv:2511.13625v3 Announce Type: replace 
Abstract: Bayesian optimization (BO) efficiently finds high-performing parameters by maximizing an acquisition function, which models the promise of parameters. A major computational bottleneck arises in acquisition function optimization, where multi-start optimization (MSO) with quasi-Newton (QN) methods is required due to the non-convexity of the acquisition function. BoTorch, a widely used BO library, currently optimizes the summed acquisition function over multiple points, leading to the speedup of MSO owing to PyTorch batching. Nevertheless, this paper empirically demonstrates the suboptimality of this approach in terms of off-diagonal approximation errors in the inverse Hessian of a QN method, slowing down its convergence. To address this problem, we propose to decouple QN updates using a coroutine while batching the acquisition function calls. Our approach not only yields the theoretically identical convergence to the sequential MSO but also drastically reduces the wall-clock time compared to the previous approaches. Our approach is available in GPSampler in Optuna, effectively reducing its computational overhead.

تقدم دراسة جديدة طريقة لتعزيز التحسين البايزي من خلال فصل تحديثات المحسن شبه نيوتن أثناء تجميع تقييمات دالة الاكتساب. تهدف هذه الطريقة إلى معالجة عدم الكفاءة الحسابية المرتبطة بالتحسين متعدد البدء في دوال الاكتساب غير المحدبة، كما يتضح من خلال التحليل التجريبي.

Un nuevo estudio presenta un método para mejorar la optimización bayesiana al desacoplar las actualizaciones del optimizador cuasi-Newton mientras se agrupan las evaluaciones de la función de adquisición. Este enfoque aborda las ineficiencias computacionales asociadas con la optimización de múltiples inicios en funciones de adquisición no convexas, como se demuestra a través de un análisis empírico.

Une nouvelle étude propose une méthode pour améliorer l'optimisation bayésienne en découplant les mises à jour de l'optimiseur quasi-Newton tout en regroupant les évaluations de la fonction d'acquisition. Cette approche vise à résoudre les inefficacités computationnelles associées à l'optimisation multi-démarrages dans les fonctions d'acquisition non convexes, comme le montre une analyse empirique.

A new study presents a method to enhance Bayesian optimization by decoupling quasi-Newton optimizer updates while batching acquisition function evaluations. This approach addresses the computational inefficiencies associated with multi-start optimization in non-convex acquisition functions, as demonstrated through empirical analysis.

Batch Acquisition Function Evaluations and Decouple Optimizer Updates for Faster Bayesian Optimization

Was this article worth reading? Share it

Airparser

Hypertune

Octofy

Gonogo by Trampoline

Prompt Builder

Pointron

Ready to build your own newsroom?