Enhancing Gaussian Process Surrogates for Optimization and Posterior Approximation via Random Exploration
This paper proposes novel noise-free Bayesian optimization strategies that rely on a random exploration step to enhance the accuracy of Gaussian process surrogate models. The new algorithms retain the ease of implementation of the classical GP-UCB algorithm, but the additional random exploration ste...
Saved in:
Main Authors | , |
---|---|
Format | Journal Article |
Language | English |
Published |
30.01.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | This paper proposes novel noise-free Bayesian optimization strategies that
rely on a random exploration step to enhance the accuracy of Gaussian process
surrogate models. The new algorithms retain the ease of implementation of the
classical GP-UCB algorithm, but the additional random exploration step
accelerates their convergence, nearly achieving the optimal convergence rate.
Furthermore, to facilitate Bayesian inference with an intractable likelihood,
we propose to utilize optimization iterates for maximum a posteriori estimation
to build a Gaussian process surrogate model for the unnormalized log-posterior
density. We provide bounds for the Hellinger distance between the true and the
approximate posterior distributions in terms of the number of design points. We
demonstrate the effectiveness of our Bayesian optimization algorithms in
non-convex benchmark objective functions, in a machine learning hyperparameter
tuning problem, and in a black-box engineering design problem. The
effectiveness of our posterior approximation approach is demonstrated in two
Bayesian inference problems for parameters of dynamical systems. |
---|---|
DOI: | 10.48550/arxiv.2401.17037 |