Solving higher dimensional expensive black box global optimization problems using sparse directional search on surrogates

• Published in: Mathematical programming computation Vol. 16; no. 4; pp. 665 - 693
• Main Authors: Liu, Limeng, Yang, Muming, Shoemaker, Christine A., Xie, Tingting
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.12.2024; Springer Nature B.V
• Subjects: Algorithms; Black boxes; Full Length Paper; Gaussian process; Global optimization; Mathematics; Mathematics and Statistics; Mathematics of Computing; Operations Research/Decision Theory; Optimization; Radial basis function; Theory of Computation; Trajectory optimization; Trajectory planning; 90-08 Computational methods for problems pertaining to operations research and mathematical programming; 90C26 Nonconvex programming, global optimization
• ISSN: 1867-2949; 1867-2957
• DOI: 10.1007/s12532-024-00267-7

We present a new algorithm Directional Optimization Search with Surrogate (DOSS), for optimizing problems with box constraints and a computationally expensive black-box objective function. DOSS is a radial basis function (RBF) based method that mainly focuses on higher dimensional and computationally expensive objective functions that can be multimodal. DOSS introduces three new techniques not previously used in earlier RBF algorithms, including using a combination of the coordinates knowledge level, fewer initial sampling points, and the surrogate’s gradient information. Numerical results on a test set including 14 test problems with 36, 48, and 60 dimensions show that DOSS outperforms two recently published algorithms RBFOpt and TuRBO, and earlier RBF algorithms such as DYCORS. TuRBO is a Gaussian process based optimization algorithm, which outperformed earlier state-of-the-art methods. DOSS algorithm also has a good performance on real-world optimization problems, including robot pushing and rover trajectory planning problems. Almost sure convergence for the DOSS algorithm is also proven in this paper. An implementation of DOSS is available at https://doi.org/10.5281/zenodo.13731558 .