RAMBO: Resource-Aware Model-Based Optimization with Scheduling for Heterogeneous Runtimes and a Comparison with Asynchronous Model-Based Optimization

• Published in: Learning and Intelligent Optimization Vol. 10556; pp. 180 - 195
• Main Authors: Kotthaus, Helena, Richter, Jakob, Lang, Andreas, Thomas, Janek, Bischl, Bernd, Marwedel, Peter, Rahnenführer, Jörg, Lang, Michel
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Black-box optimization; Global optimization; Model-based optimization; Parallelization; Performance management; Resource-aware scheduling
• ISBN: 9783319694030; 3319694030
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-69404-7_13

Sequential model-based optimization is a popular technique for global optimization of expensive black-box functions. It uses a regression model to approximate the objective function and iteratively proposes new interesting points. Deviating from the original formulation, it is often indispensable to apply parallelization to speed up the computation. This is usually achieved by evaluating as many points per iteration as there are workers available. However, if runtimes of the objective function are heterogeneous, resources might be wasted by idle workers. Our new knapsack-based scheduling approach aims at increasing the effectiveness of parallel optimization by efficient resource utilization. Derived from an extra regression model we use runtime predictions of point evaluations to efficiently map evaluations to workers and reduce idling. We compare our approach to five established parallelization strategies on a set of continuous functions with heterogeneous runtimes. Our benchmark covers comparisons of synchronous and asynchronous model-based approaches and investigates the scalability.