A Comparative Study of Large-Scale Variants of CMA-ES

• Published in: Lecture notes in computer science Vol. 11101; pp. 3 - 15
• Main Authors: Varelas, Konstantinos, Auger, Anne, Brockhoff, Dimo, Hansen, Nikolaus, ElHara, Ouassim Ait, Semet, Yann, Kassab, Rami, Barbaresco, Frédéric
• Format: Book Chapter Conference Proceeding
• Language: English
• Published: Cham Springer International Publishing; Springer Science+Business Media
• Series: Lecture Notes in Computer Science
• Subjects: Computer Science; Mathematics; Neural and Evolutionary Computing; Optimization and Control
• DOI: 10.1007/978-3-319-99253-2_1
• ISBN: 9783319992525; 331999252X
• ISSN: 0302-9743; 1611-3349

The CMA-ES is one of the most powerful stochastic numerical optimizers to address difficult black-box problems. Its intrinsic time and space complexity is quadratic—limiting its applicability with increasing problem dimensionality. To circumvent this limitation, different large-scale variants of CMA-ES with subquadratic complexity have been proposed over the past ten years. To-date however, these variants have been tested and compared only in rather restrictive settings, due to the lack of a comprehensive large-scale testbed to assess their performance. In this context, we introduce a new large-scale testbed with dimension up to 640, implemented within the COCO benchmarking platform. We use this testbed to assess the performance of several promising variants of CMA-ES and the standard limited-memory L-BFGS. In all tested dimensions, the best CMA-ES variant solves more problems than L-BFGS for larger budgets while L-BFGS outperforms the best CMA-ES variant for smaller budgets. However, over all functions, the cumulative runtime distributions between L-BFGS and the best CMA-ES variants are close (less than a factor of 4 in high dimension). Our results illustrate different scaling behaviors of the methods, expose a few defects of the algorithms and reveal that for dimension larger than 80, LM-CMA solves more problems than VkD-CMA while in the cumulative runtime distribution over all functions the VkD-CMA dominates LM-CMA for budgets up to 104\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10^4$$\end{document} times dimension and for all budgets up to dimension 80.