Interactive Diffusions for Global Optimization

• Published in: Journal of optimization theory and applications Vol. 163; no. 2; pp. 491 - 509
• Main Authors: Sun, Yue, Garcia, Alfredo
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.11.2014; Springer Nature B.V
• Subjects: Annealing; Applications of Mathematics; Calculus of Variations and Optimal Control; Optimization; Computer science; Convergence; Diffusion; Drift; Dynamical systems; Engineering; Heat treating; Interactive; Mathematical models; Mathematics; Mathematics and Statistics; Noise; Operations Research/Decision Theory; Optimization; Optimization algorithms; Parallel processing; Partial differential equations; Studies; Theory of Computation; Time; United States > US; Parallel computing; Diffusions; Simulated annealing
• ISSN: 0022-3239; 1573-2878
• DOI: 10.1007/s10957-013-0394-5

We present a novel approach, in which parallel annealing processes interact in a manner that expedites the identification of a globally optimal solution. A first annealing process operates at a faster time scale and has a drift function that converges to a non-zero (but relatively small) noise level. A second annealing process (operating at a slower time scale) is subject to a modified drift term in which the steepest descent direction is perturbed with the first annealing process density gradient. This additional term ensures that the second process is “repelled” from regions already explored. As a result, the first annealing process (which quickly identifies locally optimal solutions) allows the second annealing process to bypass locally optimal solutions recently identified, so that it can be made to converge to global optima at a faster rate. We show that, when compared to independent annealing processes, the proposed interactive diffusions can increase the speed of convergence at the expense of minimal additional computational overhead.