A Derivative-Free Optimization Algorithm Combining Line-Search and Trust-Region Techniques

The speeding-up and slowing-down (SUSD) direction is a novel direction, which is proved to converge to the gradient descent direction under some conditions. The authors propose the derivative-free optimization algorithm SUSD-TR, which combines the SUSD direction based on the covariance matrix of int...

Full description

Saved in:
Bibliographic Details
Published inChinese annals of mathematics. Serie B Vol. 44; no. 5; pp. 719 - 734
Main Authors Xie, Pengcheng, Yuan, Ya-xiang
Format Journal Article
LanguageEnglish
Published Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2023
Springer Nature B.V
Subjects
Algorithms
Applications of Mathematics
Convergence
Covariance matrix
Interpolation
Iterative methods
Mathematical analysis
Mathematics
Mathematics and Statistics
Optimization
90C30
65K05
Quadratic model
Nonlinear optimization
Trust-Region
90C56
90C90
Line-Search
Derivative-Free
Online AccessGet full text

Cover

More Information
Summary:The speeding-up and slowing-down (SUSD) direction is a novel direction, which is proved to converge to the gradient descent direction under some conditions. The authors propose the derivative-free optimization algorithm SUSD-TR, which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the interpolation model function at the current iteration step. They analyze the optimization dynamics and convergence of the algorithm SUSD-TR. Details of the trial step and structure step are given. Numerical results show their algorithm’s efficiency, and the comparison indicates that SUSD-TR greatly improves the method’s performance based on the method that only goes along the SUSD direction. Their algorithm is competitive with state-of-the-art mathematical derivative-free optimization algorithms.
ISSN:0252-9599
1860-6261
DOI:10.1007/s11401-023-0040-y