Gradient-type penalty method with inertial effects for solving constrained convex optimization problems with smooth data

• Published in: Optimization letters Vol. 12; no. 1; pp. 17 - 33
• Main Authors: Boţ, Radu Ioan, Csetnek, Ernö Robert, Nimana, Nimit
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.01.2018
• Subjects: Computational Intelligence; Mathematics; Mathematics and Statistics; Numerical and Computational Physics; Operations Research/Decision Theory; Optimization; Original Paper; Simulation; Penalization; Fenchel conjugate; Gradient method; Inertial algorithm
• ISSN: 1862-4472; 1862-4480
• DOI: 10.1007/s11590-017-1158-1

We consider the problem of minimizing a smooth convex objective function subject to the set of minima of another differentiable convex function. In order to solve this problem, we propose an algorithm which combines the gradient method with a penalization technique. Moreover, we insert in our algorithm an inertial term, which is able to take advantage of the history of the iterates. We show weak convergence of the generated sequence of iterates to an optimal solution of the optimization problem, provided a condition expressed via the Fenchel conjugate of the constraint function is fulfilled. We also prove convergence for the objective function values to the optimal objective value. The convergence analysis carried out in this paper relies on the celebrated Opial Lemma and generalized Fejér monotonicity techniques. We illustrate the functionality of the method via a numerical experiment addressing image classification via support vector machines.