A Unifying Framework for Sparsity-Constrained Optimization

• Published in: Journal of optimization theory and applications Vol. 199; no. 2; pp. 663 - 692
• Main Authors: Lapucci, Matteo, Levato, Tommaso, Rinaldi, Francesco, Sciandrone, Marco
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2023; Springer Nature B.V
• Subjects: Algorithms; Applications of Mathematics; Calculus of Variations and Optimal Control; Optimization; Constraints; Engineering; Greedy algorithms; Impact analysis; Mathematics; Mathematics and Statistics; Mixed integer; Neighborhoods; Operations Research/Decision Theory; Optimization; Penalty function; Sparsity; Theory of Computation; Sparsity-constrained problems; Optimality conditions; 90C30; 65K05; 90C46; Numerical methods; Stationarity; Sparse logistic regression; Asymptotic convergence
• ISSN: 0022-3239; 1573-2878
• DOI: 10.1007/s10957-023-02306-0

In this paper, we consider the optimization problem of minimizing a continuously differentiable function subject to both convex constraints and sparsity constraints. By exploiting a mixed-integer reformulation from the literature, we define a necessary optimality condition based on a tailored neighborhood that allows to take into account potential changes of the support set. We then propose an algorithmic framework to tackle the considered class of problems and prove its convergence to points satisfying the newly introduced concept of stationarity. We further show that, by suitably choosing the neighborhood, other well-known optimality conditions from the literature can be recovered at the limit points of the sequence produced by the algorithm. Finally, we analyze the computational impact of the neighborhood size within our framework and in the comparison with some state-of-the-art algorithms, namely, the Penalty Decomposition method and the Greedy Sparse-Simplex method. The algorithms have been tested using a benchmark related to sparse logistic regression problems.