Accelerated Log-Regularized Convolutional Transform Learning and Its Convergence Guarantee

• Published in: IEEE transactions on cybernetics Vol. 52; no. 10; pp. 10785 - 10799
• Main Authors: Li, Zhenni, Zhao, Haoli, Guo, Yongcheng, Yang, Zuyuan, Xie, Shengli
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">log -regularizer; Acceleration; Algorithms; Approximation algorithms; Artificial neural networks; Convergence; Convergence guarantee; Convolution; convolutional transform learning (CTL); Extrapolation; Optimization; Representations; Sparsity; the difference of convex; Transforms; Unsupervised learning
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2021.3067352

Convolutional transform learning (CTL), learning filters by minimizing the data fidelity loss function in an unsupervised way, is becoming very pervasive, resulting from keeping the best of both worlds: the benefit of unsupervised learning and the success of the convolutional neural network. There have been growing interests in developing efficient CTL algorithms. However, developing a convergent and accelerated CTL algorithm with accurate representations simultaneously with proper sparsity is an open problem. This article presents a new CTL framework with a <inline-formula> <tex-math notation="LaTeX">log </tex-math></inline-formula> regularizer that can not only obtain accurate representations but also yield strong sparsity. To efficiently address our nonconvex composite optimization, we propose to employ the proximal difference of the convex algorithm (PDCA) which relies on decomposing the nonconvex regularizer into the difference of two convex parts and then optimizes the convex subproblems. Furthermore, we introduce the extrapolation technology to accelerate the algorithm, leading to a fast and efficient CTL algorithm. In particular, we provide a rigorous convergence analysis for the proposed algorithm under the accelerated PDCA. The experimental results demonstrate that the proposed algorithm can converge more stably to desirable solutions with lower approximation error and simultaneously with stronger sparsity and, thus, learn filters efficiently. Meanwhile, the convergence speed is faster than the existing CTL algorithms.