Quickest convergence of online algorithms via data selection

• Published in: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 6185 - 6189
• Main Authors: Romero, Daniel, Berberidis, Dimitris, Giannakis, Georgios B.
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.03.2016
• Subjects: Acceleration; adaptive data selection; Algorithms; Approximation algorithms; Big data; Computation; Computational efficiency; Computer simulation; Convergence; Electronics; Linear regression; Mathematical analysis; Online; online learning; Prediction algorithms; Regression; stochastic approximation
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP.2016.7472866

Big data applications demand efficient solvers capable of providing accurate solutions to large-scale problems at affordable computational costs. Processing data sequentially, online algorithms offer attractive means to deal with massive data sets. However, they may incur prohibitive complexity in high-dimensional scenarios if the entire data set is processed. It is therefore necessary to confine computations to an informative subset. While existing approaches have focused on selecting a prescribed fraction of the available data vectors, the present paper capitalizes on this degree of freedom to accelerate the convergence of a generic class of online algorithms in terms of processing time/computational resources by balancing the required burden with a metric of how informative each datum is. The proposed method is illustrated in a linear regression setting, and simulations corroborate the superior convergence rate of the recursive least-squares algorithm when the novel data selection is effected.