Mixed-Precision Kernel Recursive Least Squares

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 3; pp. 1284 - 1298
• Main Authors: Lee, JunKyu, Nikolopoulos, Dimitrios S., Vandierendonck, Hans
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Budget machine learning; Chaos theory; Footprints; Kernel; kernel method; Kernels; Learning algorithms; Least squares; Machine learning; Mathematics; Memory management; mixed-precision training; online learning; Prediction algorithms; Predictions; Sea surface temperature; Sunspots; Support vector machines; Throughput; Time series; Time series analysis; Training
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2020.3041677

Kernel recursive least squares (KRLS) is a widely used online machine learning algorithm for time series predictions. In this article, we present the mixed-precision KRLS, producing equivalent prediction accuracy to double-precision KRLS with a higher training throughput and a lower memory footprint. The mixed-precision KRLS applies single-precision arithmetic to the computation components being not only numerically resilient but also computationally intensive. Our mixed-precision KRLS demonstrates the 1.32, 1.15, 1.29, 1.09, and <inline-formula> <tex-math notation="LaTeX">1.08\times </tex-math></inline-formula> training throughput improvements using 24.95%, 24.74%, 24.89%, 24.48%, and 24.20% less memory footprint without losing any prediction accuracy compared to double-precision KRLS for a 3-D nonlinear regression, a Lorenz chaotic time series, a Mackey-Glass chaotic time series, a sunspot number time series, and a sea surface temperature time series, respectively.