Efficient and High-quality Recommendations via Momentum-incorporated Parallel Stochastic Gradient Descent-Based Learning

• Published in: IEEE/CAA journal of automatica sinica Vol. 8; no. 2; pp. 402 - 411
• Main Authors: Luo, Xin, Qin, Wen, Dong, Ani, Sedraoui, Khaled, Zhou, MengChu
• Format: Journal Article
• Language: English
• Published: Piscataway The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 01.02.2021; Hengrui (Chongqing) Artificial Intelligence Research Center, Department of Big Data Analyses Techniques, Cloudwalk, Chongqing 401331, China%School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065; the Center of Research Excellence in Renewable Energy and Power Systems, King Abdulaziz University, Jeddah 21481, Saudi Arabia; Chongqing Engineering Research Center of Big Data Application for Smart Cities, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400714, China%Department of Computer and Information Science, City College of Dongguan University of Technology, Dongguan 523419, China%Department of Electrical and Computer Engineering, Faculty of Engineering, and Center of Research Excellence in Renewable Energy and Power Systems, King Abdulaziz University, Jeddah 21481, Saudi Arabia%Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ 07102 USA; School of Computer Science and Technology, Dongguan University of Technology, Dongguan 523808
• Subjects: Algorithms; Big data; Biological system modeling; Computational modeling; Convergence; Electronic mail; Factor analysis; industrial application; industrial data; latent factor analysis; Machine learning; Momentum; parallel algorithm; Parallel processing; recommender system (RS); Recommender systems; Sparse matrices; stochastic gradient descent (SGD); Stochastic processes; Task analysis; Training; latent factor analysis; recommender system (RS); parallel algorithm; Big data; machine learning; industrial data; stochastic gradient descent (SGD); industrial application
• ISSN: 2329-9266; 2329-9274
• DOI: 10.1109/JAS.2020.1003396

A recommender system (RS) relying on latent factor analysis usually adopts stochastic gradient descent (SGD) as its learning algorithm. However, owing to its serial mechanism, an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems. Aiming at addressing this issue, this study proposes a momentum-incorporated parallel stochastic gradient descent (MPSGD) algorithm, whose main idea is two-fold: a) implementing parallelization via a novel data-splitting strategy, and b) accelerating convergence rate by integrating momentum effects into its training process. With it, an MPSGD-based latent factor (MLF) model is achieved, which is capable of performing efficient and high-quality recommendations. Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm, an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.