Recent Advances in Stochastic Gradient Descent in Deep Learning

• Published in: Mathematics (Basel) Vol. 11; no. 3; p. 682
• Main Authors: Tian, Yingjie, Zhang, Yuqi, Zhang, Haibin
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.01.2023
• Subjects: Algorithms; Analysis; Artificial intelligence; Audio data; Computers; Data processing; Datasets; Deep learning; Distance learning; Food science; Language; Machine learning; Methods; Natural language processing; Neural networks; Optimization algorithms; Speech; stochastic gradient descent; Stochastic processes; China

In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective. This study provides a detailed analysis of contemporar...