A Survey of Stochastic Computing Neural Networks for Machine Learning Applications

• Published in: IEEE transaction on neural networks and learning systems Vol. 32; no. 7; pp. 2809 - 2824
• Main Authors: Liu, Yidong, Liu, Siting, Wang, Yanzhi, Lombardi, Fabrizio, Han, Jie
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Belief networks; Biological neural networks; Computation; Energy consumption; Hardware; Humans; Inference; Learning algorithms; Machine Learning; Multilayer perceptrons; Neural network (NN); Neural networks; Neural Networks, Computer; Neurons; Polls & surveys; Power consumption; stochastic computing (SC); Stochastic Processes; Stochasticity; Training
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2020.3009047

Neural networks (NNs) are effective machine learning models that require significant hardware and energy consumption in their computing process. To implement NNs, stochastic computing (SC) has been proposed to achieve a tradeoff between hardware efficiency and computing performance. In an SC NN, hardware requirements and power consumption are significantly reduced by moderately sacrificing the inference accuracy and computation speed. With recent developments in SC techniques, however, the performance of SC NNs has substantially been improved, making it comparable with conventional binary designs yet by utilizing less hardware. In this article, we begin with the design of a basic SC neuron and then survey different types of SC NNs, including multilayer perceptrons, deep belief networks, convolutional NNs, and recurrent NNs. Recent progress in SC designs that further improve the hardware efficiency and performance of NNs is subsequently discussed. The generality and versatility of SC NNs are illustrated for both the training and inference processes. Finally, the advantages and challenges of SC NNs are discussed with respect to binary counterparts.