Compact Mixed-Signal Convolutional Neural Network Using a Single Modular Neuron

• Published in: IEEE transactions on circuits and systems. I, Regular papers Vol. 67; no. 12; pp. 5189 - 5199
• Main Authors: Chang, Dong-Jin, Nam, Byeong-Gyu, Ryu, Seung-Tak
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Analog circuits; Artificial neural networks; Attenuation; Biological neural networks; Circuit design; compact neural network; computational cost; Computational efficiency; Computing costs; Convolution; convolutional neural network; Deep neural network; Design; Hardware; mixed-signal neuron; modular neuron; Modular units; network retraining; Neural networks; Neurons; Parameters; representational cost; Software
• ISSN: 1549-8328; 1558-0806
• DOI: 10.1109/TCSI.2020.3007447

This paper demonstrates a compact mixed-signal (MS) convolutional neural network (CNN) design procedure by proposing a MS modular neuron unit that alleviates analog circuit related design issues such as noise. Through the first step of the proposed procedure, we design a CNN in software with a minimized number of channels for each layer while satisfying the network performance to the target, which creates low representational and computational cost. Then, the network is reconstructed and retrained with a single modular neuron that is recursively utilized for the entire network for the maximum hardware efficiency with a fixed number of parameters that consider signal attenuation. For the last step of the proposed procedure, the parameters of the networks are quantized to an implementable level of MS neurons. We designed the networks for MNIST and Cifar-10 and achieved compact CNNs with a single MS neuron with 97% accuracy for MNIST and 85% accuracy for Cifar-10 whose representational cost and computational cost are reduced to least two times smaller than prior works. The estimated energy per classification of the hardware network for Cifar-10 with a single MS neuron, designed with optimum noise and matching requirements, is <inline-formula> <tex-math notation="LaTeX">0.5\mu </tex-math></inline-formula> J, which is five times smaller than its digital counterpart.