MixedNet: Network Design Strategies For A Cost-Effective Quantized CNNs

• Published in: IEEE access Vol. 9; p. 1
• Main Authors: Chang, Dong-Jin, Nam, Byeong-Gyu, Ryu, Seung-Tak
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Chips (memory devices); Classification; Convolution; Convolutional neural network; Convolutional neural networks; deep neural network; Degradation; Hardware; Low cost; memory access number; memory cost; Network architecture; Network design; Neural networks; on-chip memory size; Quantization (signal); quantized neural networks; System-on-chip
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3106658

This paper proposes design strategies for a low-cost quantized neural network. To prevent the classification accuracy from being degraded by quantization, a structure-design strategy that utilizes a large number of channels rather than deep layers is proposed. In addition, a squeeze-and-excitation (SE) layer is adopted to enhance the performance of the quantized network. Through a quantitative analysis and simulations of the quantized key convolution layers of ResNet and MobileNets, a low-cost layer-design strategy for use when building a neural network is proposed. With this strategy, a low-cost network referred to as a MixedNet is constructed. A 4-bit quantized MixedNet example achieves an on-chip memory size reduction of 60% and fewer memory access by 53% with negligible classification accuracy degradation in comparison with conventional networks while also showing classification accuracy rates of approximately 73% for Cifar-100 and 93% for Cifar-10.