High-Performance FPGA-Based CNN Accelerator With Block-Floating-Point Arithmetic

• Published in: IEEE transactions on very large scale integration (VLSI) systems Vol. 27; no. 8; pp. 1874 - 1885
• Main Authors: Lian, Xiaocong, Liu, Zhenyu, Song, Zhourui, Dai, Jiwu, Zhou, Wei, Ji, Xiangyang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accelerators; Artificial neural networks; Block floating point (BFP); Computational modeling; Computer memory; Computer vision; convolutional neural network (CNN) accelerator; Embedded systems; Feature maps; Field programmable gate arrays; field-programmable gate array (FPGA); Floating point arithmetic; Hardware; Mathematical model; Memory management; Neural networks; Quantization (signal); Retraining; Rounding; Speech processing; three-level parallel
• DOI: 10.1109/TVLSI.2019.2913958
• ISSN: 1063-8210

Convolutional neural networks (CNNs) are widely used and have achieved great success in computer vision and speech processing applications. However, deploying the large-scale CNN model in the embedded system is subject to the constraints of computation and memory. An optimized block-floating-point (BFP) arithmetic is adopted in our accelerator for efficient inference of deep neural networks in this paper. The feature maps and model parameters are represented in 16-bit and 8-bit formats, respectively, in the off-chip memory, which can reduce memory and off-chip bandwidth requirements by 50% and 75% compared to the 32-bit FP counterpart. The proposed 8-bit BFP arithmetic with optimized rounding and shifting-operation-based quantization schemes improves the energy and hardware efficiency by three times. One CNN model can be deployed in our accelerator without retraining at the cost of an accuracy loss of not more than 0.12%. The proposed reconfigurable accelerator with three parallelism dimensions, ping-pong off-chip DDR3 memory access, and an optimized on-chip buffer group is implemented on the Xilinx VC709 evaluation board. Our accelerator achieves a performance of 760.83 GOP/s and 82.88 GOP/s/W under a 200-MHz working frequency, significantly outperforming previous accelerators.