Real-Time SSDLite Object Detection on FPGA

• Published in: IEEE transactions on very large scale integration (VLSI) systems Vol. 29; no. 6; pp. 1192 - 1205
• Main Authors: Kim, Suchang, Na, Seungho, Kong, Byeong Yong, Choi, Jaewoong, Park, In-Cheol
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Computer architecture; Deep neural network (DNN); Embedded systems; Feature extraction; Field programmable gate arrays; field-programmable gate array (FPGA); Hardware; Kernel; Object detection; Object recognition; Optimization; Optimization techniques; Real time; real-time applications; Real-time systems; Throughput; very-large-scale integration (VLSI) architecture
• ISSN: 1063-8210; 1557-9999
• DOI: 10.1109/TVLSI.2021.3064639

Deep neural network (DNN)-based object detection has been investigated and applied to various real-time applications. However, it is hard to employ the DNNs in embedded systems due to their high computational complexity and deep-layered structure. Although several field-programmable gate array (FPGA) implementations have been presented recently for real-time object detection, they suffer from either low throughput or low detection accuracy. In this article, we propose an efficient computing system for real-time SSDLite object detection on FPGA devices, which includes novel hardware architecture and system optimization techniques. In the proposed hardware architecture, a neural processing unit (NPU) that consists of heterogeneous units, such as band processing, scaling, and accumulating, and data fetching and formatting units is designed to accelerate the DNNs efficiently. In addition, system optimization techniques are presented to improve the throughput further. A task control unit is employed to balance the workload and increase the utilization of heterogeneous units in the NPU, and the object detection algorithm is refined accordingly. The proposed architecture is realized on an Intel Arria 10 FPGA and enhances the throughput by up to <inline-formula> <tex-math notation="LaTeX">13.6\times </tex-math></inline-formula> compared to the state-of-the-art FPGA implementation.