Recent advances in efficient computation of deep convolutional neural networks

Deep neural networks have evolved remarkably over the past few years and they are currently the fundamental tools of many intelligent systems. At the same time, the computational complexity and resource consumption of these networks continue to increase. This poses a significant challenge to the dep...

Full description

Saved in:
Bibliographic Details
Published inFrontiers of information technology & electronic engineering Vol. 19; no. 1; pp. 64 - 77
Main Authors Cheng, Jian, Wang, Pei-song, Li, Gang, Hu, Qing-hao, Lu, Han-qing
Format Journal Article
LanguageEnglish
Published Hangzhou Zhejiang University Press 2018
Springer Nature B.V
Subjects
Accelerators
Algorithms
Application specific integrated circuits
Artificial neural networks
Communications Engineering
Computer Hardware
Computer Science
Computer Systems Organization and Communication Networks
Electrical Engineering
Electronics and Microelectronics
Field programmable gate arrays
Hardware
Instrumentation
Machine learning
Network design
Networks
Neural networks
Review
Hardware accelerator
Compression
Deep neural networks
Acceleration
TP3
Online AccessGet full text

Cover

More Information
Summary:Deep neural networks have evolved remarkably over the past few years and they are currently the fundamental tools of many intelligent systems. At the same time, the computational complexity and resource consumption of these networks continue to increase. This poses a significant challenge to the deployment of such networks, especially in real-time applications or on resource-limited devices. Thus, network acceleration has become a hot topic within the deep learning community. As for hardware implementation of deep neural networks, a batch of accelerators based on a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) have been proposed in recent years. In this paper, we provide a comprehensive survey of recent advances in network acceleration, compression, and accelerator design from both algorithm and hardware points of view. Specifically, we provide a thorough analysis of each of the following topics: network pruning, low-rank approximation, network quantization, teacher–student networks, compact network design, and hardware accelerators. Finally, we introduce and discuss a few possible future directions.
ISSN:2095-9184
2095-9230
DOI:10.1631/FITEE.1700789