Optimizing Depthwise Separable Convolution Operations on GPUs

• Published in: IEEE transactions on parallel and distributed systems Vol. 33; no. 1; pp. 70 - 87
• Main Authors: Lu, Gangzhao, Zhang, Weizhe, Wang, Zheng
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Computer architecture; Convolution; depthwise; Floating point arithmetic; GPU utilization; Graphics processing units; Inference; Instruction sets; Kernel; memory optimization; Network latency; Performance optimization; pointwise; Registers; Training; Two dimensional models

The depthwise separable convolution is commonly seen in convolutional neural networks (CNNs), and is widely used to reduce the computation overhead of a standard multi-channel 2D convolution. Existing implementations of depthwise separable convolutions target accelerating model training with large b...