SRGC-Nets: Sparse Repeated Group Convolutional Neural Networks

• Published in: IEEE transaction on neural networks and learning systems Vol. 31; no. 8; pp. 2889 - 2902
• Main Authors: Lu, Yao, Lu, Guangming, Lin, Rui, Li, Jinxing, Zhang, David
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Computer applications; Convolution; Convolutional neural networks; Convolutional neural networks (CNNs); deep learning; group convolution; Kernel; Kernels; Learning systems; Neural networks; Parameters; Redundancy; Retinal ganglion cells; sparse repeated group convolution (SRGC); SRGC neural networks (SRGC-Nets); Wireless networks
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2019.2933665

Group convolution is widely used in many mobile networks to remove the filter's redundancy from the channel extent. In order to further reduce the redundancy of group convolution, this article proposes a novel repeated group convolutional (RGC) kernel, which has <inline-formula> <tex-math notation="LaTeX">M </tex-math></inline-formula> primary groups, and each primary group includes <inline-formula> <tex-math notation="LaTeX">N </tex-math></inline-formula> tiny groups. In every primary group, the same convolutional kernel is repeated in all the tiny groups. The RGC filter is the first kernel to remove the redundancy from group extent. Based on RGC, a sparse RGC (SRGC) kernel is also introduced in this article, and its corresponding network is called SRGC neural networks (SRGC-Net). The SRGC kernel is the summation of RGC kernel and pointwise group convolutional (PGC) kernel. The number of PGC's groups is <inline-formula> <tex-math notation="LaTeX">M </tex-math></inline-formula>. Accordingly, in each primary group, besides the center locations in all channels, the values of parameters located in other <inline-formula> <tex-math notation="LaTeX">N-1 </tex-math></inline-formula> tiny groups are all zero. Therefore, SRGC can significantly reduce the parameters. Moreover, it can also effectively retrieve spatial and channel-difference features by utilizing RGC and PGC to preserve the richness of produced features. Comparative experiments were performed on the benchmark classification data sets. Compared with the traditional popular networks, SRGC-Nets can perform better with timely reducing the model size and computational complexity. Furthermore, it can also achieve better performances than other latest state-of-the-art mobile networks on most of the databases and effectively decrease the test and training runtime.