SuperHCA: An Efficient Deep-Learning Edge Super-Resolution Accelerator With Sparsity-Aware Heterogeneous Core Architecture

• Published in: IEEE transactions on circuits and systems. I, Regular papers pp. 1 - 12
• Main Authors: Hu, Zhicheng, Zeng, Jiahao, Zhao, Xin, Zhou, Liang, Chang, Liang
• Format: Journal Article
• Language: English
• Published: IEEE 17.07.2024
• Subjects: Computer architecture; deep learning accelerator; Hardware; heterogeneous; Image edge detection; Image super-resolution; layer fusion; Performance evaluation; Quantization (signal); sparsity; Superresolution; System-on-chip
• ISSN: 1549-8328; 1558-0806
• DOI: 10.1109/TCSI.2024.3425753

Deep learning-based super-resolution (SR) generative models have recently emerged as a promising approach for generating high-quality images. While large SR networks can achieve a high peak signal-to-noise ratio (PSNR) to assess image quality, they often come with a high number of parameters, leading to increased computational and memory requirements that can be challenging to deploy on embedded hardware. In this study, we introduce the Anchor-Based Shuffle Net (ABSN), which is designed to create a hardware accelerator using a dynamic-scale fixed-point (DSFP) quantization method. Additionally, we incorporate dynamic quantization adaptation in the hardware design. Our Super-resolution Heterogeneous Accelerator, SuperHCA, utilizes a sparsity-aware heterogeneous architecture to optimize inference efficiency by distinguishing between dense and sparse workloads. We also propose Slice Layer Fusion (SLF) dataflow and feature-sharing bit interleaving (FSBI) methods in the heterogeneous cores to reduce on-chip buffer sizes. The SuperHCA achieves a frame rate of 91 fps at a target resolution of FHD, with the highest throughput area ratio (TAR) of 22.75 fps/mm<inline-formula> <tex-math notation="LaTeX">^2</tex-math> </inline-formula> compared to existing state-of-the-art works.