Accelerating Lossy and Lossless Compression on Emerging BlueField DPU Architectures

• Published in: 2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS) pp. 373 - 385
• Main Authors: Li, Yuke, Kashyap, Arjun, Chen, Weicong, Guo, Yanfei, Lu, Xiaoyi
• Format: Conference Proceeding
• Language: English
• Published: IEEE 27.05.2024
• Subjects: BlueField DPU; Computer architecture; Data compression; Data processing; Data Processing Unit (DPU); Deep learning; Distributed processing; High performance computing; Lossless Compression; Lossy Compression; System performance
• ISSN: 1530-2075
• DOI: 10.1109/IPDPS57955.2024.00040

Data compression has become a crucial technique in addressing performance bottlenecks caused by increasing data volumes in High-Performance Computing (HPC), Big Data, and Deep Learning (DL). Despite its potential to boost system performance, recent studies have identified significant challenges with existing compression methods, mainly due to their high computational demands amidst continuously growing data sizes. Concurrently, the advent of Data Processing Units (DPUs), equipped with programmable System-on-Chip (SoC) and specialized compression accelerators, offers a promising opportunity to alter the landscape of data compression. This paper explores the complexities and potential of leveraging NVIDIA BlueField DPUs to accelerate lossy and lossless compression. Towards this, we introduce PEDAL, an innovative library that leverages the hardware capabilities of DPUs to unify and optimize data compression designs. Moreover, we seamlessly co-design PEDAL with the popular MPICH MPI library, demonstrating up to 101x speedup in compression time and 88x decrease in communication latency. Drawing on these achievements, we share our experience with various research communities about accelerating data compression on DPUs in communication-oriented HPC scenarios.