GPES: a preemptive execution system for GPGPU computing

• Published in: 21st IEEE Real-Time and Embedded Technology and Applications Symposium pp. 87 - 97
• Main Authors: Husheng Zhou, Guangmo Tong, Cong Liu
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.04.2015
• Subjects: Computer architecture; Context; Graphics processing units; Hardware; Kernel; Processor scheduling; Real-time systems
• ISSN: 1545-3421; 2642-7346
• DOI: 10.1109/RTAS.2015.7108420

Graphics processing units (GPUs) are being widely used as co-processors in many application domains to accelerate general-purpose workloads that are computationally intensive, known as GPGPU computing. Real-time multi-tasking support is a critical requirement for many emerging GPGPU computing domains. However, due to the asynchronous and non-preemptive nature of GPU processing, in multi-tasking environments, tasks with higher priority may be blocked by lower priority tasks for a lengthy duration. This severely harms the system's timing predictability and is a serious impediment limiting the applicability of GPGPU in many real-time and embedded systems. In this paper, we present an efficient GPGPU preemptive execution system (GPES), which combines user-level and driverlevel runtime engines to reduce the pending time of high-priority GPGPU tasks that may be blocked by long-freezing low-priority competing workloads. GPES automatically slices a long-running kernel execution into multiple subkernel launches and splits data transaction into multiple chunks at user-level, then inserts preemption points between subkernel launches and memorycopy operations at driver-level. We implement a prototype of GPES, and use real-world benchmarks and case studies for evaluation. Experimental results demonstrate that GPES is able to reduce the pending time of high-priority tasks in a multitasking environment by up to 90% over the existing GPU driver solutions, while introducing small overheads.