Evaluating Garbage Collection Performance Across Managed Language Runtimes

• Published in: Proceedings / International Conference on Software Engineering pp. 1806 - 1818
• Main Authors: Wang, Yicheng, Dou, Wensheng, Liang, Yu, Wang, Yi, Wang, Wei, Wei, Jun, Huang, Tao
• Format: Conference Proceeding
• Language: English
• Published: IEEE 26.04.2025
• Subjects: C# languages; Computer bugs; Garbage collection; Gears; Instruments; Java; Logic; managed language runtime; Performance evaluation; Runtime; Transforms; Virtual machining
• DOI: 10.1109/ICSE55347.2025.00218

Modern managed language runtimes (e.g., Java, Go and C#) rely on garbage collection (GC) mechanisms to automatically allocate and reclaim in-memory objects. The efficiency of GC implementations can greatly impact the overall performance of runtime-based applications. To improve GC performance, the academic and industrial communities have proposed several approaches to evaluate the GC implementations in an individual runtime. However, these approaches target a specific managed language (e.g., Java), and cannot be used to compare the GC implementations in different runtimes. In this paper, we propose GEAR, an automated approach to construct consistent GC workloads for different managed language runtimes, which can further be used to evaluate GC implementations across different runtimes. Specifically, we design a group of runtime-agnostic Memory Operation Primitives (MOP), which can portray the memory usage information that influences GC. GEAR can further automatically convert a MOP program into runtime-specific programs for the target runtimes, which serve as a consistent GC workload for different runtimes. To build MOP programs with real-world GC workloads, we instrument the commonly-used runtime Java Virtual Machine (JVM) to collect the memory operation trace during a Java application's execution, and then transform the memory operation trace into a MOP program. The experimental result on three widely-used runtimes (i.e., Java, Go and C#) shows that GEAR can generate consistent GC workloads for different runtimes. We further conduct a comprehensive study on these three runtimes, and reveal some interesting findings about their GC performance, providing useful guidance for improving their GC implementations.