LLAMA: The low‐level abstraction for memory access

• Published in: Software, practice & experience Vol. 53; no. 1; pp. 115 - 141
• Main Authors: Gruber, Bernhard Manfred, Amadio, Guilherme, Blomer, Jakob, Matthes, Alexander, Widera, René, Bussmann, Michael
• Format: Journal Article
• Language: English
• Published: Bognor Regis Wiley Subscription Services, Inc 01.01.2023
• Subjects: Arrays; C plus plus; C++ (programming language); Computer memory; Copying; Data structures; Hardware; Layouts; Libraries; memory layout; performance portability; programming techniques; software implementation; Structured data
• ISSN: 0038-0644; 1097-024X
• DOI: 10.1002/spe.3077

The performance gap between CPU and memory widens continuously. Choosing the best memory layout for each hardware architecture is increasingly important as more and more programs become memory bound. For portable codes that run across heterogeneous hardware architectures, the choice of the memory layout for data structures is ideally decoupled from the rest of a program. This can be accomplished via a zero‐runtime‐overhead ion layer, underneath which memory layouts can be freely exchanged. We present the low‐level ion of memory access (LLAMA), a C++ library that provides such a data structure ion layer with example implementations for multidimensional arrays of nested, structured data. LLAMA provides fully C++ compliant methods for defining and switching custom memory layouts for user‐defined data types. The library is extensible with third‐party allocators. Providing two close‐to‐life examples, we show that the LLAMA‐generated array of structs and struct of arrays layouts produce identical code with the same performance characteristics as manually written data structures. Integrations into the SPEC CPU® lbm benchmark and the particle‐in‐cell simulation PIConGPU demonstrate LLAMA's abilities in real‐world applications. LLAMA's layout‐aware copy routines can significantly speed up transfer and reshuffling of data between layouts compared with naive element‐wise copying. LLAMA provides a novel tool for the development of high‐performance C++ applications in a heterogeneous environment.