pMEMCPY: a simple, lightweight, and portable I/O library for storing data in persistent memory

Persistent memory (PMEM) devices can achieve comparable performance to DRAM while providing significantly more capacity. This has made the technology compelling as an expansion to main memory. Rethinking PMEM as storage devices can offer a high performance buffering layer for HPC applications to tem...

Full description

Saved in:
Bibliographic Details
Published in2021 IEEE International Conference on Cluster Computing (CLUSTER) pp. 664 - 670
Main Authors Logan, Luke, Lofstead, Jay, Levy, Scott, Widener, Patrick, Sun, Xian-He, Kougkas, Anthony
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.09.2021
Subjects
Buffer storage
Cluster computing
Conferences
Data Layout
Layout
Libraries
Memory Management
Memory Mapped I/O
Metadata
Parallel I/O
Performance evaluation
Persistent Memory
Random access memory
Storage Management
Online AccessGet full text

Cover

More Information
Summary:Persistent memory (PMEM) devices can achieve comparable performance to DRAM while providing significantly more capacity. This has made the technology compelling as an expansion to main memory. Rethinking PMEM as storage devices can offer a high performance buffering layer for HPC applications to temporarily, but safely store data. However, modern parallel I/O libraries, such as HDF5 and pNetCDF, are complicated and introduce significant software and metadata overheads when persisting data to these storage devices, wasting much of their potential. In this work, we explore the potential of PMEM as storage through pMEMCPY: a simple, lightweight, and portable I/O library for storing data in persistent memory. We demonstrate that our approach is up to 2x faster than other popular parallel I/O libraries under real workloads.
ISSN:2168-9253
DOI:10.1109/Cluster48925.2021.00098