The Pangeo Ecosystem: Interactive Computing Tools for the Geosciences: Benchmarking on HPC

The Pangeo ecosystem is an interactive computing software stack for HPC and public cloud infrastructures. In this paper, we show benchmarking results of the Pangeo platform on two different HPC systems. Four different geoscience operations were considered in this benchmarking study with varying chun...

Full description

Saved in:
Bibliographic Details
Published inTools and Techniques for High Performance Computing Vol. 1190; pp. 190 - 204
Main Authors Odaka, Tina Erica, Banihirwe, Anderson, Eynard-Bontemps, Guillaume, Ponte, Aurelien, Maze, Guillaume, Paul, Kevin, Baker, Jared, Abernathey, Ryan
Format Book Chapter
LanguageEnglish
Published Switzerland Springer International Publishing AG 2020
Springer International Publishing
SeriesCommunications in Computer and Information Science
Subjects
Benchmarking
Cloud
Dask
HPC
Interactive computing
Pangeo
Xarray
Online AccessGet full text

Cover

More Information
Summary:The Pangeo ecosystem is an interactive computing software stack for HPC and public cloud infrastructures. In this paper, we show benchmarking results of the Pangeo platform on two different HPC systems. Four different geoscience operations were considered in this benchmarking study with varying chunk sizes and chunking schemes. Both strong and weak scaling analyses were performed. Chunk sizes between 64 MB to 512 MB were considered, with the best scalability obtained for 512 MB. Compared to certain manual chunking schemes, the auto chunking scheme scaled well.
ISBN:3030447278
9783030447274
ISSN:1865-0929
1865-0937
DOI:10.1007/978-3-030-44728-1_12