Near-global climate simulation at 1 km resolution: establishing a performance baseline on 4888 GPUs with COSMO 5.0

• Published in: Geoscientific Model Development Vol. 11; no. 4; pp. 1665 - 1681
• Main Authors: Fuhrer, Oliver, Chadha, Tarun, Hoefler, Torsten, Kwasniewski, Grzegorz, Lapillonne, Xavier, Leutwyler, David, Lüthi, Daniel, Osuna, Carlos, Schär, Christoph, Schulthess, Thomas C, Vogt, Hannes
• Format: Journal Article
• Language: English
• Published: Katlenburg-Lindau Copernicus GmbH 02.05.2018; Copernicus Publications
• Subjects: Accelerators; Analysis; Atmospheric models; Climate; Climate models; Computer simulation; Domain specific languages; DSL (Digital subscriber line); Energy consumption; Global climate; Global climate models; Graphics; Graphics processing units; Learning; Meteorological research; Resolution; Running; Scale (ratio); Simulation
• ISSN: 1991-9603; 1991-959X; 1991-962X; 1991-9603; 1991-962X
• DOI: 10.5194/gmd-11-1665-2018

The best hope for reducing long-standing global climate model biases is by increasing resolution to the kilometer scale. Here we present results from an ultrahigh-resolution non-hydrostatic climate model for a near-global setup running on the full Piz Daint supercomputer on 4888 GPUs (graphics processing units). The dynamical core of the model has been completely rewritten using a domain-specific language (DSL) for performance portability across different hardware architectures. Physical parameterizations and diagnostics have been ported using compiler directives. To our knowledge this represents the first complete atmospheric model being run entirely on accelerators on this scale. At a grid spacing of 930 m (1.9 km), we achieve a simulation throughput of 0.043 (0.23) simulated years per day and an energy consumption of 596 MWh per simulated year. Furthermore, we propose a new memory usage efficiency (MUE) metric that considers how efficiently the memory bandwidth – the dominant bottleneck of climate codes – is being used.