Analyzing I/O Performance of a Hierarchical HPC Storage System for Distributed Deep Learning

Today, deep learning is an essential technology for our life. To solve more complex problems with deep learning, both sizes of training datasets and neural networks are increasing. To train a model with large datasets and networks, distributed deep neural network (DDNN) training technique is necessa...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Fukai, Takaaki, Sato, Kento, Hirofuchi, Takahiro
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 04.01.2023
Subjects
Artificial neural networks
Clusters
Datasets
Deep learning
Input output analysis
Machine learning
Storage systems
Training
Workload
Online AccessGet full text

Cover

More Information
Summary:Today, deep learning is an essential technology for our life. To solve more complex problems with deep learning, both sizes of training datasets and neural networks are increasing. To train a model with large datasets and networks, distributed deep neural network (DDNN) training technique is necessary. For large-scale DDNN training, HPC clusters are a promising computation environment. In large-scale DDNN on HPC clusters, I/O performance is critical because it is becoming a bottleneck. Most flagship-class HPC clusters have hierarchical storage systems. For designing future HPC storage systems, it is necessary to quantify the performance improvement effect of the hierarchical storage system on the workloads. This paper demonstrates the quantitative performance analysis of the hierarchical storage system for DDNN workload in a flagship-class supercomputer. Our analysis shows how much performance improvement and volume increment of the storage will be required to meet the performance goal.
ISSN:2331-8422