Gfarm/BB — Gfarm File System for Node-Local Burst Buffer

• Published in: Journal of computer science and technology Vol. 35; no. 1; pp. 61 - 71
• Main Authors: Tatebe, Osamu, Moriwake, Shukuko, Oyama, Yoshihiro
• Format: Journal Article
• Language: English
• Published: New York Springer US 2020; Springer; Springer Nature B.V; Center for Computational Sciences, University of Tsukuba, Ibaraki 3058577, Japan%SURIGIKEN Co., Ltd., Tokyo 1000004, Japan%Faculty of Engineering, Information and Systems, University of Tsukuba, Ibaraki 3058573, Japan
• Subjects: Artificial Intelligence; Benchmarks; Buffers; Bursty traffic; Computer Science; Data Structures and Information Theory; Information Systems Applications (incl.Internet); Metadata; Nodes; Redundancy; Regular Paper; Software Engineering; Storage systems; Theory of Computation; on-demand file system; burst buffer; node-local storage; remote direct memory access
• ISSN: 1000-9000; 1860-4749
• DOI: 10.1007/s11390-020-9803-z

Burst buffer has become a major component to meet the I/O performance requirement of HPC bursty traffic. This paper proposes Gfarm/BB that is a file system for a burst buffer efficiently exploiting node-local storage systems. Although node-local storages improve storage performance, they are only available during the job allocation. Gfarm/BB should have better access and metadata performance while it should be constructed on-demand before the job execution. To improve the read and write performance, it exploits the file descriptor passing and remote direct memory access (RDMA). It improves the metadata performance by omitting the persistency and the redundancy since it is a temporal file system. Using RDMA, writes and reads bandwidth are improved by 1.7x and 2.2x compared with IP over InfiniBand (IPoIB), respectively. It achieves 14 700 operations per second in the directory creation performance, which is 13.4x faster than the fully persistent and redundant case. The construction of Gfarm/BB takes 0.31 seconds using 2 nodes. IOR benchmark and ARGOT-IO application I/O benchmark show the scalable performance improvement by exploiting the locality of node-local storages. Compared with BeeOND, Gfarm/BB shows 2.6x and 2.4x better performance in IOR write and read benchmarks, respectively, and it shows 2.5x better performance in ARGOT-IO.