Cost-Aware Region-Level Data Placement in Multi-Tiered Parallel I/O Systems

Multi-tiered Parallel I/O systems that combine traditional HDDs with emerging SSDs mitigate the cost burden of SSDs while benefiting from their superior I/O performance. While a multi-tiered parallel I/O system is promising for data-intensive applications in high-performance (HPC) domains, placing d...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on parallel and distributed systems Vol. 28; no. 7; pp. 1853 - 1865
Main Authors He, Shuibing, Wang, Yang, Li, Zheng, Sun, Xian-He, Xu, Chenzhong
Format Journal Article
LanguageEnglish
Published New York IEEE 01.07.2017
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Computers
Data models
data placement
Distributed databases
Domains
Input output
Optimization
parallel file system
Parallel I/O system
Placement
Servers
Software
solid state drive
System performance
Windows (intervals)
Online AccessGet full text

Cover

More Information
Summary:Multi-tiered Parallel I/O systems that combine traditional HDDs with emerging SSDs mitigate the cost burden of SSDs while benefiting from their superior I/O performance. While a multi-tiered parallel I/O system is promising for data-intensive applications in high-performance (HPC) domains, placing data on each tier of the system to achieve high I/O performance remains a challenge. In this paper, we propose a cost-aware region-level (CARL) data placement scheme in multi-tiered parallel I/O systems. CARL divides a large file into several small regions, and then places regions on different types of servers based on region access costs. CARL includes a static policy S-CARL and a dynamic policy D-CARL. For applications whose I/O access patterns are completely known, S-CARL calculates the region costs within the entire workload duration, and uses a static data placement scheme to selectively place regions on the proper servers. To adapt to applications whose access patterns are unknown in advance, D-CARL uses a dynamic data placement scheme which migrates data among different servers within each time window. We have implemented CARL under MPI-IO library and OrangeFS parallel file system environment. Our evaluation with representative benchmarks and an application shows that CARL is both feasible and able to improve I/O performance significantly.
ISSN:1045-9219
1558-2183
DOI:10.1109/TPDS.2016.2636837