Reliable and Efficient Distributed Checkpointing System for Grid Environments

• Published in: Journal of grid computing Vol. 12; no. 4; pp. 593 - 613
• Main Authors: Islam, Tanzima Zerin, Bagchi, Saurabh, Eigenmann, Rudolf
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.12.2014; Springer; Springer Nature B.V
• Subjects: Algorithms; Applied sciences; checkpoint; checkpoint/restart; Checkpointing; Coding, codes; Computer Science; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Computer systems performance. Reliability; condor; cycle sharing system; data parallel checkpointing; Disks; efficient; erasure encoding; Exact sciences and technology; Failure; Failure analysis; FGCS; Geographical distribution; Information, signal and communications theory; Management of Computing and Information Systems; Mathematical models; Performance degradation; Processor Architectures; recovery; reliability; Repositories; Resource scheduling; Servers; Signal and communications theory; Software; Telecommunications and information theory; User Interfaces and Human Computer Interaction; Variations; Checkpointing; Erasure encoding; FGCS; Condor; Checkpoint; Checkpoint/Restart; Cycle sharing system; Efficient; Reliability; Data parallel checkpointing; Recovery; Grid; Distributed computing; Modeling; Delay; Execution time; Availability; High performance; Transmission channel; Resource sharing; Distributed system; Lead time; University; Parallelism; Error correcting code; Central unit
• ISSN: 1570-7873; 1572-9184
• DOI: 10.1007/s10723-014-9297-4

In Fine-Grained Cycle Sharing (FGCS) systems, machine owners voluntarily share their unused CPU cycles with guest jobs, as long as their performance degradation is tolerable. However, unpredictable evictions of guest jobs lead to fluctuating completion times. Checkpoint-recovery is an attractive mechanism for recovering from such “failures”. Today’s FGCS systems often use expensive, high-performance dedicated checkpoint servers. However, in geographically distributed clusters, this may incur high checkpoint transfer latencies. In this paper we present a distributed checkpointing system called Falcon that uses available disk resources of the FGCS machines as shared checkpoint repositories. However, an unavailable storage host may lead to loss of checkpoint data. Therefore, we model the failures of a storage host and develop a prediction algorithm for choosing reliable checkpoint repositories. We experiment with Falcon in the university-wide Condor testbed at Purdue and show improved and consistent performance for guest jobs in the presence of irregular resource availability.