Job-Site Level Fault Tolerance for Cluster and Grid environments

• Published in: 2005 IEEE International Conference on Cluster Computing pp. 1 - 9
• Main Authors: Limaye, K., Leangsuksun, B., Greenwood, Z., Scott, S.L., Engelmann, C., Libby, R., Chanchio, K.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.09.2005
• Subjects: Aggregates; Availability; Collaborative work; Contracts; Distributed computing; Fault tolerance; Fault tolerant systems; Grid computing; Mission critical systems; Scheduling
• ISBN: 9780780394858; 0780394852
• ISSN: 1552-5244; 2168-9253
• DOI: 10.1109/CLUSTR.2005.347043

In order to adopt high performance clusters and grid computing for mission critical applications, fault tolerance is a necessity. Common fault tolerance techniques in distributed systems are normally achieved with checkpoint-recovery and job replication on alternative resources, in cases of a system outage. The first approach depends on the system's MTTR while the latter approach depends on the availability of alternative sites to run replicas. There is a need for complementing these approaches by proactively handling failures at a job-site level, ensuring the system high availability with no loss of user submitted jobs. This paper discusses a novel fault tolerance technique that enables the job-site recovery in Beowulf cluster-based grid environments, whereas existing techniques give up a failed system by seeking alternative resources. Our results suggest sizable aggregate performance improvement during an implementation of our method in Globus-enabled HA-OSCAR. The technique called ''smart failover" provides a transparent and graceful recovery mechanism that saves job states in a local job-manager queue and transfers those states to the backup server periodically, and in critical system events. Thus whenever a failover occurs, the backup server is able to restart the jobs from their last saved state