A Constraint Programming-Based Resource Management Technique for Processing MapReduce Jobs with SLAs on Clouds

• Published in: 2014 43rd International Conference on Parallel Processing pp. 411 - 421
• Main Authors: Lim, Norman, Majumdar, Shikharesh, Ashwood-Smith, Peter
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.09.2014
• Subjects: Cloud computing; Computational modeling; MapReduce with deadlines; MapReduce with SLAs; matchmaking and scheduling on clouds; Open systems; Optimization; Programming; Resource management; resource management on clouds; Schedules; SLAs on clouds
• DOI: 10.1109/ICPP.2014.50
• ISSN: 0190-3918

Clouds that are rapidly gaining in popularity require an effective resource manager that can harness the power of the underlying resource pool, and provide resources on demand to its users. This paper focuses on resource management on clouds for workflow requests characterized by Service Level Agreements (SLAs). Specifically, we devise a novel MapReduce constraint programming based resource manager (MRCP-RM) that can effectively perform matchmaking and scheduling of MapReduce jobs, each characterized by an SLA comprising an earliest start time, execution time, and an end-to-end deadline. Using discrete event simulation a performance evaluation of MRCP-RM is conducted for an open system subjected to a stream of job arrivals. The simulation results demonstrate the effectiveness of the resource manager and provide insights into system behaviour and performance.