Enabling Prioritized Cloud I/O Service in Hadoop Distributed File System

Cloud computing has become more and more popular nowadays. Both governments and enterprises provide service through the construction of public and private clouds accordingly. Among the platforms used in cloud computing, Hadoop is considered one of the most practical and stable systems. Nevertheless,...

Full description

Saved in:
Bibliographic Details
Published in2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS) pp. 256 - 259
Main Authors Tsozen Yeh, Yifeng Sun
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.08.2014
Subjects
Benchmark testing
Cloud computing
Computers
Conferences
Hadoop
HDFS
Operating systems
Writing
Online AccessGet full text

Cover

More Information
Summary:Cloud computing has become more and more popular nowadays. Both governments and enterprises provide service through the construction of public and private clouds accordingly. Among the platforms used in cloud computing, Hadoop is considered one of the most practical and stable systems. Nevertheless, as with other regular software, Hadoop still needs to rely on the underlying operating system to communicate with hardware to function appropriately. For modern computer systems, CPUs excessively outrun hard drives (hard disks). The computer hard disk has become a major bottleneck to the overall system performance. Consequently, computer programs can execute faster if their corresponding I/O operation can be completed sooner. This is important in particular when we want to expedite the execution of urgent programs in a busy system. Unfortunately, under the current Hadoop environment, users cannot prioritize operation of disk and memory for programs which they would like them to run faster. With the help of prioritized I/O service we developed earlier, we proposed and implemented a Hadoop environment with the ability of providing prioritized I/O service. Our Hadoop environment could accelerate the execution of programs with high priority assigned by users. We evaluated our design by executing prioritized programs in environments with different busy levels. Experimental results show that programs can improve their performance by up to 33.79% if executed with high priority.
DOI:10.1109/HPCC.2014.45