Performance Implications of SSDs in Virtualized Hadoop Clusters

BigData manipulates a massive volume of data for which the traditional techniques are not effective. Apache Hadoop is currently a most popular software framework supporting BigData analysis. As the scale of Hadoop cluster grows larger, building Hadoop clusters in virtualized environment draws a grea...

Full description

Saved in:
Bibliographic Details
Published in2014 IEEE International Congress on Big Data pp. 586 - 593
Main Authors Sungyong Ahn, Sangkyu Park, Jae-Ki Hong, Wooseok Chang
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2014
Subjects
Bandwidth
Benchmark testing
BigData
Cloud computing
Degradation
Hadoop
Performance evaluation
Servers
SSD
Virtual machine monitors
Virtualization
Online AccessGet full text

Cover

More Information
Summary:BigData manipulates a massive volume of data for which the traditional techniques are not effective. Apache Hadoop is currently a most popular software framework supporting BigData analysis. As the scale of Hadoop cluster grows larger, building Hadoop clusters in virtualized environment draws a great attention. However, the performance optimization of Hadoop cluster in virtualized environment is difficult because of the virtualization overhead. In this paper the performance implications of SSDs in virtualized Hadoop clusters is identified and the overhead of virtualization is shown to be minimized with SSDs. The study presented in this paper reveals that the main virtualization overhead is I/O bottleneck due to fragmented and randomized I/O workload aggravated by virtualization. However, SSDs are more tolerable to the workload than HDDs. As a result, the virtualization overhead with SSDs is much less than with HDDs. Also, in the case of SSDs, the virtualized Hadoop cluster sustains good performance regardless of the number of VMs.
ISSN:2379-7703
DOI:10.1109/BigData.Congress.2014.90