Towards a Heterogeneous Compute Engine for Software Acceleration on Big Data Infrastructures

Big Data processing requires significant amount of computing resources that are typically distributed in traditional clusters or modern cloud infrastructures. Software stacks, such as Hadoop, have been evolved for managing distributed resources, schedule workloads, support data storage and sharing....

Full description

Saved in:
Bibliographic Details
Published in2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing pp. 1533 - 1537
Main Authors Margiolas, Christos, Manousakis, Ioannis
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2015
Subjects
Acceleration
Big data
Cloud computing
Computer architecture
Engines
Hadoop
Heterogeneous Scheduling
Processor scheduling
Online AccessGet full text

Cover

More Information
Summary:Big Data processing requires significant amount of computing resources that are typically distributed in traditional clusters or modern cloud infrastructures. Software stacks, such as Hadoop, have been evolved for managing distributed resources, schedule workloads, support data storage and sharing. However, these stacks focus on homogeneous node architectures and only consider CPU processors while ignoring the popular trend of computational accelerators, such as GPUs. The computational power and energy efficiency of accelerators make them desirable components for Big Data infrastructures. This paper addresses this issue and enables accelerator integration in Big Data software stacks. We present preliminary work on the design and implementation of a Heterogeneous Compute Engine that allows heterogeneous parallel processing on Big Data infrastructures. Our work seamlessly integrates with Hadoop and enables computation acceleration.
DOI:10.1109/CIT/IUCC/DASC/PICOM.2015.230