From Scripted HPC-Based NGS Pipelines to Workflows on the Cloud

In this paper we describe our initial experiences in the Cloud-e-Genome project with moving the whole exome sequencing pipeline from the scripted HPC-based solution to a workflow enactment system running in the cloud. We discuss shortcomings of the existing approach based on scripts and list benefit...

Full description

Saved in:
Bibliographic Details
Published in2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing pp. 694 - 700
Main Authors Cala, Jacek, Yaobo Xu, Wijaya, Eldarina Azfar, Missier, Paolo
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.05.2014
Subjects
Bioinformatics
Cloud computing
Computational modeling
Genomics
Libraries
Pipelines
Online AccessGet full text

Cover

More Information
Summary:In this paper we describe our initial experiences in the Cloud-e-Genome project with moving the whole exome sequencing pipeline from the scripted HPC-based solution to a workflow enactment system running in the cloud. We discuss shortcomings of the existing approach based on scripts and list benefits that a workflow-based solution can provide. Despite the effort it involved to wrap all required tools in the form of workflow blocks and the restrictions of the dataflow model used to represent workflows we expect the migration to significantly improve the current status of the pipeline. Our target is to enable flexibility, traceability and reproducibility of the solution, so that it can better fit the evolution of tools, data and pipeline itself and allow us to run it at national scale. This work will become foundation for the more complete system that includes variant filtering and interpretation for the diagnostic purposes.
DOI:10.1109/CCGrid.2014.128