The Widening Gulf between Genomics Data Generation and Consumption: A Practical Guide to Big Data Transfer Technology
In the last decade, high-throughput DNA sequencing has become a disruptive technology and pushed the life sciences into a distributed ecosystem of sequence data producers and consumers. Given the power of genomics and declining sequencing costs, biology is an emerging “Big Data” discipline that will...
Saved in:
Published in | Bioinformatics and Biology Insights Vol. 2015; no. S1; pp. 9 - 19 |
---|---|
Main Authors | , , , , , , , |
Format | Journal Article Book Review |
Language | English |
Published |
London, England
Libertas Academica
01.01.2015
SAGE Publishing SAGE Publications |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | In the last decade, high-throughput DNA sequencing has become a disruptive technology and pushed the life sciences into a distributed ecosystem of sequence data producers and consumers. Given the power of genomics and declining sequencing costs, biology is an emerging “Big Data” discipline that will soon enter the exabyte data range when all subdisciplines are combined. These datasets must be transferred across commercial and research networks in creative ways since sending data without thought can have serious consequences on data processing time frames. Thus, it is imperative that biologists, bioinformaticians, and information technology engineers recalibrate data processing paradigms to fit this emerging reality. This review attempts to provide a snapshot of Big Data transfer across networks, which is often overlooked by many biologists. Specifically, we discuss four key areas: 1) data transfer networks, protocols, and applications; 2) data transfer security including encryption, access, firewalls, and the Science DMZ; 3) data flow control with software-defined networking; and 4) data storage, staging, archiving and access. A primary intention of this article is to orient the biologist in key aspects of the data transfer process in order to frame their genomics-oriented needs to enterprise IT professionals. |
---|---|
Bibliography: | ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Feature-3 content type line 23 ObjectType-Review-1 |
ISSN: | 1177-9322 1177-9322 |
DOI: | 10.4137/BBI.S28988 |