In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement a...

Full description

Saved in:
Bibliographic Details
Published in2016 Second International Workshop on Extreme Scale Programming Models and Middlewar (ESPM2) pp. 2 - 9
Main Authors Qian Sun, Romanus, Melissa, Tong Jin, Hongfeng Yu, Bremer, Peer-Timo, Petruzza, Steve, Klasky, Scott, Parashar, Manish
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.11.2016
Subjects
Adaptation models
Couplings
Data models
Data processing
Data storage systems
Distributed databases
Monitoring
Runtime
Online AccessGet full text

Cover

More Information
Summary:Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.
DOI:10.1109/ESPM2.2016.006