Scalable workflow-driven hydrologic analysis in hydroframe

Shweta Purawat, Cathie Olschanowsky, Laura E. Condon, Reed Maxwell, Ilkay Altintas

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The HydroFrame project is a community platform designed to facilitate integrated hydrologic modeling across the US. As a part of HydroFrame, we seek to design innovative workflow solutions that create pathways to enable hydrologic analysis for three target user groups: the modeler, the analyzer, and the domain science educator. We present the initial progress on the HydroFrame community platform using an automated Kepler workflow. This workflow performs end-to-end hydrology simulations involving data ingestion, preprocessing, analysis, modeling, and visualization. We demonstrate how different modules of the workflow can be reused and repurposed for the three target user groups. The Kepler workflow ensures complete reproducibility through a built-in provenance framework that collects workflow specific parameters, software versions, and hardware system configuration. In addition, we aim to optimize the utilization of large-scale computational resources to adjust to the needs of all three user groups. Towards this goal, we present a design that leverages provenance data and machine learning techniques to predict performance and forecast failures using an automatic performance collection component of the pipeline.

Original languageEnglish (US)
Title of host publicationComputational Science – ICCS 2020 - 20th International Conference, Proceedings
EditorsValeria V. Krzhizhanovskaya, Gábor Závodszky, Michael H. Lees, Peter M.A. Sloot, Peter M.A. Sloot, Peter M.A. Sloot, Jack J. Dongarra, Sérgio Brissos, João Teixeira
PublisherSpringer
Pages276-289
Number of pages14
ISBN (Print)9783030503703
DOIs
StatePublished - 2020
Externally publishedYes
Event20th International Conference on Computational Science, ICCS 2020 - Amsterdam, Netherlands
Duration: Jun 3 2020Jun 5 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12137 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Computational Science, ICCS 2020
CountryNetherlands
CityAmsterdam
Period6/3/206/5/20

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Keywords

  • Computational hydrology
  • Machine learning
  • Reproducibility
  • Scientific Workflow

Fingerprint Dive into the research topics of 'Scalable workflow-driven hydrologic analysis in hydroframe'. Together they form a unique fingerprint.

Cite this