TY - GEN
T1 - Scalable workflow-driven hydrologic analysis in hydroframe
AU - Purawat, Shweta
AU - Olschanowsky, Cathie
AU - Condon, Laura E.
AU - Maxwell, Reed
AU - Altintas, Ilkay
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2020
Y1 - 2020
N2 - The HydroFrame project is a community platform designed to facilitate integrated hydrologic modeling across the US. As a part of HydroFrame, we seek to design innovative workflow solutions that create pathways to enable hydrologic analysis for three target user groups: the modeler, the analyzer, and the domain science educator. We present the initial progress on the HydroFrame community platform using an automated Kepler workflow. This workflow performs end-to-end hydrology simulations involving data ingestion, preprocessing, analysis, modeling, and visualization. We demonstrate how different modules of the workflow can be reused and repurposed for the three target user groups. The Kepler workflow ensures complete reproducibility through a built-in provenance framework that collects workflow specific parameters, software versions, and hardware system configuration. In addition, we aim to optimize the utilization of large-scale computational resources to adjust to the needs of all three user groups. Towards this goal, we present a design that leverages provenance data and machine learning techniques to predict performance and forecast failures using an automatic performance collection component of the pipeline.
AB - The HydroFrame project is a community platform designed to facilitate integrated hydrologic modeling across the US. As a part of HydroFrame, we seek to design innovative workflow solutions that create pathways to enable hydrologic analysis for three target user groups: the modeler, the analyzer, and the domain science educator. We present the initial progress on the HydroFrame community platform using an automated Kepler workflow. This workflow performs end-to-end hydrology simulations involving data ingestion, preprocessing, analysis, modeling, and visualization. We demonstrate how different modules of the workflow can be reused and repurposed for the three target user groups. The Kepler workflow ensures complete reproducibility through a built-in provenance framework that collects workflow specific parameters, software versions, and hardware system configuration. In addition, we aim to optimize the utilization of large-scale computational resources to adjust to the needs of all three user groups. Towards this goal, we present a design that leverages provenance data and machine learning techniques to predict performance and forecast failures using an automatic performance collection component of the pipeline.
KW - Computational hydrology
KW - Machine learning
KW - Reproducibility
KW - Scientific Workflow
UR - http://www.scopus.com/inward/record.url?scp=85087533625&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087533625&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-50371-0_20
DO - 10.1007/978-3-030-50371-0_20
M3 - Conference contribution
AN - SCOPUS:85087533625
SN - 9783030503703
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 276
EP - 289
BT - Computational Science – ICCS 2020 - 20th International Conference, Proceedings
A2 - Krzhizhanovskaya, Valeria V.
A2 - Závodszky, Gábor
A2 - Lees, Michael H.
A2 - Sloot, Peter M.A.
A2 - Sloot, Peter M.A.
A2 - Sloot, Peter M.A.
A2 - Dongarra, Jack J.
A2 - Brissos, Sérgio
A2 - Teixeira, João
PB - Springer Science and Business Media Deutschland GmbH
T2 - 20th International Conference on Computational Science, ICCS 2020
Y2 - 3 June 2020 through 5 June 2020
ER -