TY - JOUR
T1 - Performance-portable Numerical Relativity with AthenaK
AU - Zhu, Hengrui
AU - Fields, Jacob
AU - Zappa, Francesco
AU - Radice, David
AU - Stone, James M.
AU - Rashti, Alireza
AU - Cook, William
AU - Bernuzzi, Sebastiano
AU - Daszuta, Boris
N1 - Publisher Copyright:
© 2025. The Author(s). Published by the American Astronomical Society.
PY - 2025/6/1
Y1 - 2025/6/1
N2 - We present the numerical relativity module within AthenaK, an open-source performance-portable astrophysics code designed for exascale computing applications. This module employs the Z4c formulation to solve the Einstein equations. We demonstrate its accuracy through a series of standard numerical relativity tests, including convergence of the gravitational waveform from binary black hole coalescence. Furthermore, we conduct scaling tests on OLCF Frontier, NERSC Perlmutter, and ALCF Aurora, where AthenaK exhibits excellent weak-scaling efficiency of 80% on up to 65,536 AMD MI250X GPUs on Frontier (relative to four GPUs) and 67% on Aurora up to 24,576 Intel Data Center Max Series GPUs (relative to 12 GPUs) and strong-scaling efficiencies of 84% and 77% on AMD MI250X and NVIDIA A100 GPUs on Frontier and Perlmutter, respectively. Additionally, we observe a significant performance boost, with 2 orders of magnitude speedup (≳200×) on a GPU compared to a single CPU core, affirming that AthenaK is well suited for exascale computing, and thereby expanding the potential for breakthroughs in numerical relativity research.
AB - We present the numerical relativity module within AthenaK, an open-source performance-portable astrophysics code designed for exascale computing applications. This module employs the Z4c formulation to solve the Einstein equations. We demonstrate its accuracy through a series of standard numerical relativity tests, including convergence of the gravitational waveform from binary black hole coalescence. Furthermore, we conduct scaling tests on OLCF Frontier, NERSC Perlmutter, and ALCF Aurora, where AthenaK exhibits excellent weak-scaling efficiency of 80% on up to 65,536 AMD MI250X GPUs on Frontier (relative to four GPUs) and 67% on Aurora up to 24,576 Intel Data Center Max Series GPUs (relative to 12 GPUs) and strong-scaling efficiencies of 84% and 77% on AMD MI250X and NVIDIA A100 GPUs on Frontier and Perlmutter, respectively. Additionally, we observe a significant performance boost, with 2 orders of magnitude speedup (≳200×) on a GPU compared to a single CPU core, affirming that AthenaK is well suited for exascale computing, and thereby expanding the potential for breakthroughs in numerical relativity research.
UR - https://www.scopus.com/pages/publications/105007757787
UR - https://www.scopus.com/pages/publications/105007757787#tab=citedBy
U2 - 10.3847/1538-4365/adcf96
DO - 10.3847/1538-4365/adcf96
M3 - Article
AN - SCOPUS:105007757787
SN - 0067-0049
VL - 278
JO - Astrophysical Journal, Supplement Series
JF - Astrophysical Journal, Supplement Series
IS - 2
M1 - 50
ER -