Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives

Wenlu Zhang; Wayne Joubert; Peng Wang; Bei Wang; William Tang; Matthew Niemerg; Lei Shi; Sam Taimourzadeh; Jian Bao; Zhihong Lin

doi:10.1007/978-3-030-12274-4_1

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives

Wenlu Zhang
, Wayne Joubert
, Peng Wang
, Bei Wang
, William Tang
, Matthew Niemerg
, Lei Shi
, Sam Taimourzadeh
, Jian Bao
, Zhihong Lin

PPPL Computational Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Scopus citations

Abstract

The latest production version of the fusion particle simulation code, Gyrokinetic Toroidal Code (GTC), has been ported to and optimized for the next generation exascale GPU supercomputing platform. Heterogeneous programming using directives has been utilized to balance the continuously implemented physical capabilities and rapidly evolving software/hardware systems. The original code has been refactored to a set of unified functions/calls to enable the acceleration for all the species of particles. Extensive GPU optimization has been performed on GTC to boost the performance of the particle push and shift operations. In order to identify the hotspots, the code was the first benchmarked on up to 8000 nodes of the Titan supercomputer, which shows about 2–3 times overall speedup comparing NVidia M2050 GPUs to Intel Xeon X5670 CPUs. This Phase I optimization was followed by further optimizations in Phase II, where single-node tests show an overall speedup of about 34 times on SummitDev and 7.9 times on Titan. The real physics tests on Summit machine showed impressive scaling properties that reaches roughly 50% efficiency on 928 nodes of Summit. The GPU + CPU speed up from purely CPU is over 20 times, leading to an unprecedented speed.

Original language	English (US)
Title of host publication	Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings
Editors	Guido Juckeland, Sandra Wienke, Sunita Chandrasekaran
Publisher	Springer Verlag
Pages	3-21
Number of pages	19
ISBN (Print)	9783030122737
DOIs	https://doi.org/10.1007/978-3-030-12274-4_1
State	Published - 2019
Event	5th International Workshop on Accelerator Programming Using Directives, WACCPD 2018 - Dallas, United States Duration: Nov 11 2018 → Nov 17 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11381 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	5th International Workshop on Accelerator Programming Using Directives, WACCPD 2018
Country/Territory	United States
City	Dallas
Period	11/11/18 → 11/17/18

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Keywords

Directives
Fusion plasma
GPU
Heterogeneous programming
Massively parallel computing
OpenACC
Particle in cell

Access to Document

10.1007/978-3-030-12274-4_1

Cite this

Zhang, W., Joubert, W., Wang, P., Wang, B., Tang, W., Niemerg, M., Shi, L., Taimourzadeh, S., Bao, J., & Lin, Z. (2019). Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives. In G. Juckeland, S. Wienke, & S. Chandrasekaran (Eds.), Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings (pp. 3-21). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11381 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-12274-4_1

Zhang, Wenlu ; Joubert, Wayne ; Wang, Peng et al. / Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives. Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings. editor / Guido Juckeland ; Sandra Wienke ; Sunita Chandrasekaran. Springer Verlag, 2019. pp. 3-21 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d972ecb95b12472fb67aac0e6c422eb8,

title = "Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives",

abstract = "The latest production version of the fusion particle simulation code, Gyrokinetic Toroidal Code (GTC), has been ported to and optimized for the next generation exascale GPU supercomputing platform. Heterogeneous programming using directives has been utilized to balance the continuously implemented physical capabilities and rapidly evolving software/hardware systems. The original code has been refactored to a set of unified functions/calls to enable the acceleration for all the species of particles. Extensive GPU optimization has been performed on GTC to boost the performance of the particle push and shift operations. In order to identify the hotspots, the code was the first benchmarked on up to 8000 nodes of the Titan supercomputer, which shows about 2–3 times overall speedup comparing NVidia M2050 GPUs to Intel Xeon X5670 CPUs. This Phase I optimization was followed by further optimizations in Phase II, where single-node tests show an overall speedup of about 34 times on SummitDev and 7.9 times on Titan. The real physics tests on Summit machine showed impressive scaling properties that reaches roughly 50\% efficiency on 928 nodes of Summit. The GPU + CPU speed up from purely CPU is over 20 times, leading to an unprecedented speed.",

keywords = "Directives, Fusion plasma, GPU, Heterogeneous programming, Massively parallel computing, OpenACC, Particle in cell",

author = "Wenlu Zhang and Wayne Joubert and Peng Wang and Bei Wang and William Tang and Matthew Niemerg and Lei Shi and Sam Taimourzadeh and Jian Bao and Zhihong Lin",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 5th International Workshop on Accelerator Programming Using Directives, WACCPD 2018 ; Conference date: 11-11-2018 Through 17-11-2018",

year = "2019",

doi = "10.1007/978-3-030-12274-4\_1",

language = "English (US)",

isbn = "9783030122737",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "3--21",

editor = "Guido Juckeland and Sandra Wienke and Sunita Chandrasekaran",

booktitle = "Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings",

address = "Germany",

}

Zhang, W, Joubert, W, Wang, P, Wang, B, Tang, W, Niemerg, M, Shi, L, Taimourzadeh, S, Bao, J & Lin, Z 2019, Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives. in G Juckeland, S Wienke & S Chandrasekaran (eds), Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11381 LNCS, Springer Verlag, pp. 3-21, 5th International Workshop on Accelerator Programming Using Directives, WACCPD 2018, Dallas, United States, 11/11/18. https://doi.org/10.1007/978-3-030-12274-4_1

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives. / Zhang, Wenlu; Joubert, Wayne; Wang, Peng et al.
Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings. ed. / Guido Juckeland; Sandra Wienke; Sunita Chandrasekaran. Springer Verlag, 2019. p. 3-21 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11381 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives

AU - Zhang, Wenlu

AU - Joubert, Wayne

AU - Wang, Peng

AU - Wang, Bei

AU - Tang, William

AU - Niemerg, Matthew

AU - Shi, Lei

AU - Taimourzadeh, Sam

AU - Bao, Jian

AU - Lin, Zhihong

PY - 2019

Y1 - 2019

N2 - The latest production version of the fusion particle simulation code, Gyrokinetic Toroidal Code (GTC), has been ported to and optimized for the next generation exascale GPU supercomputing platform. Heterogeneous programming using directives has been utilized to balance the continuously implemented physical capabilities and rapidly evolving software/hardware systems. The original code has been refactored to a set of unified functions/calls to enable the acceleration for all the species of particles. Extensive GPU optimization has been performed on GTC to boost the performance of the particle push and shift operations. In order to identify the hotspots, the code was the first benchmarked on up to 8000 nodes of the Titan supercomputer, which shows about 2–3 times overall speedup comparing NVidia M2050 GPUs to Intel Xeon X5670 CPUs. This Phase I optimization was followed by further optimizations in Phase II, where single-node tests show an overall speedup of about 34 times on SummitDev and 7.9 times on Titan. The real physics tests on Summit machine showed impressive scaling properties that reaches roughly 50% efficiency on 928 nodes of Summit. The GPU + CPU speed up from purely CPU is over 20 times, leading to an unprecedented speed.

AB - The latest production version of the fusion particle simulation code, Gyrokinetic Toroidal Code (GTC), has been ported to and optimized for the next generation exascale GPU supercomputing platform. Heterogeneous programming using directives has been utilized to balance the continuously implemented physical capabilities and rapidly evolving software/hardware systems. The original code has been refactored to a set of unified functions/calls to enable the acceleration for all the species of particles. Extensive GPU optimization has been performed on GTC to boost the performance of the particle push and shift operations. In order to identify the hotspots, the code was the first benchmarked on up to 8000 nodes of the Titan supercomputer, which shows about 2–3 times overall speedup comparing NVidia M2050 GPUs to Intel Xeon X5670 CPUs. This Phase I optimization was followed by further optimizations in Phase II, where single-node tests show an overall speedup of about 34 times on SummitDev and 7.9 times on Titan. The real physics tests on Summit machine showed impressive scaling properties that reaches roughly 50% efficiency on 928 nodes of Summit. The GPU + CPU speed up from purely CPU is over 20 times, leading to an unprecedented speed.

KW - Directives

KW - Fusion plasma

KW - GPU

KW - Heterogeneous programming

KW - Massively parallel computing

KW - OpenACC

KW - Particle in cell

UR - https://www.scopus.com/pages/publications/85066152862

UR - https://www.scopus.com/pages/publications/85066152862#tab=citedBy

U2 - 10.1007/978-3-030-12274-4_1

DO - 10.1007/978-3-030-12274-4_1

M3 - Conference contribution

AN - SCOPUS:85066152862

SN - 9783030122737

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 3

EP - 21

BT - Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings

A2 - Juckeland, Guido

A2 - Wienke, Sandra

A2 - Chandrasekaran, Sunita

PB - Springer Verlag

T2 - 5th International Workshop on Accelerator Programming Using Directives, WACCPD 2018

Y2 - 11 November 2018 through 17 November 2018

ER -

Zhang W, Joubert W, Wang P, Wang B, Tang W, Niemerg M et al. Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives. In Juckeland G, Wienke S, Chandrasekaran S, editors, Accelerator Programming Using Directives - 5th International Workshop, WACCPD 2018, Proceedings. Springer Verlag. 2019. p. 3-21. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-12274-4_1

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code Using Directives

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this