Large-database cross-verification and validation of tokamak transport models using baselines for comparison

State-of-the-art 1D transport solvers ASTRA and TRANSP are verified, then validated across a large database of semi-randomly selected, time-dependent DIII-D discharges. Various empirical models are provided as baselines to contextualize the validation figures of merit using statistical hypothesis tests. For predicting plasma temperature profiles, no statistically significant advantage is found for the ASTRA and TRANSP simulators over a baseline empirical (two-parameter) model. For predicting stored energy, a significant advantage is found for the simulators over a baseline empirical model based on confinement time scaling. Uncertainty in the results due to diagnostic and profile fitting uncertainties is approximated and determined to be insignificant due in part to the large quantity of discharges employed in the study. Advantages are discussed for validation methodologies like this one that employ (1) large databases and (2) baselines for comparison that are specific to the intended use-case of the model.