Efficient and robust optimization of nuclear and electronic orbitals within the nuclear Hartree product representation

The nuclear–electronic orbital (NEO) approach treats electrons and specified nuclei quantum mechanically on the same level, providing a straightforward framework for including nuclear quantum effects beyond the standard Born–Oppenheimer approximation. Both density functional theory and wave function methods have been developed within the NEO framework. Despite its success, some NEO applications are limited by the intrinsically difficult self-consistent field (SCF) convergence of the Hartree–Fock or Kohn–Sham electronic and nuclear orbitals. Herein, we demonstrate that using the nuclear Hartree product representation improves the efficiency and stability of the NEO-SCF optimization procedure compared to the nuclear Slater determinant representation. Both representations lead to the same NEO-SCF energy because the proton–proton exchange energy is typically much smaller than the SCF energy convergence threshold. Faster and more robust NEO-SCF convergence is observed using this strategy for a test set of 92 molecules containing multiple quantum protons. We determine that the improved convergence behavior stems from the absence of nuclear self-Coulomb and self-exchange terms in the Hartree product representation. Furthermore, we show that both electronic and nuclear orbitals can be optimized simultaneously through extensions of the geometric direct minimization and trust-radius augmented Hessian algorithms. These algorithms can be useful for converging challenging systems, as demonstrated by calculations on the enzyme ribonucleotide reductase and the UO₂(OH)₄ complex. The findings in this work will enable more efficient and robust NEO calculations for a wide range of applications.