Development of QMCPACK for Exascale Scientific Computing
Anouar Benali, David M. Ceperley, Eduardo F. D'Azevedo, Mark Dewing, Paul R. C. Kent, Jeongnim Kim, Jaron T. Krogel, Ying Wai Li, Ye Luo, Tyler McDaniel, Miguel A. Morales, Amrita Mathuriya, Luke Shulenburger, Norm M. Tubman
This chapter outlines what drives the choice of algorithms used in quantum Monte Carlo (QMC) and QMCPACK. Internally, QMC codes are similar to particle-based classical molecular dynamics codes with the addition of a limited amount of simple dense linear algebra. The scientific possibilities for QMC with exascale computing will lead too many new applications Availability of exascale hardware will naturally alleviate the floating-point limitation on system size, N, but the memory limitations are expected to become more complex. The optimizations were supported and realized through an Intel Parallel Computing Center at Argonne National Laboratory and Sandia National Laboratories and a full article describing the methods can be found in. Unit testing enables projects to scale to a larger number of developers, and provides assurances to developers that changes are not breaking the code. Unit tests also help design by driving refactoring, and can lead to a less tightly-coupled architecture.
This chapter outlines what drives the choice of algorithms used in quantum Monte Carlo (QMC) and QMCPACK. Internally, QMC codes are similar to particle-based classical molecular dynamics codes with the addition of a limited amount of simple dense linear algebra. The scientific possibilities for QMC with exascale computing will lead too many new applications Availability of exascale hardware will naturally alleviate the floating-point limitation on system size, N, but the memory limitations are expected to become more complex. The optimizations were supported and realized through an Intel Parallel Computing Center at Argonne National Laboratory and Sandia National Laboratories and a full article describing the methods can be found in. Unit testing enables projects to scale to a larger number of developers, and provides assurances to developers that changes are not breaking the code. Unit tests also help design by driving refactoring, and can lead to a less tightly-coupled architecture.
With advances in algorithms and growing computing powers, quantum Monte Carlo (QMC) methods have become a leading contender for high accuracy calculations for the electronic structure of realistic systems. The performance gain on recent HPC systems is largely driven by increasing parallelism: the number of compute cores of a SMP and the number of SMPs have been going up, as the Top500 list attests. However, the available memory as well as the communication and memory bandwidth per element has not kept pace with the increasing parallelism. This severely limits the applicability of QMC and the problem size it can handle. OpenMP/MPI hybrid programming provides applications with simple but effective solutions to overcome efficiency and scalability bottlenecks on large-scale clusters based on multi/many-core SMPs. We discuss the design and implementation of hybrid methods in QMCPACK and analyze its performance on current HPC platforms characterized by various memory and communication hierarchies.
There are three different Quantum Monte Carlo (QMC) codes in the Targeting REal chemical accuracy at the eXascale (TREX) Center of Excellence (CoE), TurboRVB, Champ, and QMC=Chem, each with its own strengths and weaknesses. Instead of optimizing the three codes independently for exascale architectures, or re-writing a new monolithic code, our strategy is instead to design a new library, Quantum Monte Carlo kernel library (QMCkl), containing the state-of-the-art expertise in the implementation of specific kernels present in each of the three codes. The functions available in this library will allow all the codes to benefit from the optimal implementation of the major kernels of QMC. The three main objectives driving the development of QMCkl are performance, productivity and portability.
Quantum Monte Carlo (QMC) methods sample the wave function, in principle the exact one, instead of optimizing analytical functions as standard ab initio approaches. They have emerged as suitable alternative to ab initio methods to deal with the dynamical correlation in the description of the electronic structure of atoms and molecules. Differently from standard quantum chemistry approaches, QMC enjoys several features that make it the method of choice for complicated systems. Among these features, it is important to notice its intrinsically parallelizable nature, the slow (˜N 3 ) scaling of the computational cost with number of particles N, the limited amount of memory required and its ability to deal with substantially different systems within the same theoretical/algorithmic structure (e.g. it can be easily applied to both bosons and fermions, to the description of systems containing electrons and positrons as well as of vibrational properties of molecules). One of the strongest points characterizing QMC is the fact that it may use any kind of basis sets, albeit uncommon ones, depending on the species. This fact allows QMC either to quickly converge to the exact answer in the case of bosonic systems, or to easily recover 90–95% of the correlation energy in electronic species without the inverse cubic convergence with respect to the size of the basis set that plagues more common ab initio methods. Thanks to these characteristics, different flavours of QMC have been applied to a wide set of species/problems spanning a range that stretches from molecules as small as water up to pieces of bulk matter as large as silicon and germanium crystals and that includes molecules such as porphyrins and C 20 . Despite the strong points highlighted, QMC suffers from a relatively high computational cost mainly due to the necessity of evaluating many times a reference wave function. Thus, a substantial reduction of this cost may come from central processing units (CPU) particularly tailored to compute exponentials, polynomials and rational functions, from better algorithms that require less function evaluations and from variance reduction techniques. Besides, more robust approaches to reduce the so called “nodal error” would help improve the already appreciable accuracy afforded by QMC. With the above issues improved, there are several avenues that would become possible to pursue on a routine basis. Among these, we foresee the calculations of intermolecular interactions for very large systems (e.g. parts of DNA with or without interacting species), the calculation of nuclear magnetic resonance (NMR) parameters for difficult systems, the automatic optimization of molecular structures and, even better, the chance of running molecular dynamics simulation à la Car–Parrinello using QMC computed atomic forces. In these circumstances, the study of phase transitions, bulk matter, interfaces and large biological systems may reach an unexpected level of accuracy that is currently unavailable due to methodological limitations.
QMCPACK is an open source quantum Monte Carlo package for ab initio electronic structure calculations. It supports calculations of metallic and insulating solids, molecules, atoms, and some model Hamiltonians. Implemented real space quantum Monte Carlo algorithms include variational, diffusion, and reptation Monte Carlo. QMCPACK uses Slater-Jastrow type trial wavefunctions in conjunction with a sophisticated optimizer capable of optimizing tens of thousands of parameters. The orbital space auxiliary-field quantum Monte Carlo method is also implemented, enabling cross validation between different highly accurate methods. The code is specifically optimized for calculations with large numbers of electrons on the latest high performance computing architectures, including multicore central processing unit and graphical processing unit systems. We detail the program's capabilities, outline its structure, and give examples of its use in current research calculations. The package is available at http://qmcpack.org.
Various strategies to implement efficiently quantum Monte Carlo (QMC) simulations for large chemical systems are presented. These include: (i) the introduction of an efficient algorithm to calculate the computationally expensive Slater matrices. This novel scheme is based on the use of the highly localized character of atomic Gaussian basis functions (not the molecular orbitals as usually done), (ii) the possibility of keeping the memory footprint minimal, (iii) the important enhancement of single-core performance when efficient optimization tools are used, and (iv) the definition of a universal, dynamic, fault-tolerant, and load-balanced framework adapted to all kinds of computational platforms (massively parallel machines, clusters, or distributed grids). These strategies have been implemented in the QMC=Chem code developed at Toulouse and illustrated with numerical applications on small peptides of increasing sizes (158, 434, 1056, and 1731 electrons). Using 10-80 k computing cores of the Curie machine (GENCI-TGCC-CEA, France), QMC=Chem has been shown to be capable of running at the petascale level, thus demonstrating that for this machine a large part of the peak performance can be achieved. Implementation of large-scale QMC simulations for future exascale platforms with a comparable level of efficiency is expected to be feasible.
We review recent advances in the capabilities of the open source ab initio Quantum Monte Carlo (QMC) package QMCPACK and the workflow tool Nexus used for greater efficiency and reproducibility. The auxiliary field QMC (AFQMC) implementation has been greatly expanded to include k-point symmetries, tensor-hypercontraction, and accelerated graphical processing unit (GPU) support. These scaling and memory reductions greatly increase the number of orbitals that can practically be included in AFQMC calculations, increasing the accuracy. Advances in real space methods include techniques for accurate computation of bandgaps and for systematically improving the nodal surface of ground state wavefunctions. Results of these calculations can be used to validate application of more approximate electronic structure methods, including GW and density functional based techniques. To provide an improved foundation for these calculations, we utilize a new set of correlation-consistent effective core potentials (pseudopotentials) that are more accurate than previous sets; these can also be applied in quantum-chemical and other many-body applications, not only QMC. These advances increase the efficiency, accuracy, and range of properties that can be studied in both molecules and materials with QMC and QMCPACK.