Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Fast Semistochastic Heat-Bath Configuration Interaction

Junhao Li

Published Jan 1, 2019• 129 citations
Publisher

Abstract

In this thesis, I present my work on the fast semistochastic heatbath configuration interaction (Fast SHCI), which is an efficient algorithm for doing essentially exact electronic structure calculations within a finite basis. There are Hamiltonians for which the entire Hilbert space is enormous, but the important part of Hilbert space is of manageable size, say $10^{12}$. Quantum chemistry Hamiltonians, for reasonably small systems, have this property. For such Hamiltonians, selected configuration interaction plus perturbation theory (SCI+PT) methods can be useful. The most important part of the Hilbert space is treated variationally, and the resulting energy is improved by using perturbation theory. Fast SHCI is more than an order of magnitude faster than other SCI+PT algorithms, and also much faster than other essentially exact algorithms for many chemical systems. This thesis provides an in-depth description of the Fast SHCI algorithm and its implementation. I use SHCI to compute the electronic structure of several chemical systems and the homogeneous electron gas. Some of these calculations are more accurate than those achieved by other high-order quantum chemistry methods. Others treat systems larger than those that can be treated by other equally accurate methods. My implementation of SHCI uses a modular design, which not only makes the library highly extensible but also contributes several generic distributed computing building blocks to the open-source community. In this thesis, I also describe my design and implementation of these generic components. Finally, I also provide a brief discussion of the usability of general software engineering best practices for the development of medium-scale scientific software packages with lessons learned from designing, developing, and leading the development of our SHCI package. Medium-scale scientific software packages are common in scientific research where a small group of researchers works on the same code base. Due to the differences in the requirements, some best practices that are common in the industry need to be adjusted to be useful for these projects.

DOI
10.7298/xnha-nn23

Related papers

Fast semistochastic heat-bath configuration interaction

Junhao Li, Matthew Otten, Adam A. Holmes +2 more

This paper presents in detail our fast semistochastic heat-bath configuration interaction (SHCI) method for solving the many-body Schrödinger equation. We identify and eliminate computational bottlenecks in both the variational and perturbative steps of the SHCI algorithm. We also describe the parallelization and the key data structures in our implementation, such as the distributed hash table. The improved SHCI algorithm enables us to include in our variational wavefunction two orders of magnitude more determinants than has been reported previously with other selected configuration interaction methods. We use our algorithm to calculate an accurate benchmark energy for the chromium dimer with the X2C relativistic Hamiltonian in the cc-pVDZ-DK basis, correlating 28 electrons in 76 spatial orbitals. Our largest calculation uses two billion Slater determinants in the variational space and semistochastically includes perturbative contributions from at least trillions of additional determinants with better than 10−5 Ha statistical uncertainty.

arXiv:1809.04600•The Journal of Chemical Physics•Dec 6, 2018•159 citations

Almost exact energies for the Gaussian-2 set with the semistochastic heat-bath configuration interaction method

Yuan Yao, Emmanuel Giner, Junhao Li +2 more

The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.

arXiv:2004.10059•The Journal of Chemical Physics•Sep 28, 2020•60 citations

Semistochastic Heat-Bath Configuration Interaction Method: Selected Configuration Interaction with Semistochastic Perturbation Theory

Sandeep Sharma, Adam A. Holmes, Guillaume Jeanmairet +2 more

We extend the recently proposed heat-bath configuration interaction (HCI) method [Holmes, Tubman, Umrigar, J. Chem. Theory Comput. 2016, 12, 3674], by introducing a semistochastic algorithm for performing multireference Epstein–Nesbet perturbation theory, in order to completely eliminate the severe memory bottleneck of the original method. The proposed algorithm has several attractive features. First, there is no sign problem that plagues several quantum Monte Carlo methods. Second, instead of using Metropolis–Hastings sampling, we use the Alias method to directly sample determinants from the reference wave function, thus avoiding correlations between consecutive samples. Third, in addition to removing the memory bottleneck, semistochastic HCI (SHCI) is faster than the deterministic variant for many systems if a stochastic error of 0.1 mHa is acceptable. Fourth, within the SHCI algorithm one can trade memory for a modest increase in computer time. Fifth, the perturbative calculation is embarrassingly parallel. The SHCI algorithm extends the range of applicability of the original algorithm, allowing us to calculate the correlation energy of very large active spaces. We demonstrate this by performing calculations on several first row dimers including F 2 with an active space of (14e, 108o), Mn-Salen cluster with an active space of (28e, 22o), and Cr 2 dimer with up to a quadruple-ζ basis set with an active space of (12e, 190o). For these systems we were able to obtain better than 1 mHa accuracy with a wall time of merely 55 s, 37 s, and 56 min on 1, 1, and 4 nodes, respectively.

arXiv:1610.06660•Journal of Chemical Theory and Computation•Mar 6, 2017•380 citations

Development of Efficient Algorithms for Quantum Chemistry Calculations of Large Molecules

和也 石村, カズヤ イシムラ, Kazuya Ishimura

Quantum chemistry plays an important role in elucidating molecular geometries, electronic states, and reaction mechanisms, because of the developments of a variety of theoretical methods, such as Hartree-Fock (HF), Møler-Plesset (MP) perturbation, configuration interaction (CI), coupled-cluster (CC), and density functional theory (DFT) methods. Electronic structure calculations have been carried out by not only theoretical chemists but also experimental chemists. DFT is currently most widely used to investigate large molecules in the ground state as well as small molecules because of the low computational cost. However, the generally used functionals fail to describe correctly non-covalent interactions that are important for host-guest molecules, self-assembly, and molecular recognition, and they tend to underestimate reaction barriers. Many attempts have been made to develop new functionals and add semiempirical or empirical correction terms to standard functionals, but no generally accepted DFT method has emerged yet. Second-order Møler-Plesset perturbation theory (MP2) is the simplest method that includes electron correlation important for non-covalent interactions and reaction barriers nonempirically. However, the computational cost of MP2 is considerably higher than that of DFT. In addition, much larger sizes of fast memory and hard disk are required in MP2 calculations. These make MP2 calculations increasingly difficult for larger molecules. Since workstation or personal computer (PC) clusters have become popular for quantum chemistry calculations, an efficient parallel calculation is a solution of the problem. Therefore, new parallel algorithms for MP2 energy and gradient calculations are presented in this thesis. Furthermore, an efficient algorithm for the generation of two-electron repulsion integrals (ERIs) which is important in quantum chemistry calculations is also presented. For the calculations of excited states, different approaches are required: for example, CI, multi-configuration self-consistent field (MCSCF), time-dependent DFT (TDDFT), and symmetry adapted cluster (SAC)/SAC-CI methods. One of the most accurate methods is SAC/SAC-CI, as demonstrated for many molecules. In this thesis, SAC/SAC-CI calculations of ground, ionized, and excited states are presented. This thesis consists of five chapters: a new algorithm of two-electron repulsion integral calculations (Chapter I), a new parallel algorithm of MP2 energy calculations (Chapter II), a new parallel algorithm of MP2 energy gradient calculations (Chapter III), applications of MP2 calculations (Chapter IV), and SAC/SAC-CI calculations of ionized and excited states (Chapter V). In quantum chemistry calculations, the generation of ERIs is one of the most basic subjects and is the most time-consuming step especially in direct SCF calculations. Many algorithms have been developed to reduce the computational cost. In Pople-Hehre algorithm, Cartesian axes are rotated to make several coordinate components zero or constant, so that these components are skipped in the generation of ERIs. In McMurchie-Davidson algorithm, ERIs are generated from (ss|ss) type integrals using a recurrence relation derived from Hermite polynomials. By combining these two algorithms, a new algorithm is developed in Chapter I. The results show that the new algorithm reduces the computational cost by 10 - 40%, as compared with the original algorithms. It is notable that the generation of ERIs including d functions is considerably fast. The program implemented officially in GAMESS in 2004 has been used all over the world. In quantum mechanics, perturbation methods can be used for adding corrections to reference solutions. In the MP perturbation method, a sum over Fock operators is used as the reference term, and the exact two-electron repulsion operator minus twice the average two-electron repulsion operator is used as the perturbation term. It is the advantage that the MP perturbation method is size consistent and size extensive, unlike truncated CI methods. The zero-order wave function is the HF Slater determinant, and the zero-order energy is expressed as a sum of occupied molecular orbital (MO) energies. The first-order perturbation is the correction for the overcounting of two-electron repulsions at zero-order, and the first-order energy corresponds to the HF energy. The MP correlation starts at second-order. In general, second-order (MP2) accounts for 80 - 90% of electron correlation. Therefore, MP2 is focused in this thesis since it is applicable to large molecules with considerable reliability and low computational cost. The formal computational scaling of MP2 energy calculations with respect to molecular size is fifth order, much higher than that of DFT energy calculations. Therefore, less expensive methods, such as Local MP2, density fitting (resolution of identity, RI) MP2, and Laplace Transform MP2, have been developed. However, all of these methods include approximations or cut-offs that need to be checked against full MP2 energies. An alternative approach to reduce the computational cost is to parallelize MP2 energy calculations. A number of papers on parallel MP2 energy calculations have been published. Almost all of them are based on simple parallelization methods that distribute only atomic orbital (AO) or MO indices to each processor. These methods have a disadvantage since intermediate integrals are broadcasted to all CPUs or the same AO ERIs are generated in all processors. Baker and Pulay developed a new parallel algorithm using SaebøAlmlöf integral transformation method. This algorithm parallelizes the first half transformation by AO indices and the second half transformation by MO indices. The advantages are that the total amount of network communication is independent of the number of processors and the AO integrals are generated only once. The disadvantage is the I/O overhead for the sorting of half-transformed integrals. A new parallel algorithm for MP2 energy calculations based on the two-step parallelization idea is presented in Chapter II. In this algorithm, AO indices are distributed in the AO integral generation and the first three quarter transformation, and MO indices are distributed in the last quarter transformation and MP2 energy calculation. Because the algorithm makes the sorting of intermediate integrals very simple, the parallel efficiency is highly improved and the I/O overhead is removed. Furthermore, the algorithm reduces highly the floating point operation (FLOP) count as well as the required memory and hard disk space, in comparison with other algorithms. Test calculations of taxol (C47H51NO14) and luciferin (C11H8N2O3S2) were performed on a cluster of Pentium 4 computers connected by gigabit Ethernet. The parallel scaling of the developed code is excellent up to the largest number of processors we have tested. For instance, the elapsed time for the MP2 energy calculations on 16 processors is on average 15.4 times faster than that on the single-processor. Determination of molecular geometries and reaction paths is a fundamental task in quantum chemistry and requires energy gradients with respect to nuclear coordinates. In Chapter III, a new parallel algorithm for MP2 energy gradient calculations is presented. The algorithm consists of 5 steps, the integral transformation, the MP2 amplitude calculation, the MP2 Lagrangian calculation, the coupled-perturbed HF calculation, and the integral derivative calculation. All steps are parallelized by distributing AO or MO indices. The algorithm also reduces the FLOP count, the required memory, and hard disk space. Test calculations of MP2 energy gradients were performed for taxol and luciferin on a cluster of Pentium 4 computers. The speedups are very good up to 80 CPU cores we have tested. For instance, the speedup ratios are 28.2 - 33.0 on 32 processors, corresponding to 88% - 103% of linear speedup. This indicates the high parallel efficiency of the present algorithm. The calculation of taxol with 6-31G(d) (1032 contracted basis functions) finishes within 2 hours on 32 processors, which requires only 1.8GB memory and 13.4GB hard disk per processor. Therefore, geometry optimization of molecules with 1000 basis functions can be easily performed using standard PC clusters. In Chapter IV, several applications of MP2 are performed using the program developed in Chapters II and III. Some molecules that DFT cannot treat well are optimized at the MP2 level. Geometry optimization is also carried out using the spin-component scaled (SCS) MP2 method. In this method, a different scaling is employed for the same and opposite spin components of the MP2 energy, so that SCS-MP2 performs as well as the much more costly CCSD(T) method at a high level of theory. SAC theory is developed for ground states and based on CC theory that describes higher-order electron correlation. The main factor of electron correlation is collisions of two electrons. In CC theory, most collisions of four electrons can be taken in as the product of collisions of tw

•Sep 28, 2007•0 citations

One programming model to rule them all: High-performance computational chemistry and applications via OpenMP offloading mode

Melisa Alkan

This dissertation is focused around enabling large-scale quantum chemistry calculations by developing novel algorithms on new heterogeneous hardware architectures. Section 2 contains a study on assessing the importance of many-body dispersion phenomena in molecular clusters and understanding the need to incorporate the phenomena into existing quantum chemistry methods to accurately capture long-range interactions. It is known that highly accurate quantum chemistry methods are computationally expensive; therefore, there is a need for new computer algorithms that will be able to vastly improve the time to solution and enable correlated quantum chemistry calculations on large molecular systems. Sections 3 and 4 focus on the development and implementation of various algorithms that enable quantum chemistry calculations on molecules of unprecedent sizes. Specifically, Section 3 describes a novel CUDA-based algorithm that enables GPU-acceleration of the main parts of the Self-Consistent Field (SCF) cycle, resulting in the fastest SCF code to date. Section 4 introduces a novel OpenMP-based implementation of the Fock build formation, which enables GPU-accelerated calculations on all existing GPU and CPU hardware. Section 5 expands upon the OpenMP work and describes a new GPU-accelerated library named LibERI for evaluating molecular integrals – often the main computational bottleneck in many quantum chemistry methods. LibERI is intended to serve as a library for fast quantum chemistry calculations that can be interfaced with any existing quantum chemistry software packages through the MolSSI Driver Interface (MDI). MDI is a standard interface between scientific computational codes. One of LibERI’s objectives is to seamlessly integrate with a Molecular Mechanics (MM) package using MDI to enable fast QM/MM calculations. This integration would allow for efficient GPU-accelerated treatment of large QM regions via LibERI. Lastly, Section 6 concludes this dissertation with a study that applies existing quantum chemistry methods to unravel the mechanism of a Pd-catalyzed carboacylation reaction involving the activation of ester C-O bonds.

•Jan 1, 2023•0 citations

Accelerating Dynamic Correlation Algorithms for Quantum Chemistry

Mikael Kovtun

Relativistic quantum mechanics provides a fundamental description of virtually all chemical phenomena, though in practice quantum chemistry is formulated through approximate solutions of the Dirac equation.Among systematic approaches, configuration interaction (CI) yields the exact solution in the complete-basis-set limit but suffers from factorial growth in computer memory requirements, restricting its application to the smallest systems. The recently developed small tensor product distributed active space (STP-DAS) CI framework significantly shrinks these resource requirements, yet large correlated calculations remain out of reach. In contrast, density functional theory (DFT) offers an approximate solution with much more favorable scaling. While not systematically improvable, DFT is widely used for large molecular systems where CI is not tractable. The landscape of scientific computing today is moving away from traditional processors in favor of graphics processing units (GPUs). GPUs offer the potential of tremendous speedup, but their unique computing paradigm requires careful algorithmic design to maximize their effectiveness. In this dissertation, we present two GPU-accelerated algorithms that accelerate critical rate-limiting steps in DFT and STP-DAS CI calculations, enabling faster correlated methods across length scales. We describe the design considerations of these algorithms, contextualize their role in chemical applications, and demonstrate their performance on large systems.

•Jan 1, 2026•0 citations
In this thesis, I present my work on the fast semistochastic heatbath configuration interaction (Fast SHCI), which is an efficient algorithm for doing essentially exact electronic structure calculations within a finite basis. There are Hamiltonians for which the entire Hilbert space is enormous, but the important part of Hilbert space is of manageable size, say 101210^{12}1012. Quantum chemistry Hamiltonians, for reasonably small systems, have this property. For such Hamiltonians, selected configuration interaction plus perturbation theory (SCI+PT) methods can be useful. The most important part of the Hilbert space is treated variationally, and the resulting energy is improved by using perturbation theory. Fast SHCI is more than an order of magnitude faster than other SCI+PT algorithms, and also much faster than other essentially exact algorithms for many chemical systems. This thesis provides an in-depth description of the Fast SHCI algorithm and its implementation. I use SHCI to compute the electronic structure of several chemical systems and the homogeneous electron gas. Some of these calculations are more accurate than those achieved by other high-order quantum chemistry methods. Others treat systems larger than those that can be treated by other equally accurate methods. My implementation of SHCI uses a modular design, which not only makes the library highly extensible but also contributes several generic distributed computing building blocks to the open-source community. In this thesis, I also describe my design and implementation of these generic components. Finally, I also provide a brief discussion of the usability of general software engineering best practices for the development of medium-scale scientific software packages with lessons learned from designing, developing, and leading the development of our SHCI package. Medium-scale scientific software packages are common in scientific research where a small group of researchers works on the same code base. Due to the differences in the requirements, some best practices that are common in the industry need to be adjusted to be useful for these projects.
This paper presents in detail our fast semistochastic heat-bath configuration interaction (SHCI) method for solving the many-body Schrödinger equation. We identify and eliminate computational bottlenecks in both the variational and perturbative steps of the SHCI algorithm. We also describe the parallelization and the key data structures in our implementation, such as the distributed hash table. The improved SHCI algorithm enables us to include in our variational wavefunction two orders of magnitude more determinants than has been reported previously with other selected configuration interaction methods. We use our algorithm to calculate an accurate benchmark energy for the chromium dimer with the X2C relativistic Hamiltonian in the cc-pVDZ-DK basis, correlating 28 electrons in 76 spatial orbitals. Our largest calculation uses two billion Slater determinants in the variational space and semistochastically includes perturbative contributions from at least trillions of additional determinants with better than 10−5 Ha statistical uncertainty.
The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.
We extend the recently proposed heat-bath configuration interaction (HCI) method [Holmes, Tubman, Umrigar, J. Chem. Theory Comput. 2016, 12, 3674], by introducing a semistochastic algorithm for performing multireference Epstein–Nesbet perturbation theory, in order to completely eliminate the severe memory bottleneck of the original method. The proposed algorithm has several attractive features. First, there is no sign problem that plagues several quantum Monte Carlo methods. Second, instead of using Metropolis–Hastings sampling, we use the Alias method to directly sample determinants from the reference wave function, thus avoiding correlations between consecutive samples. Third, in addition to removing the memory bottleneck, semistochastic HCI (SHCI) is faster than the deterministic variant for many systems if a stochastic error of 0.1 mHa is acceptable. Fourth, within the SHCI algorithm one can trade memory for a modest increase in computer time. Fifth, the perturbative calculation is embarrassingly parallel. The SHCI algorithm extends the range of applicability of the original algorithm, allowing us to calculate the correlation energy of very large active spaces. We demonstrate this by performing calculations on several first row dimers including F 2 with an active space of (14e, 108o), Mn-Salen cluster with an active space of (28e, 22o), and Cr 2 dimer with up to a quadruple-ζ basis set with an active space of (12e, 190o). For these systems we were able to obtain better than 1 mHa accuracy with a wall time of merely 55 s, 37 s, and 56 min on 1, 1, and 4 nodes, respectively.
Quantum chemistry plays an important role in elucidating molecular geometries, electronic states, and reaction mechanisms, because of the developments of a variety of theoretical methods, such as Hartree-Fock (HF), Møler-Plesset (MP) perturbation, configuration interaction (CI), coupled-cluster (CC), and density functional theory (DFT) methods. Electronic structure calculations have been carried out by not only theoretical chemists but also experimental chemists. DFT is currently most widely used to investigate large molecules in the ground state as well as small molecules because of the low computational cost. However, the generally used functionals fail to describe correctly non-covalent interactions that are important for host-guest molecules, self-assembly, and molecular recognition, and they tend to underestimate reaction barriers. Many attempts have been made to develop new functionals and add semiempirical or empirical correction terms to standard functionals, but no generally accepted DFT method has emerged yet. Second-order Møler-Plesset perturbation theory (MP2) is the simplest method that includes electron correlation important for non-covalent interactions and reaction barriers nonempirically. However, the computational cost of MP2 is considerably higher than that of DFT. In addition, much larger sizes of fast memory and hard disk are required in MP2 calculations. These make MP2 calculations increasingly difficult for larger molecules. Since workstation or personal computer (PC) clusters have become popular for quantum chemistry calculations, an efficient parallel calculation is a solution of the problem. Therefore, new parallel algorithms for MP2 energy and gradient calculations are presented in this thesis. Furthermore, an efficient algorithm for the generation of two-electron repulsion integrals (ERIs) which is important in quantum chemistry calculations is also presented. For the calculations of excited states, different approaches are required: for example, CI, multi-configuration self-consistent field (MCSCF), time-dependent DFT (TDDFT), and symmetry adapted cluster (SAC)/SAC-CI methods. One of the most accurate methods is SAC/SAC-CI, as demonstrated for many molecules. In this thesis, SAC/SAC-CI calculations of ground, ionized, and excited states are presented. This thesis consists of five chapters: a new algorithm of two-electron repulsion integral calculations (Chapter I), a new parallel algorithm of MP2 energy calculations (Chapter II), a new parallel algorithm of MP2 energy gradient calculations (Chapter III), applications of MP2 calculations (Chapter IV), and SAC/SAC-CI calculations of ionized and excited states (Chapter V). In quantum chemistry calculations, the generation of ERIs is one of the most basic subjects and is the most time-consuming step especially in direct SCF calculations. Many algorithms have been developed to reduce the computational cost. In Pople-Hehre algorithm, Cartesian axes are rotated to make several coordinate components zero or constant, so that these components are skipped in the generation of ERIs. In McMurchie-Davidson algorithm, ERIs are generated from (ss|ss) type integrals using a recurrence relation derived from Hermite polynomials. By combining these two algorithms, a new algorithm is developed in Chapter I. The results show that the new algorithm reduces the computational cost by 10 - 40%, as compared with the original algorithms. It is notable that the generation of ERIs including d functions is considerably fast. The program implemented officially in GAMESS in 2004 has been used all over the world. In quantum mechanics, perturbation methods can be used for adding corrections to reference solutions. In the MP perturbation method, a sum over Fock operators is used as the reference term, and the exact two-electron repulsion operator minus twice the average two-electron repulsion operator is used as the perturbation term. It is the advantage that the MP perturbation method is size consistent and size extensive, unlike truncated CI methods. The zero-order wave function is the HF Slater determinant, and the zero-order energy is expressed as a sum of occupied molecular orbital (MO) energies. The first-order perturbation is the correction for the overcounting of two-electron repulsions at zero-order, and the first-order energy corresponds to the HF energy. The MP correlation starts at second-order. In general, second-order (MP2) accounts for 80 - 90% of electron correlation. Therefore, MP2 is focused in this thesis since it is applicable to large molecules with considerable reliability and low computational cost. The formal computational scaling of MP2 energy calculations with respect to molecular size is fifth order, much higher than that of DFT energy calculations. Therefore, less expensive methods, such as Local MP2, density fitting (resolution of identity, RI) MP2, and Laplace Transform MP2, have been developed. However, all of these methods include approximations or cut-offs that need to be checked against full MP2 energies. An alternative approach to reduce the computational cost is to parallelize MP2 energy calculations. A number of papers on parallel MP2 energy calculations have been published. Almost all of them are based on simple parallelization methods that distribute only atomic orbital (AO) or MO indices to each processor. These methods have a disadvantage since intermediate integrals are broadcasted to all CPUs or the same AO ERIs are generated in all processors. Baker and Pulay developed a new parallel algorithm using SaebøAlmlöf integral transformation method. This algorithm parallelizes the first half transformation by AO indices and the second half transformation by MO indices. The advantages are that the total amount of network communication is independent of the number of processors and the AO integrals are generated only once. The disadvantage is the I/O overhead for the sorting of half-transformed integrals. A new parallel algorithm for MP2 energy calculations based on the two-step parallelization idea is presented in Chapter II. In this algorithm, AO indices are distributed in the AO integral generation and the first three quarter transformation, and MO indices are distributed in the last quarter transformation and MP2 energy calculation. Because the algorithm makes the sorting of intermediate integrals very simple, the parallel efficiency is highly improved and the I/O overhead is removed. Furthermore, the algorithm reduces highly the floating point operation (FLOP) count as well as the required memory and hard disk space, in comparison with other algorithms. Test calculations of taxol (C47H51NO14) and luciferin (C11H8N2O3S2) were performed on a cluster of Pentium 4 computers connected by gigabit Ethernet. The parallel scaling of the developed code is excellent up to the largest number of processors we have tested. For instance, the elapsed time for the MP2 energy calculations on 16 processors is on average 15.4 times faster than that on the single-processor. Determination of molecular geometries and reaction paths is a fundamental task in quantum chemistry and requires energy gradients with respect to nuclear coordinates. In Chapter III, a new parallel algorithm for MP2 energy gradient calculations is presented. The algorithm consists of 5 steps, the integral transformation, the MP2 amplitude calculation, the MP2 Lagrangian calculation, the coupled-perturbed HF calculation, and the integral derivative calculation. All steps are parallelized by distributing AO or MO indices. The algorithm also reduces the FLOP count, the required memory, and hard disk space. Test calculations of MP2 energy gradients were performed for taxol and luciferin on a cluster of Pentium 4 computers. The speedups are very good up to 80 CPU cores we have tested. For instance, the speedup ratios are 28.2 - 33.0 on 32 processors, corresponding to 88% - 103% of linear speedup. This indicates the high parallel efficiency of the present algorithm. The calculation of taxol with 6-31G(d) (1032 contracted basis functions) finishes within 2 hours on 32 processors, which requires only 1.8GB memory and 13.4GB hard disk per processor. Therefore, geometry optimization of molecules with 1000 basis functions can be easily performed using standard PC clusters. In Chapter IV, several applications of MP2 are performed using the program developed in Chapters II and III. Some molecules that DFT cannot treat well are optimized at the MP2 level. Geometry optimization is also carried out using the spin-component scaled (SCS) MP2 method. In this method, a different scaling is employed for the same and opposite spin components of the MP2 energy, so that SCS-MP2 performs as well as the much more costly CCSD(T) method at a high level of theory. SAC theory is developed for ground states and based on CC theory that describes higher-order electron correlation. The main factor of electron correlation is collisions of two electrons. In CC theory, most collisions of four electrons can be taken in as the product of collisions of tw
This dissertation is focused around enabling large-scale quantum chemistry calculations by developing novel algorithms on new heterogeneous hardware architectures. Section 2 contains a study on assessing the importance of many-body dispersion phenomena in molecular clusters and understanding the need to incorporate the phenomena into existing quantum chemistry methods to accurately capture long-range interactions. It is known that highly accurate quantum chemistry methods are computationally expensive; therefore, there is a need for new computer algorithms that will be able to vastly improve the time to solution and enable correlated quantum chemistry calculations on large molecular systems. Sections 3 and 4 focus on the development and implementation of various algorithms that enable quantum chemistry calculations on molecules of unprecedent sizes. Specifically, Section 3 describes a novel CUDA-based algorithm that enables GPU-acceleration of the main parts of the Self-Consistent Field (SCF) cycle, resulting in the fastest SCF code to date. Section 4 introduces a novel OpenMP-based implementation of the Fock build formation, which enables GPU-accelerated calculations on all existing GPU and CPU hardware. Section 5 expands upon the OpenMP work and describes a new GPU-accelerated library named LibERI for evaluating molecular integrals – often the main computational bottleneck in many quantum chemistry methods. LibERI is intended to serve as a library for fast quantum chemistry calculations that can be interfaced with any existing quantum chemistry software packages through the MolSSI Driver Interface (MDI). MDI is a standard interface between scientific computational codes. One of LibERI’s objectives is to seamlessly integrate with a Molecular Mechanics (MM) package using MDI to enable fast QM/MM calculations. This integration would allow for efficient GPU-accelerated treatment of large QM regions via LibERI. Lastly, Section 6 concludes this dissertation with a study that applies existing quantum chemistry methods to unravel the mechanism of a Pd-catalyzed carboacylation reaction involving the activation of ester C-O bonds.
Relativistic quantum mechanics provides a fundamental description of virtually all chemical phenomena, though in practice quantum chemistry is formulated through approximate solutions of the Dirac equation.Among systematic approaches, configuration interaction (CI) yields the exact solution in the complete-basis-set limit but suffers from factorial growth in computer memory requirements, restricting its application to the smallest systems. The recently developed small tensor product distributed active space (STP-DAS) CI framework significantly shrinks these resource requirements, yet large correlated calculations remain out of reach. In contrast, density functional theory (DFT) offers an approximate solution with much more favorable scaling. While not systematically improvable, DFT is widely used for large molecular systems where CI is not tractable. The landscape of scientific computing today is moving away from traditional processors in favor of graphics processing units (GPUs). GPUs offer the potential of tremendous speedup, but their unique computing paradigm requires careful algorithmic design to maximize their effectiveness. In this dissertation, we present two GPU-accelerated algorithms that accelerate critical rate-limiting steps in DFT and STP-DAS CI calculations, enabling faster correlated methods across length scales. We describe the design considerations of these algorithms, contextualize their role in chemical applications, and demonstrate their performance on large systems.