Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2502.05383 · Physical review. B./Physical review. B

Self-attention neural network for solving correlated electron problems in solids

Max Geier, Khachatur Nazaryan, Timothy Zaklama, Liang Fu

Published Jun 16, 2025• 36 citations
arXivHTMLPDF

Abstract

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as $N^2$ with the number of electrons, which opens a path towards efficient large-scale simulations.

DOI
10.1103/qxc3-bkc7

Related papers

Transferable neural wavefunctions for solids

Leon Gerard, Michael Scherbela, Halvard Sutterud +2 more

Abstract Deep-learning-based variational Monte Carlo has emerged as a highly accurate method for solving the many-electron Schrödinger equation. Despite favorable scaling with the number of electrons, $${\mathcal{O}}({{n}_{{\rm{el}}}}^{4})$$ O ( n el 4 ) , the practical value of deep-learning-based variational Monte Carlo is limited by the high cost of optimizing the neural network weights for every system studied. Recent research has proposed optimizing a single neural network across multiple systems, reducing the cost per system. Here we extend this approach to solids, which require numerous calculations across different geometries, boundary conditions and supercell sizes. We demonstrate that optimization of a single ansatz across these variations significantly reduces optimization steps. Furthermore, we successfully transfer a network trained on 2 × 2 × 2 supercells of LiH, to 3 × 3 × 3 supercells, reducing the number of optimization steps required to simulate the large system by a factor of 50 compared with previous work.

arXiv:2405.07599•Nature Computational Science•Oct 22, 2025•14 citations

A Self-Attention Ansatz for Ab-initio Quantum Chemistry

Ingrid von Glehn, James S. Spencer, David Pfau

We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.

arXiv:2211.13672•Nov 24, 2022•134 citations

Simulating moiré quantum matter with neural network

Di Luo, David D. Dai, Liang Fu

Moiré materials provide an ideal platform for exploring quantum phases of matter. However, solving the many-electron problem in moiré systems is challenging due to strong correlation effects. We introduce a powerful variational representation of quantum states, many-body neural Bloch wavefunction, to solve many-electron problems in moiré materials accurately and efficiently. Applying our method to the semiconductor heterobilayer WSe2/WS2 , we obtain a generalized Wigner crystal at filling factor n = 1/3, a Mott insulator n = 1, and a correlated insulator with local magnetic moments and antiferromagnetic spin correlation at n = 2. Our neural network approach improves the simulation accuracy of strongly interacting moiré materials and paves the way for discovery of new quantum phases with variational learning principle in a unified framework.

arXiv:2406.17645•Jun 25, 2024•30 citations

Smart Machine Learning for Solving the Full Electron-Nuclear Schrödinger Equation with a Quantum-Chemically Inspired Wavefunction Ansatz

Yaolong Zhang, Bin Jiang, Hua Guo

Directly solving the full electron-nuclear Schrödinger equation remains one of the grand challenges in quantum mechanics. Here, we propose a quantum-chemically motivated orbital-based neural network wavefunction ansatz with explicit many-body electron-electron correlation terms, with parameters depending on the entire electron-nuclear configuration, enabling an accurate description of strongly correlated and/or multi-reference electronic structures by a single Slater-determinant across a broad nuclear configuration space. To train this wavefunction, we further develop an efficient hybrid optimization strategy that combines the variational Monte Carlo technique with local-energy constraints, mitigating the intrinsic sampling bias of the former and considerably reducing the statistical error. These advances are embedded in a unified neural network framework, Schrödinger, enabling the determination of globally accurate potential energy surfaces with a single training for systems ranging from strongly correlated diatomic molecules, triatomic reactions containing conical intersections, to polyatomic multi-reference molecules. This work offers a practical machine-learning tool for solving the electron-nuclear Schrödinger equation and capturing full quantum effects beyond the Born-Oppenheimer approximation.

physics.chem-phCross-listedarXiv:2609.40112•Oct 1, 2026•0 citations

Variational Quantum Monte Carlo Solution of the Many-Electron Schrödinger Equation Based on Deep Neural Networks

Huiping Su, Hongbo Gao, Xinmiao Wang +2 more

The solution of Schrödinger equation generates the quantisation properties of the system, which can completely describe the quantum behaviour of microscopic particles in a physical system. However, solving the Schrödinger equation is a non-deterministic polynomial time (NP)-hard problem. Numerical methods for solving the Schrödinger equation require careful selection of basis configurations, and the complete basis set has high accuracy but its complexity increases exponentially in the number of particles. Therefore, it is crucial to find an effective numerical method. To solve this problem, this paper present a deep learning architecture, VMCNet, using the powerful computational efficiency of neural networks to improve the speed of numerical computation. Moreover, this paper proposes a suitable wavefunction ansatz, witch determines the accuracy of neural networks, that achieves more accurate energy solutions of the many-electron Schrödinger equation compared to the conventional single-determinant variational molecular Monte Carlo. We introduce non-local and local blocks to represent quantum mechanical interactions, which contain more physical information about the particle system. The experimental results show that VMCNet can cover more than 93% of the correlation energy of the main group elements (Be–Ne) for the monoatomic system and more than 90% of the correlation energy for the diatomic system.

CAAI Artificial Intelligence Research•Feb 1, 2024•1 citation

Neural network based nodal structure optimization for interacting fermionic systems

William Freitas, Bruno Ricardi de Abreu, S. A. Vitiello

Simulating strongly correlated fermionic systems remains a fundamental challenge in quantum physics, largely due to the sign problem in quantum Monte Carlo (QMC) methods. We present a neural network-based variational Monte Carlo (NN-VMC) approach, leveraging a flexible neural network ansatz to represent the many-body wavefunction. Focusing on quantum dots with up to 30 electrons, we demonstrate that NN-VMC significantly reduces variational bias and achieves ground-state energies surpassing those of fixed-node diffusion Monte Carlo (DMC). A key feature is that the neural network adaptively learns and optimizes nodal structures during energy minimization. We provide qualitative insights into the nodal structure of fermionic wavefunctions by comparing the nodal structures generated by NN-VMC with those obtained from traditional trial functions. Additionally, we reveal spin-resolved radial distributions and electron density profiles, highlighting the versatility and accuracy of NN-VMC. This work underscores the potential of machine learning to advance quantum simulations and deepen our understanding of strongly correlated systems.

arXiv:2411.02257•Physical review. B./Physical review. B•Sep 9, 2025•2 citations
The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as N2N^2N2 with the number of electrons, which opens a path towards efficient large-scale simulations.
Abstract Deep-learning-based variational Monte Carlo has emerged as a highly accurate method for solving the many-electron Schrödinger equation. Despite favorable scaling with the number of electrons, O(nel4){\mathcal{O}}({{n}_{{\rm{el}}}}^{4})O(nel​4) O ( n el 4 ) , the practical value of deep-learning-based variational Monte Carlo is limited by the high cost of optimizing the neural network weights for every system studied. Recent research has proposed optimizing a single neural network across multiple systems, reducing the cost per system. Here we extend this approach to solids, which require numerous calculations across different geometries, boundary conditions and supercell sizes. We demonstrate that optimization of a single ansatz across these variations significantly reduces optimization steps. Furthermore, we successfully transfer a network trained on 2 × 2 × 2 supercells of LiH, to 3 × 3 × 3 supercells, reducing the number of optimization steps required to simulate the large system by a factor of 50 compared with previous work.
We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.
Moiré materials provide an ideal platform for exploring quantum phases of matter. However, solving the many-electron problem in moiré systems is challenging due to strong correlation effects. We introduce a powerful variational representation of quantum states, many-body neural Bloch wavefunction, to solve many-electron problems in moiré materials accurately and efficiently. Applying our method to the semiconductor heterobilayer WSe2/WS2 , we obtain a generalized Wigner crystal at filling factor n = 1/3, a Mott insulator n = 1, and a correlated insulator with local magnetic moments and antiferromagnetic spin correlation at n = 2. Our neural network approach improves the simulation accuracy of strongly interacting moiré materials and paves the way for discovery of new quantum phases with variational learning principle in a unified framework.
Directly solving the full electron-nuclear Schrödinger equation remains one of the grand challenges in quantum mechanics. Here, we propose a quantum-chemically motivated orbital-based neural network wavefunction ansatz with explicit many-body electron-electron correlation terms, with parameters depending on the entire electron-nuclear configuration, enabling an accurate description of strongly correlated and/or multi-reference electronic structures by a single Slater-determinant across a broad nuclear configuration space. To train this wavefunction, we further develop an efficient hybrid optimization strategy that combines the variational Monte Carlo technique with local-energy constraints, mitigating the intrinsic sampling bias of the former and considerably reducing the statistical error. These advances are embedded in a unified neural network framework, Schrödinger, enabling the determination of globally accurate potential energy surfaces with a single training for systems ranging from strongly correlated diatomic molecules, triatomic reactions containing conical intersections, to polyatomic multi-reference molecules. This work offers a practical machine-learning tool for solving the electron-nuclear Schrödinger equation and capturing full quantum effects beyond the Born-Oppenheimer approximation.
The solution of Schrödinger equation generates the quantisation properties of the system, which can completely describe the quantum behaviour of microscopic particles in a physical system. However, solving the Schrödinger equation is a non-deterministic polynomial time (NP)-hard problem. Numerical methods for solving the Schrödinger equation require careful selection of basis configurations, and the complete basis set has high accuracy but its complexity increases exponentially in the number of particles. Therefore, it is crucial to find an effective numerical method. To solve this problem, this paper present a deep learning architecture, VMCNet, using the powerful computational efficiency of neural networks to improve the speed of numerical computation. Moreover, this paper proposes a suitable wavefunction ansatz, witch determines the accuracy of neural networks, that achieves more accurate energy solutions of the many-electron Schrödinger equation compared to the conventional single-determinant variational molecular Monte Carlo. We introduce non-local and local blocks to represent quantum mechanical interactions, which contain more physical information about the particle system. The experimental results show that VMCNet can cover more than 93% of the correlation energy of the main group elements (Be–Ne) for the monoatomic system and more than 90% of the correlation energy for the diatomic system.
Simulating strongly correlated fermionic systems remains a fundamental challenge in quantum physics, largely due to the sign problem in quantum Monte Carlo (QMC) methods. We present a neural network-based variational Monte Carlo (NN-VMC) approach, leveraging a flexible neural network ansatz to represent the many-body wavefunction. Focusing on quantum dots with up to 30 electrons, we demonstrate that NN-VMC significantly reduces variational bias and achieves ground-state energies surpassing those of fixed-node diffusion Monte Carlo (DMC). A key feature is that the neural network adaptively learns and optimizes nodal structures during energy minimization. We provide qualitative insights into the nodal structure of fermionic wavefunctions by comparing the nodal structures generated by NN-VMC with those obtained from traditional trial functions. Additionally, we reveal spin-resolved radial distributions and electron density profiles, highlighting the versatility and accuracy of NN-VMC. This work underscores the potential of machine learning to advance quantum simulations and deepen our understanding of strongly correlated systems.