Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2501.11454 · Machine Learning Science and Technology

Improving thermal state preparation of Sachdev–Ye–Kitaev model with reinforcement learning on quantum hardware

Akash Kundu

Published Jun 10, 2025• 5 citations
arXivHTMLPDF

Abstract

Abstract The Sachdev–Ye–Kitaev (SYK) model, known for its strong quantum correlations and chaotic behavior, serves as a key platform for quantum gravity studies. However, variationally preparing thermal states on near-term quantum processors for large systems ( N > 12, where N is the number of Majorana fermions) presents a significant challenge due to the rapid growth in the complexity of parameterized quantum circuits. This paper addresses this challenge by integrating reinforcement learning (RL) with convolutional neural networks, employing an iterative approach to optimize the quantum circuit and its parameters. The refinement process is guided by a composite reward signal derived from entropy and the expectation values of the SYK Hamiltonian. This approach reduces the number of controlled-NOT gates by two orders of magnitude for systems N ⩾ 12 compared to traditional methods like first-order Trotterization. We demonstrate the effectiveness of the RL framework in both noiseless and noisy quantum hardware environments, maintaining high accuracy in thermal state preparation. This work advances a scalable, RL-based framework with applications for quantum gravity studies and out-of-time-ordered thermal correlators computation in quantum many-body systems on near-term quantum hardware. The code is available at solving_SYK_with_RL repository.

DOI
10.1088/2632-2153/ade361

Related papers

Quantum compiling with reinforcement learning on a superconducting processor

Ziting Wang, Qiuhao Chen, Yuxuan Du +15 more

Abstract Effectively implementing quantum algorithms on noisy intermediate-scale quantum (NISQ) processors is a central task in modern quantum technology. NISQ processors feature tens to a few hundreds of noisy qubits with limited coherence times and gate operations with errors, so NISQ algorithms naturally require employing circuits of short lengths via quantum compilation. Here, we evaluate a reinforcement learning (RL)-based quantum compiler on a superconducting processor. Our experiments reveal that for two-qubit circuits, the RL-based compiler surpasses conventional methods, demonstrating its ability to discover hardware-amenable circuits with near-optimal lengths. However, for three-qubit circuits, the RL-based compiler does not achieve unity theoretical fidelity. To address this limitation, we integrate a variational strategy with the RL-based compiler, highlighting their complementary strengths. Systematic experiments show that this variational RL-based compiler consistently identifies near-optimal circuits, even under stringent hardware constraints, outperforming conventional techniques. Furthermore, we analyze the impact of decoherence and gate errors, providing critical insights into the practical performance of RL-based compilers on quantum hardware. These findings exemplify the codesign of the software with hardware for efficient quantum compilation, offering valuable insights for the advancement of RL-based compilers.

arXiv:2406.12195•Machine Learning Science and Technology•Jun 26, 2026•15 citations

Entropy-Topology Correspondence in Quantum Chaotic Systems: A Metric-Preserving Learning Approach to the SYK Model

S. M. Yousuf Iqbal Tomal, Abdullah Al Shafin

The reconciliation of unitary quantum mechanics with the smooth geometry of General Relativity remains a central challenge in quantum gravity, epitomized by the Firewall Paradox. In this work, we investigate the emergent geometry of the black hole information manifold using a metric-preserving machine learning framework. We simulate a 1D quantum black hole using the Sachdev-Ye-Kitaev (SYK) model and reconstruct its bulk geometry via an autoencoder constrained by the Quantum Bures Metric. Our analysis reveals a strong negative correlation between the thermodynamic entropy and the box-counting dimension of the emergent manifold, with finite size scaling consistent with perfect anti-correlation in the thermodynamic limit. Crucially, control experiments reveal that this correspondence is chaos-dependent: integrable free fermion systems show dramatically weaker correlations, while spatially local Heisenberg chains exhibit intermediate behavior. This establishes chaos as a key requirement for strong entropy-topology correspondence, distinguishing maximally chaotic from integrable thermal quantum states. We observe no topological phase transitions across the thermal crossover, supporting the hypothesis that high-entropy chaotic thermalization drives geometric smoothing, consistent with the ER=EPR conjecture.

•Jan 14, 2026•0 citations

Polynomial-time classical algorithms for mean-field models up to the glass transition

Alexander Schmidhuber, Alexander Zlokapa

The Sachdev-Ye-Kitaev model is a strongly interacting fermionic system that has been well-studied in condensed matter and high energy physics. It is highly quantum: Gaussian states are far from the thermal state (Hastings and O'Donnell, STOC'22) and representing the thermal state requires large polynomial-size quantum circuits (Anschuetz et al., QIP'25). Very recently, it was nonetheless proven that classical algorithms can estimate local thermal expectations at sufficiently high temperature in quasipolynomial time (Zlokapa, FOCS'26). We show that classical algorithms can in fact estimate local observables at all constant temperatures in polynomial time. Our techniques also extend straightforwardly to classical systems: we resolve an open question about computing thermal expectations of a classical spin glass up to its phase transition (Bencs et al., STOC'26). Our proof develops a fully rigorous quantum cavity method. Due to the success of the classical cavity method in optimization, sampling, inference and learning, we expect the quantum cavity method to find further applications of independent interest. As an example, we give a quantum algorithm that learns SYK Hamiltonians from the Gibbs state at any constant temperature with polynomial time and sample complexity.

quant-pharXiv:2610.06807•Oct 6, 2026•4 citations

Scalable quantum dynamics compilation via quantum machine learning

Yuxuan Zhang, Roeland Wiersema, Juan Carrasquilla +2 more

Quantum dynamics compilation is an important task for improving quantum simulation efficiency: It aims to synthesize multiqubit target dynamics into a circuit consisting of as few elementary gates as possible. Compared to deterministic methods such as Trotterization, variational quantum compilation (VQC) methods employ variational optimization to reduce gate costs while maintaining high accuracy. In this work, we make a paradigm shift in VQC’s framework design. First, we use a cost function from recently established out-of-distribution generalization results in quantum machine learning: By learning the action of a given many-body dynamics on a small dataset of product states, we can obtain a unitary circuit that generalizes to highly entangled states such as the Haar random states. Moreover, a key observation is that the low complexity in the data states allows us to use tensor-network methods to compress such time-evolved product states by exploiting their low entanglement features. Utilizing warm start methods, our approach exceeds state-of-the-art compilation results in both system size and accuracy in one dimension. For the first time, we extend VQC to systems to two-dimensional systems, demonstrating a significant resource advantage over standard Trotterization methods. The flexibility of our framework highlights the method's promise for advancing quantum simulation tasks on near-term quantum processors.

arXiv:2409.16346•Physical Review Research•Feb 4, 2026•22 citations

Deep Reinforcement Learning for quantum state preparation with noisy gates in the NISQ-era

Narvaez Hernandez, Roberto

Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.

•Sep 9, 2024•0 citations

Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning

Remmy A. M. Zen, Jan Ollé, Luis Colmenárez +3 more

The realization of large-scale quantum computers requires not only quantum error correction but also fault-tolerant (FT) operations to handle errors that propagate into harmful errors. Recently, flag-based protocols have been introduced that use ancillary qubits to flag harmful errors. However, there is no clear recipe for finding a FT quantum circuit with flag-based protocols, especially when we consider hardware constraints, such as the qubit connectivity and available gate set. This work presents a novel approach to automatically discover compact and hardware-adapted FT quantum circuits to make significant progress towards scalable FT quantum computing. We employ reinforcement learning (RL) as an enabling tool, leveraging a fast, parallelized stabilizer quantum circuit simulator and a nontrivial reward function specifically adapted to the problem. We show that, in the task of FT logical state preparation, RL discovers not only circuits with fewer gates and ancillary qubits than published results but also novel circuits without and with hardware constraints of up to distance-5 codes with 25 physical qubits, and they can be implemented directly in experiments. Furthermore, RL allows for straightforward exploration of different qubit connectivities and the use of transfer learning to accelerate the discovery. More generally, our work sets the framework towards the use of RL or other machine learning techniques for FT quantum circuit discovery with hardware constraints to make real progress towards the realization of large-scale quantum computers, addressing tasks beyond state preparation, including magic state preparation, logical gate synthesis, and syndrome measurement.

arXiv:2402.17761•Physical Review X•Sep 11, 2025•38 citations
Abstract The Sachdev–Ye–Kitaev (SYK) model, known for its strong quantum correlations and chaotic behavior, serves as a key platform for quantum gravity studies. However, variationally preparing thermal states on near-term quantum processors for large systems ( N > 12, where N is the number of Majorana fermions) presents a significant challenge due to the rapid growth in the complexity of parameterized quantum circuits. This paper addresses this challenge by integrating reinforcement learning (RL) with convolutional neural networks, employing an iterative approach to optimize the quantum circuit and its parameters. The refinement process is guided by a composite reward signal derived from entropy and the expectation values of the SYK Hamiltonian. This approach reduces the number of controlled-NOT gates by two orders of magnitude for systems N ⩾ 12 compared to traditional methods like first-order Trotterization. We demonstrate the effectiveness of the RL framework in both noiseless and noisy quantum hardware environments, maintaining high accuracy in thermal state preparation. This work advances a scalable, RL-based framework with applications for quantum gravity studies and out-of-time-ordered thermal correlators computation in quantum many-body systems on near-term quantum hardware. The code is available at solving_SYK_with_RL repository.
Abstract Effectively implementing quantum algorithms on noisy intermediate-scale quantum (NISQ) processors is a central task in modern quantum technology. NISQ processors feature tens to a few hundreds of noisy qubits with limited coherence times and gate operations with errors, so NISQ algorithms naturally require employing circuits of short lengths via quantum compilation. Here, we evaluate a reinforcement learning (RL)-based quantum compiler on a superconducting processor. Our experiments reveal that for two-qubit circuits, the RL-based compiler surpasses conventional methods, demonstrating its ability to discover hardware-amenable circuits with near-optimal lengths. However, for three-qubit circuits, the RL-based compiler does not achieve unity theoretical fidelity. To address this limitation, we integrate a variational strategy with the RL-based compiler, highlighting their complementary strengths. Systematic experiments show that this variational RL-based compiler consistently identifies near-optimal circuits, even under stringent hardware constraints, outperforming conventional techniques. Furthermore, we analyze the impact of decoherence and gate errors, providing critical insights into the practical performance of RL-based compilers on quantum hardware. These findings exemplify the codesign of the software with hardware for efficient quantum compilation, offering valuable insights for the advancement of RL-based compilers.
The reconciliation of unitary quantum mechanics with the smooth geometry of General Relativity remains a central challenge in quantum gravity, epitomized by the Firewall Paradox. In this work, we investigate the emergent geometry of the black hole information manifold using a metric-preserving machine learning framework. We simulate a 1D quantum black hole using the Sachdev-Ye-Kitaev (SYK) model and reconstruct its bulk geometry via an autoencoder constrained by the Quantum Bures Metric. Our analysis reveals a strong negative correlation between the thermodynamic entropy and the box-counting dimension of the emergent manifold, with finite size scaling consistent with perfect anti-correlation in the thermodynamic limit. Crucially, control experiments reveal that this correspondence is chaos-dependent: integrable free fermion systems show dramatically weaker correlations, while spatially local Heisenberg chains exhibit intermediate behavior. This establishes chaos as a key requirement for strong entropy-topology correspondence, distinguishing maximally chaotic from integrable thermal quantum states. We observe no topological phase transitions across the thermal crossover, supporting the hypothesis that high-entropy chaotic thermalization drives geometric smoothing, consistent with the ER=EPR conjecture.
The Sachdev-Ye-Kitaev model is a strongly interacting fermionic system that has been well-studied in condensed matter and high energy physics. It is highly quantum: Gaussian states are far from the thermal state (Hastings and O'Donnell, STOC'22) and representing the thermal state requires large polynomial-size quantum circuits (Anschuetz et al., QIP'25). Very recently, it was nonetheless proven that classical algorithms can estimate local thermal expectations at sufficiently high temperature in quasipolynomial time (Zlokapa, FOCS'26). We show that classical algorithms can in fact estimate local observables at all constant temperatures in polynomial time. Our techniques also extend straightforwardly to classical systems: we resolve an open question about computing thermal expectations of a classical spin glass up to its phase transition (Bencs et al., STOC'26). Our proof develops a fully rigorous quantum cavity method. Due to the success of the classical cavity method in optimization, sampling, inference and learning, we expect the quantum cavity method to find further applications of independent interest. As an example, we give a quantum algorithm that learns SYK Hamiltonians from the Gibbs state at any constant temperature with polynomial time and sample complexity.
Quantum dynamics compilation is an important task for improving quantum simulation efficiency: It aims to synthesize multiqubit target dynamics into a circuit consisting of as few elementary gates as possible. Compared to deterministic methods such as Trotterization, variational quantum compilation (VQC) methods employ variational optimization to reduce gate costs while maintaining high accuracy. In this work, we make a paradigm shift in VQC’s framework design. First, we use a cost function from recently established out-of-distribution generalization results in quantum machine learning: By learning the action of a given many-body dynamics on a small dataset of product states, we can obtain a unitary circuit that generalizes to highly entangled states such as the Haar random states. Moreover, a key observation is that the low complexity in the data states allows us to use tensor-network methods to compress such time-evolved product states by exploiting their low entanglement features. Utilizing warm start methods, our approach exceeds state-of-the-art compilation results in both system size and accuracy in one dimension. For the first time, we extend VQC to systems to two-dimensional systems, demonstrating a significant resource advantage over standard Trotterization methods. The flexibility of our framework highlights the method's promise for advancing quantum simulation tasks on near-term quantum processors.
Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.
The realization of large-scale quantum computers requires not only quantum error correction but also fault-tolerant (FT) operations to handle errors that propagate into harmful errors. Recently, flag-based protocols have been introduced that use ancillary qubits to flag harmful errors. However, there is no clear recipe for finding a FT quantum circuit with flag-based protocols, especially when we consider hardware constraints, such as the qubit connectivity and available gate set. This work presents a novel approach to automatically discover compact and hardware-adapted FT quantum circuits to make significant progress towards scalable FT quantum computing. We employ reinforcement learning (RL) as an enabling tool, leveraging a fast, parallelized stabilizer quantum circuit simulator and a nontrivial reward function specifically adapted to the problem. We show that, in the task of FT logical state preparation, RL discovers not only circuits with fewer gates and ancillary qubits than published results but also novel circuits without and with hardware constraints of up to distance-5 codes with 25 physical qubits, and they can be implemented directly in experiments. Furthermore, RL allows for straightforward exploration of different qubit connectivities and the use of transfer learning to accelerate the discovery. More generally, our work sets the framework towards the use of RL or other machine learning techniques for FT quantum circuit discovery with hardware constraints to make real progress towards the realization of large-scale quantum computers, addressing tasks beyond state preparation, including magic state preparation, logical gate synthesis, and syndrome measurement.