Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2609.31370

Adaptive Dissipative State Preparation through Reinforcement Learning

Nathan M. Myers, Chenxu Liu, Yulong Dong, Nicholas P. Bauman, Karol Kowalski

quant-phSubmitted Sep 25, 2026• Announced Sep 28, 2026• 0 citations
arXivHTMLPDF

Abstract

Dissipative algorithms approach the problem of ground state preparation by mimicking the natural thermalization of a quantum system in contact with a large, low-temperature thermal environment. The environment can be efficiently simulated by a single ancilla qubit with a variable energy gap that is repeatedly coupled to the system qubits to generate a dissipative channel, and then reset after each interaction. Here we present an adaptive implementation of the dissipative algorithm, RL-Adapt, that uses single-shot reinforcement learning to optimize the selection of ancilla frequency and system-bath interaction operator to maximize energy dissipation without relying on a priori knowledge of the system spectrum. The adaptive implementation results in significantly reduced convergence times and can successfully find the ground state even for non-ideal operator pools that fail to converge using non-adaptive, uniform random operator and ancilla frequency selection.

AI Summary

Dissipative ground-state preparation with a resettable ancilla requires interaction operators that connect occupied eigenstates to the ground state and ancilla frequencies resonant with the required transitions. Broad operator pools and frequency ranges reduce reliance on spectral knowledge but can make uniform random selection inefficient. This paper introduces RL-Adapt, which adaptively selects operator–frequency pairs using the energy decrease after each collision as its reward. The method discretizes the ancilla frequency range and maintains a softmax policy over operator–frequency pairs. Standardized rewards update a two-timescale learning rule: a slow baseline is intended to retain spectral information, while a fast component tracks transitions available from the evolving state. Learning occurs within each preparation run, without prior training. Numerical benchmarks cover one-dimensional XY chains and electronic-structure Hamiltonians for $\mathrm{H}_2$, $\mathrm{H}_4$, and a 10-qubit LiH active space, including a downfolded Hamiltonian. Results averaged over 50 independent runs show approximately $7$–$8\times$ fewer collisions for XY-chain convergence than uniform sampling. Adaptation also reaches the ground state in a four-site example where coarse frequency discretization leaves uniform sampling near 80% ground-state population. Spectral-alignment and population-based diagnostics examine what the policy learns. The stated qualifications include little improvement for $\mathrm{H}_2$ with its three-operator pool, persistent requirements for connectivity and frequency coverage, and learning-rate trade-offs that can overemphasize early transitions. All benchmarks set the maximum ancilla frequency using the largest system energy gap. Finite interaction bandwidth can also induce unwanted excitations.

Little sign of AI-generated textSmall numerics onlyStrong assumptionsMissing comparisonPossible error

Central claim may not hold: The explicit failure-and-recovery example establishes robustness to coarse frequency discretization, not the abstract's distinct claim of recovery for non-ideal operator pools. All numerical benchmarks use the exact largest system energy gap to set the frequency range, leaving performance with a spectrum-independent bound untested. The simulated reward uses exact energy expectations after every collision without accounting for quantum measurement costs or disturbance, so the reported collision-count reductions do not establish net quantum-resource savings. The stated baseline update with gamma_slow = 2/n_tot makes the baseline track the current logits almost immediately, contradicting the slow-memory mechanism used to explain RL-Adapt.

Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.

Related papers

A Variational Dissipative Framework for Quantum Algorithms

Yuan Yao, Ruipeng Xing, Yongjian Gu +2 more

Dissipation engineering has attracted growing interest as an approach to controlling open quantum systems through engineered system-environment interactions. Standard variational quantum circuits are usually built from unitary operations and therefore explore only a restricted family of states. To go beyond this limitation, we introduce a variational dissipative framework in which ancilla-assisted engineered dissipation is incorporated into parameterized quantum algorithms. In this framework, system-only variational layers are combined with trainable dissipative modules, so that the circuit can prepare a broader class of mixed states through ancilla-assisted nonunitary transformations. Within this framework, the same ancilla-assisted dissipative block is used in two representative settings with different objectives. For ground-state search, it is integrated into a dissipative variational quantum eigensolver to improve the convergence toward low-energy states. For state recovery, it is trained as a recovery channel to suppress preparation noise and enhance fidelity with the target state. In both cases, the block is realized through parameterized system-ancilla couplings followed by ancilla reset and trace-out. Our results show that engineered dissipation can be incorporated into variational quantum circuits as a reusable trainable primitive rather than treated only as a source of noise. In this sense, the proposed framework identifies ancilla-assisted dissipative channels as a concrete variational resource that can support both optimization and recovery tasks within a unified design.

quant-pharXiv:2605.25841•May 26, 2026•1 citation

Rigorous error bounds for dissipative thermal state preparation from weak system-bath coupling

Christopher Ong, S. A. Parameswaran, Benedikt Placke +1 more

Thermal state preparation is a central challenge in the simulation of quantum many-body systems. Yet, provably efficient algorithms for this task were only introduced recently [Chen et al. Nature 646, 561 (2025)]. These algorithms are based on dissipative Lindbladian evolution which exactly fixes the thermal state. Controlled and efficient digital simulation of this evolution, although possible in principle, remains out of reach for present-day quantum hardware. Subsequent work has therefore focused on analog approximations of the proposed Lindbladians via `collision models' with relatively modest requirements -- a resettable bath of ancilla qubits whose couplings to the system can be tuned in time-dependent fashion -- while still admitting rigorous fixed-point error bounds. Existing rigorous approaches, however, do not exploit the fact that these constructions generically implement not only the desired Lindblad dynamics, but also an additional unitary evolution generated by the system Hamiltonian which may aid convergence to the thermal state [Lloyd and Abanin arXiv:2506.21318 (2025)]. Here, we show that this unitary contribution does indeed tighten the fixed-point error bound and demonstrate that it is rigorously controlled by the system-bath coupling strength $J$, scaling as $J^2$. This demonstrates that the effect of the spurious `Lamb shift' term generated by the system-bath interaction can be controlled by tuning $J$. We clarify the role, previously observed, of a randomized implementation in suppressing possible resonances of the drive with the many-body spectrum, and bound the additional variance that this randomization imposes on observables. Finally, we numerically study aspects of the protocol which are relevant for its practical realization, such as the mixing time.

quant-pharXiv:2605.03011•Physical Review A•May 6, 2026•4 citations

A Quantum States Preparation Method Based on Difference-Driven Reinforcement Learning

Wenjie Liu, Jing Xu, Bosi Wang

Due to the large state space of the two-qubit system, and the adoption of ladder reward function in the existing quantum state preparation methods, the convergence speed is slow and it is difficult to prepare the desired target quantum state with high fidelity under limited conditions. To solve the above problems, a difference-driven reinforcement learning (RL) algorithm for quantum state preparation of two-qubit system is proposed by improving the reward function and action selection strategy. Firstly, a model is constructed for the problem of preparing quantum states of a two-qubit system, with restrictions on the type of quantum gates and the time for quantum state evolution. In the preparation process, a weighted differential dynamic reward function is designed to assist the algorithm quickly obtain the maximum expected cumulative reward. Then, an adaptive [Formula: see text]-greedy action selection strategy is adopted to achieve a balance between exploration and utilization to a certain extent, thereby improving the fidelity of the final quantum state. The simulation results show that the proposed algorithm can prepare quantum state with high fidelity under limited conditions. Compared with other algorithms, it has different degrees of improvement in convergence speed and fidelity of the final quantum state.

arXiv:2309.16972•SPIN•May 19, 2023•5 citations

Dissipative Variational Quantum Algorithms for Gibbs State Preparation

Yigal Ilin, Itai Arad

In recent years, variational quantum algorithms have gained significant attention due to their adaptability and efficiency on near-term quantum hardware. They have shown potential in a variety of tasks, including linear algebra, search problems, Gibbs, and ground state preparation. Nevertheless, the presence of noise in current day quantum hardware severely limits their performance. In this work, we introduce dissipative variational quantum algorithms (D-VQAs) by incorporating dissipative operations, such as qubit RESET and stochastic gates, as an intrinsic part of a variational quantum circuit. We argue that such dissipative variational algorithms possess some natural resilience to dissipative noise. We demonstrate how such algorithms can prepare Gibbs states over a wide range of quantum many-body Hamiltonians and temperatures, while significantly reducing errors due to both coherent and noncoherent noise. An additional advantage of our approach is that no ancilla qubits are need. Our results highlight the potential of D-VQAs to enhance the robustness and accuracy of variational quantum computations on noisy intermediate-scale quantum (NISQ) devices.

arXiv:2407.09635•IEEE Transactions on Quantum Engineering•Dec 4, 2024•22 citations

Sublinear thermal state and ground state preparation for free fermions via system-bath interaction

Zhiyan Ding, Lin Lin, Ke Wang

Dissipative algorithms for preparing thermal and ground states typically require a total Hamiltonian simulation time that grows at least linearly with system size $N$, even when their mixing time grows only polylogarithmically. We achieve sublinear simulation time for free fermions by engineering local interactions with a bath whose size grows in proportion to the system size. Starting from arbitrary initial states, our thermal state preparation algorithm achieves trace-norm error $ε$ in total simulation time $\widetilde{O}(\sqrt{N}/ε)$ at fixed inverse temperature and bounded single-particle bandwidth, while ground state preparation of gapped systems requires only $\mathrm{polylog}(N/ε)$ time. We further establish lower bounds for thermal state preparation within the bath protocols considered, identifying two obstacles to faster preparation, namely fourth-order deviations from the target thermal state and fluctuations induced by a random frequency shared by all bath qubits. Our results demonstrate that the linear simulation time barrier in dissipative state preparation can be overcome and clarify the bath and filter-design constraints that govern further complexity improvements.

quant-pharXiv:2610.35370•Oct 27, 2026•0 citations

Quantum Markov chain Monte Carlo with digital dissipative dynamics on quantum computers

Mekena Metcalf, Emma Stone, Katherine Klymko +3 more

Abstract Modeling the dynamics of a quantum system connected to the environment is critical for advancing our understanding of complex quantum processes, as most quantum processes in nature are affected by an environment. Modeling a macroscopic environment on a quantum simulator may be achieved by coupling independent ancilla qubits that facilitate energy exchange in an appropriate manner with the system and mimic an environment. This approach requires a large, and possibly exponential number of ancillary degrees of freedom which is impractical. In contrast, we develop a digital quantum algorithm that simulates interaction with an environment using a small number of ancilla qubits. By combining periodic modulation of the ancilla energies, or spectral combing, with periodic reset operations, we are able to mimic interaction with a large environment and generate thermal states of interacting many-body systems. We evaluate the algorithm by simulating preparation of thermal states of the transverse Ising model. Our algorithm can also be viewed as a quantum Markov chain Monte Carlo process that allows sampling of the Gibbs distribution of a multivariate model. To demonstrate this we evaluate the accuracy of sampling Gibbs distributions of simple probabilistic graphical models using the algorithm.

arXiv:2103.03207•Quantum Science and Technology•Feb 11, 2022•11 citations
Dissipative algorithms approach the problem of ground state preparation by mimicking the natural thermalization of a quantum system in contact with a large, low-temperature thermal environment. The environment can be efficiently simulated by a single ancilla qubit with a variable energy gap that is repeatedly coupled to the system qubits to generate a dissipative channel, and then reset after each interaction. Here we present an adaptive implementation of the dissipative algorithm, RL-Adapt, that uses single-shot reinforcement learning to optimize the selection of ancilla frequency and system-bath interaction operator to maximize energy dissipation without relying on a priori knowledge of the system spectrum. The adaptive implementation results in significantly reduced convergence times and can successfully find the ground state even for non-ideal operator pools that fail to converge using non-adaptive, uniform random operator and ancilla frequency selection.
Dissipative ground-state preparation with a resettable ancilla requires interaction operators that connect occupied eigenstates to the ground state and ancilla frequencies resonant with the required transitions. Broad operator pools and frequency ranges reduce reliance on spectral knowledge but can make uniform random selection inefficient. This paper introduces RL-Adapt, which adaptively selects operator–frequency pairs using the energy decrease after each collision as its reward.
The method discretizes the ancilla frequency range and maintains a softmax policy over operator–frequency pairs. Standardized rewards update a two-timescale learning rule: a slow baseline is intended to retain spectral information, while a fast component tracks transitions available from the evolving state. Learning occurs within each preparation run, without prior training.
Numerical benchmarks cover one-dimensional XY chains and electronic-structure Hamiltonians for H2\mathrm{H}_2H2​, H4\mathrm{H}_4H4​, and a 10-qubit LiH active space, including a downfolded Hamiltonian. Results averaged over 50 independent runs show approximately 777–8×8\times8× fewer collisions for XY-chain convergence than uniform sampling. Adaptation also reaches the ground state in a four-site example where coarse frequency discretization leaves uniform sampling near 80% ground-state population. Spectral-alignment and population-based diagnostics examine what the policy learns.
The stated qualifications include little improvement for H2\mathrm{H}_2H2​ with its three-operator pool, persistent requirements for connectivity and frequency coverage, and learning-rate trade-offs that can overemphasize early transitions. All benchmarks set the maximum ancilla frequency using the largest system energy gap. Finite interaction bandwidth can also induce unwanted excitations.
Dissipation engineering has attracted growing interest as an approach to controlling open quantum systems through engineered system-environment interactions. Standard variational quantum circuits are usually built from unitary operations and therefore explore only a restricted family of states. To go beyond this limitation, we introduce a variational dissipative framework in which ancilla-assisted engineered dissipation is incorporated into parameterized quantum algorithms. In this framework, system-only variational layers are combined with trainable dissipative modules, so that the circuit can prepare a broader class of mixed states through ancilla-assisted nonunitary transformations. Within this framework, the same ancilla-assisted dissipative block is used in two representative settings with different objectives. For ground-state search, it is integrated into a dissipative variational quantum eigensolver to improve the convergence toward low-energy states. For state recovery, it is trained as a recovery channel to suppress preparation noise and enhance fidelity with the target state. In both cases, the block is realized through parameterized system-ancilla couplings followed by ancilla reset and trace-out. Our results show that engineered dissipation can be incorporated into variational quantum circuits as a reusable trainable primitive rather than treated only as a source of noise. In this sense, the proposed framework identifies ancilla-assisted dissipative channels as a concrete variational resource that can support both optimization and recovery tasks within a unified design.
Thermal state preparation is a central challenge in the simulation of quantum many-body systems. Yet, provably efficient algorithms for this task were only introduced recently [Chen et al. Nature 646, 561 (2025)]. These algorithms are based on dissipative Lindbladian evolution which exactly fixes the thermal state. Controlled and efficient digital simulation of this evolution, although possible in principle, remains out of reach for present-day quantum hardware. Subsequent work has therefore focused on analog approximations of the proposed Lindbladians via `collision models' with relatively modest requirements -- a resettable bath of ancilla qubits whose couplings to the system can be tuned in time-dependent fashion -- while still admitting rigorous fixed-point error bounds. Existing rigorous approaches, however, do not exploit the fact that these constructions generically implement not only the desired Lindblad dynamics, but also an additional unitary evolution generated by the system Hamiltonian which may aid convergence to the thermal state [Lloyd and Abanin arXiv:2506.21318 (2025)]. Here, we show that this unitary contribution does indeed tighten the fixed-point error bound and demonstrate that it is rigorously controlled by the system-bath coupling strength JJJ, scaling as J2J^2J2. This demonstrates that the effect of the spurious `Lamb shift' term generated by the system-bath interaction can be controlled by tuning JJJ. We clarify the role, previously observed, of a randomized implementation in suppressing possible resonances of the drive with the many-body spectrum, and bound the additional variance that this randomization imposes on observables. Finally, we numerically study aspects of the protocol which are relevant for its practical realization, such as the mixing time.
Due to the large state space of the two-qubit system, and the adoption of ladder reward function in the existing quantum state preparation methods, the convergence speed is slow and it is difficult to prepare the desired target quantum state with high fidelity under limited conditions. To solve the above problems, a difference-driven reinforcement learning (RL) algorithm for quantum state preparation of two-qubit system is proposed by improving the reward function and action selection strategy. Firstly, a model is constructed for the problem of preparing quantum states of a two-qubit system, with restrictions on the type of quantum gates and the time for quantum state evolution. In the preparation process, a weighted differential dynamic reward function is designed to assist the algorithm quickly obtain the maximum expected cumulative reward. Then, an adaptive [Formula: see text]-greedy action selection strategy is adopted to achieve a balance between exploration and utilization to a certain extent, thereby improving the fidelity of the final quantum state. The simulation results show that the proposed algorithm can prepare quantum state with high fidelity under limited conditions. Compared with other algorithms, it has different degrees of improvement in convergence speed and fidelity of the final quantum state.
In recent years, variational quantum algorithms have gained significant attention due to their adaptability and efficiency on near-term quantum hardware. They have shown potential in a variety of tasks, including linear algebra, search problems, Gibbs, and ground state preparation. Nevertheless, the presence of noise in current day quantum hardware severely limits their performance. In this work, we introduce dissipative variational quantum algorithms (D-VQAs) by incorporating dissipative operations, such as qubit RESET and stochastic gates, as an intrinsic part of a variational quantum circuit. We argue that such dissipative variational algorithms possess some natural resilience to dissipative noise. We demonstrate how such algorithms can prepare Gibbs states over a wide range of quantum many-body Hamiltonians and temperatures, while significantly reducing errors due to both coherent and noncoherent noise. An additional advantage of our approach is that no ancilla qubits are need. Our results highlight the potential of D-VQAs to enhance the robustness and accuracy of variational quantum computations on noisy intermediate-scale quantum (NISQ) devices.
Dissipative algorithms for preparing thermal and ground states typically require a total Hamiltonian simulation time that grows at least linearly with system size NNN, even when their mixing time grows only polylogarithmically. We achieve sublinear simulation time for free fermions by engineering local interactions with a bath whose size grows in proportion to the system size. Starting from arbitrary initial states, our thermal state preparation algorithm achieves trace-norm error εεε in total simulation time O~(N/ε)\widetilde{O}(\sqrt{N}/ε)O(N​/ε) at fixed inverse temperature and bounded single-particle bandwidth, while ground state preparation of gapped systems requires only polylog(N/ε)\mathrm{polylog}(N/ε)polylog(N/ε) time. We further establish lower bounds for thermal state preparation within the bath protocols considered, identifying two obstacles to faster preparation, namely fourth-order deviations from the target thermal state and fluctuations induced by a random frequency shared by all bath qubits. Our results demonstrate that the linear simulation time barrier in dissipative state preparation can be overcome and clarify the bath and filter-design constraints that govern further complexity improvements.
Abstract Modeling the dynamics of a quantum system connected to the environment is critical for advancing our understanding of complex quantum processes, as most quantum processes in nature are affected by an environment. Modeling a macroscopic environment on a quantum simulator may be achieved by coupling independent ancilla qubits that facilitate energy exchange in an appropriate manner with the system and mimic an environment. This approach requires a large, and possibly exponential number of ancillary degrees of freedom which is impractical. In contrast, we develop a digital quantum algorithm that simulates interaction with an environment using a small number of ancilla qubits. By combining periodic modulation of the ancilla energies, or spectral combing, with periodic reset operations, we are able to mimic interaction with a large environment and generate thermal states of interacting many-body systems. We evaluate the algorithm by simulating preparation of thermal states of the transverse Ising model. Our algorithm can also be viewed as a quantum Markov chain Monte Carlo process that allows sampling of the Gibbs distribution of a multivariate model. To demonstrate this we evaluate the accuracy of sampling Gibbs distributions of simple probabilistic graphical models using the algorithm.