arXiv:2609.31370
Adaptive Dissipative State Preparation through Reinforcement Learning
Nathan M. Myers, Chenxu Liu, Yulong Dong, Nicholas P. Bauman, Karol Kowalski
Abstract
Dissipative algorithms approach the problem of ground state preparation by mimicking the natural thermalization of a quantum system in contact with a large, low-temperature thermal environment. The environment can be efficiently simulated by a single ancilla qubit with a variable energy gap that is repeatedly coupled to the system qubits to generate a dissipative channel, and then reset after each interaction. Here we present an adaptive implementation of the dissipative algorithm, RL-Adapt, that uses single-shot reinforcement learning to optimize the selection of ancilla frequency and system-bath interaction operator to maximize energy dissipation without relying on a priori knowledge of the system spectrum. The adaptive implementation results in significantly reduced convergence times and can successfully find the ground state even for non-ideal operator pools that fail to converge using non-adaptive, uniform random operator and ancilla frequency selection.
AI Summary
Dissipative ground-state preparation with a resettable ancilla requires interaction operators that connect occupied eigenstates to the ground state and ancilla frequencies resonant with the required transitions. Broad operator pools and frequency ranges reduce reliance on spectral knowledge but can make uniform random selection inefficient. This paper introduces RL-Adapt, which adaptively selects operator–frequency pairs using the energy decrease after each collision as its reward. The method discretizes the ancilla frequency range and maintains a softmax policy over operator–frequency pairs. Standardized rewards update a two-timescale learning rule: a slow baseline is intended to retain spectral information, while a fast component tracks transitions available from the evolving state. Learning occurs within each preparation run, without prior training. Numerical benchmarks cover one-dimensional XY chains and electronic-structure Hamiltonians for $\mathrm{H}_2$, $\mathrm{H}_4$, and a 10-qubit LiH active space, including a downfolded Hamiltonian. Results averaged over 50 independent runs show approximately $7$–$8\times$ fewer collisions for XY-chain convergence than uniform sampling. Adaptation also reaches the ground state in a four-site example where coarse frequency discretization leaves uniform sampling near 80% ground-state population. Spectral-alignment and population-based diagnostics examine what the policy learns. The stated qualifications include little improvement for $\mathrm{H}_2$ with its three-operator pool, persistent requirements for connectivity and frequency coverage, and learning-rate trade-offs that can overemphasize early transitions. All benchmarks set the maximum ancilla frequency using the largest system energy gap. Finite interaction bandwidth can also induce unwanted excitations.
Central claim may not hold: The explicit failure-and-recovery example establishes robustness to coarse frequency discretization, not the abstract's distinct claim of recovery for non-ideal operator pools. All numerical benchmarks use the exact largest system energy gap to set the frequency range, leaving performance with a spectrum-independent bound untested. The simulated reward uses exact energy expectations after every collision without accounting for quantum measurement costs or disturbance, so the reported collision-count reductions do not establish net quantum-resource savings. The stated baseline update with gamma_slow = 2/n_tot makes the baseline track the current logits almost immediately, contradicting the slow-memory mechanism used to explain RL-Adapt.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.