arXiv:2610.21987
Quantum control of solid state qubits via Reinforcement Learning
Dennis Herb, Mirko Rossini, Sabine Wölk, Joachim Ankerhold, Dominik Maile, Alessio Belenchia
Abstract
The use of reinforcement learning (RL) in the design of quantum control strategies for pulse level control of NISQ era quantum devices is attracting growing interest. Model-free RL, not requiring preexisting knowledge of the physical system, acts as a black-box optimization tool well suited for noisy devices. In this work, we provide a simulation-based analysis of the performance of model-free RL for state preparation and qubit gate design in a challenging setup: indirect control of NV nuclear spins in a spin-bath. In particular, the spin-bath is modeled via the generalized cluster correlation expansion accounting for magnetic impurities and intra-bath interactions. We analyze the performance of the RL agents by performing simulations with minimally observable environments and noisy, sparse rewards as introduced in (Phys. Rev. X 12, 011059), and compare them to other state-of-the-art black-box optimization tools.
AI Summary
Can model-free reinforcement learning design indirect control of NV-centre nuclear-spin registers while accounting for a structured quantum spin bath and measurement-limited feedback? This simulation study uses proximal policy optimization (PPO) to choose microwave pulse phases, durations and delays on the electronic spin, sometimes also optimizing the external magnetic field. The agent observes only the episode step, optionally augmented by its previous action, and receives a terminal reward from exact fidelity or a single-shot direct-fidelity estimator. Training uses the generalized cluster correlation expansion (gCCE) for a five-spin $^{13}$C bath; resulting sequences are checked against exact system–bath simulations. For a four-pulse nuclear Hadamard gate, bath-aware training gives mean exact-bath process fidelity $96.34\%$, versus $88.27\%$ without bath-aware training; corresponding single-shot-reward results are $93.44\%$ and $89.00\%$. Bath-trained policies favor shorter sequences and larger fields. For Bell-state preparation between two nuclear spins, exact-bath state fidelity rises from $69.30\%$ without bath-aware training to $81.50\%$ with gCCE1 and $82.77\%$ with gCCE2. The longer Bell sequences require higher-order bath correlations. Comparisons with CMA-ES and Bayesian optimization show task-dependent performance, with the largest RL advantage for bath-free Bell preparation. Domain randomization and subsequent fine-tuning are tested against coupling and drive variations, entirely in simulation. Single-shot training requires roughly ten times more episodes. Bell preparation uses exact-fidelity rewards and initially assumes instantaneous echo pulses; finite-pulse tests approach that limit at Rabi frequencies around $10\,\mathrm{MHz}$. Experimental transfer is not demonstrated, and hyperparameter optimization is not systematic.
Error found: The final gCCE reconstruction incorrectly sums cumulative density-matrix approximations, producing an unnormalized state even in the noninteracting-bath limit; it should instead use the selected cumulative order or properly defined additive corrections.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
- Comments
- 22 pages, 10 figures (main), 11 pages, 8 figures (supplementary)