Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2104.11856 · Physical Review Letters

Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential

Sangkha Borah, Bijita Sarma, Michael J. Kewming, Gerard J Milburn, Jason Twamley

Published Nov 2, 2021• 87 citations
arXivHTMLPDF

Abstract

Closed loop quantum control uses measurement to control the dynamics of a quantum system to achieve either a desired target state or target dynamics. In the case when the quantum Hamiltonian is quadratic in x and p, there are known optimal control techniques to drive the dynamics toward particular states, e.g., the ground state. However, for nonlinear Hamiltonian such control techniques often fail. We apply deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity. We consider a DRL strategy which is particularly motivated by experiment where the quantum system is continuously but weakly measured. This measurement is then fed back to the neural agent and used for training. We show that the DRL can effectively learn counterintuitive strategies to cool the system to a nearly pure "cat" state, which has a high overlap fidelity with the true ground state.

DOI
10.1103/physrevlett.127.190403

Related papers

Quantum Control based on Deep Reinforcement Learning

Zhikang Wang

In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.

arXiv:2212.07385•Dec 14, 2022•2 citations

Fast State Stabilization Using Deep Reinforcement Learning for Measurement-Based Quantum Feedback Control

Chunxiang Song, Yanan Liu, Daoyi Dong +1 more

The stabilization of quantum states is a fundamental problem for realizing various quantum technologies. Measurement-based-feedback strategies have demonstrated powerful performance, and the construction of quantum control signals using measurement information has attracted great interest. However, the interaction between quantum systems and the environment is inevitable, especially when measurements are introduced, which leads to decoherence. To mitigate decoherence, it is desirable to stabilize quantum systems faster, thereby reducing the time of interaction with the environment. In this paper, we utilize information obtained from measurement and apply deep reinforcement learning (DRL) algorithms, without explicitly constructing specific complex measurement-control mappings, to rapidly drive random initial quantum state to the target state. The proposed DRL algorithm has the ability to speed up the convergence to a target state, which shortens the interaction between quantum systems and their environments to protect coherence. Simulations are performed on two-qubit and three-qubit systems, and the results show that our algorithm can successfully stabilize random initial quantum system to the target entangled state, with a convergence time faster than traditional methods such as Lyapunov feedback control and several DRL algorithms with different reward functions. Moreover, it exhibits robustness against imperfect measurements and delays in system evolution.

arXiv:2408.11328•IEEE Transactions on Quantum Engineering•Jan 1, 2025•5 citations

Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements

Riccardo Porotti, Antoine Essig, Benjamin Huard +1 more

Quantum control has been of increasing interest in recent years, e.g. for tasks like state initialization and stabilization. Feedback-based strategies are particularly powerful, but also hard to find, due to the exponentially increased search space. Deep reinforcement learning holds great promise in this regard. It may provide new answers to difficult questions, such as whether nonlinear measurements can compensate for linear, constrained control. Here we show that reinforcement learning can successfully discover such feedback strategies, without prior knowledge. We illustrate this for state preparation in a cavity subject to quantum-non-demolition detection of photon number, with a simple linear drive as control. Fock states can be produced and stabilized at very high fidelity. It is even possible to reach superposition states, provided the measurement rates for different Fock states can be controlled as well.

arXiv:2107.08816•Quantum•Jun 28, 2022•88 citations

Deep Reinforcement Learning Control of Quantum Cartpoles

Zhikang T. Wang, Yuto Ashida, Masahito Ueda

We generalize a standard benchmark of reinforcement learning, the classical cartpole balancing problem, to the quantum regime by stabilizing a particle in an unstable potential through measurement and feedback. We use state-of-the-art deep reinforcement learning to stabilize a quantum cartpole and find that our deep learning approach performs comparably to or better than other strategies in standard control theory. Our approach also applies to measurement-feedback cooling of quantum oscillators, showing the applicability of deep learning to general continuous-space quantum control.

arXiv:1910.09200•Physical Review Letters•Sep 2, 2020•72 citations

Auxiliary Task-Based Deep Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang +1 more

Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.

arXiv:2302.14312•IEEE Transactions on Cybernetics•Jan 7, 2025•9 citations

Memory-Based Deep Reinforcement Learning for Quantum Control With Partial Observations

Haixu Yu, Qi Yu, Chunlin Chen +1 more

Deep reinforcement learning (DRL) has gained increasing attention in quantum physics, especially for quantum control problems, as it effectively handles high dimensionality and complex dynamics. However, many existing DRL-based quantum control methods assume full observability of quantum states, which tends to be impractical due to the massive state copies required for precise quantum measurements. To address this, we model quantum control under partial observations and introduce a memory-based DRL (MDRL) method that efficiently handles such scenarios. The proposed MDRL comprises two key components: a memory-based exploration strategy and a memory-based reward scheme. Specifically, the agent is equipped with a biology-inspired brain to memorize information derived from partial observations, including whether an episode is successful and the number of steps spent on a successful episode. The exploration strategy leverages the memorized information to guide interactions with the quantum system, while the reward scheme assigns appropriate rewards, together enabling the agent to learn an effective control strategy. The effectiveness of MDRL is verified by three examples, i.e., one-qubit state control, two-qubit state control, and$T_{x}$gate design. Numerical results demonstrate that MDRL outperforms several representative DRL algorithms for quantum control.

IEEE Transactions on Emerging Topics in Computational Intelligence•Nov 28, 2025•2 citations
Closed loop quantum control uses measurement to control the dynamics of a quantum system to achieve either a desired target state or target dynamics. In the case when the quantum Hamiltonian is quadratic in x and p, there are known optimal control techniques to drive the dynamics toward particular states, e.g., the ground state. However, for nonlinear Hamiltonian such control techniques often fail. We apply deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity. We consider a DRL strategy which is particularly motivated by experiment where the quantum system is continuously but weakly measured. This measurement is then fed back to the neural agent and used for training. We show that the DRL can effectively learn counterintuitive strategies to cool the system to a nearly pure "cat" state, which has a high overlap fidelity with the true ground state.
In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.
The stabilization of quantum states is a fundamental problem for realizing various quantum technologies. Measurement-based-feedback strategies have demonstrated powerful performance, and the construction of quantum control signals using measurement information has attracted great interest. However, the interaction between quantum systems and the environment is inevitable, especially when measurements are introduced, which leads to decoherence. To mitigate decoherence, it is desirable to stabilize quantum systems faster, thereby reducing the time of interaction with the environment. In this paper, we utilize information obtained from measurement and apply deep reinforcement learning (DRL) algorithms, without explicitly constructing specific complex measurement-control mappings, to rapidly drive random initial quantum state to the target state. The proposed DRL algorithm has the ability to speed up the convergence to a target state, which shortens the interaction between quantum systems and their environments to protect coherence. Simulations are performed on two-qubit and three-qubit systems, and the results show that our algorithm can successfully stabilize random initial quantum system to the target entangled state, with a convergence time faster than traditional methods such as Lyapunov feedback control and several DRL algorithms with different reward functions. Moreover, it exhibits robustness against imperfect measurements and delays in system evolution.
Quantum control has been of increasing interest in recent years, e.g. for tasks like state initialization and stabilization. Feedback-based strategies are particularly powerful, but also hard to find, due to the exponentially increased search space. Deep reinforcement learning holds great promise in this regard. It may provide new answers to difficult questions, such as whether nonlinear measurements can compensate for linear, constrained control. Here we show that reinforcement learning can successfully discover such feedback strategies, without prior knowledge. We illustrate this for state preparation in a cavity subject to quantum-non-demolition detection of photon number, with a simple linear drive as control. Fock states can be produced and stabilized at very high fidelity. It is even possible to reach superposition states, provided the measurement rates for different Fock states can be controlled as well.
We generalize a standard benchmark of reinforcement learning, the classical cartpole balancing problem, to the quantum regime by stabilizing a particle in an unstable potential through measurement and feedback. We use state-of-the-art deep reinforcement learning to stabilize a quantum cartpole and find that our deep learning approach performs comparably to or better than other strategies in standard control theory. Our approach also applies to measurement-feedback cooling of quantum oscillators, showing the applicability of deep learning to general continuous-space quantum control.
Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.
Deep reinforcement learning (DRL) has gained increasing attention in quantum physics, especially for quantum control problems, as it effectively handles high dimensionality and complex dynamics. However, many existing DRL-based quantum control methods assume full observability of quantum states, which tends to be impractical due to the massive state copies required for precise quantum measurements. To address this, we model quantum control under partial observations and introduce a memory-based DRL (MDRL) method that efficiently handles such scenarios. The proposed MDRL comprises two key components: a memory-based exploration strategy and a memory-based reward scheme. Specifically, the agent is equipped with a biology-inspired brain to memorize information derived from partial observations, including whether an episode is successful and the number of steps spent on a successful episode. The exploration strategy leverages the memorized information to guide interactions with the quantum system, while the reward scheme assigns appropriate rewards, together enabling the agent to learn an effective control strategy. The effectiveness of MDRL is verified by three examples, i.e., one-qubit state control, two-qubit state control, andTxT_{x}Tx​gate design. Numerical results demonstrate that MDRL outperforms several representative DRL algorithms for quantum control.