Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2012.10711 · Quantum

Quantum reinforcement learning in continuous action space

Shaojun Wu, Shan Jin, Dingding Wen, Donghong Han, Xiaoting Wang

Published Mar 12, 2025• 83 citations
arXivHTMLPDF

Abstract

Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.

DOI
10.22331/q-2025-03-12-1660

Related papers

Application of Deep Reinforcement Learning in Quantum Control

Yiming Cheng, Yixi Yin, Jiaying Lin +1 more

Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.

Journal of Physics Conference Series•May 1, 2023•1 citation

Continuous Quantum Reinforcement Learning for Robot Navigation

Theodora-Augustina Drăgan, Alexander Künzner, Robert Wille +1 more

One of the multiple facets of quantum reinforcement learning (QRL) is enhancing reinforcement learning (RL) algorithms with quantum submodules, namely with variational quantum circuits (VQC) as function approx-imators. QRL solutions are empirically proven to require fewer training iterations or adjustable parameters than their classical counterparts, but are usually restricted to applications that have a discrete action space and thus limited industrial relevance. We propose a hybrid quantum-classical (HQC) deep deterministic policy gradient (DDPG) approach for a robot to navigate through a maze using continuous states, continuous actions and using local observations from the robot’s LiDAR sensors. We show that this HQC method can lead to models of comparable test results to the neural network (NN)-based DDPG algorithm, that need around 200 times fewer weights. We also study the scalability of our solution with respect to the number of VQC layers and qubits, and find that in general results improve as the layer and qubit counts increase. The best rewards among all similarly sized HQC and classical DDPG methods correspond to a VQC of 8 qubits and 5 layers with no other NN. This work is another step towards continuous QRL, where literature has been sparse.

International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2025•1 citation

Reinforcement Learning for Gate Synthesis in Noisy Quantum Systems

Amara Katabarwa, Collin Farquhar, Hyeongrak Choi +4 more

We present a gate synthesis application of the Deep Deterministic Policy Gradient (DDPG) algorithm for continuous control in a noisy quantum environment. Gate synthesis plays a crucial role in quantum computing, and optimizing control strategies for gate synthesis is of significant interest. The proposed approach leverages a DNN-based actor-critic architecture to optimize gate synthesis performance. The effectiveness of the algorithm is evaluated through simulations in a noiseless and noisy quantum environment, demonstrating its ability to achieve high-fidelity gate synthesis,$\mathcal{O}(10^{-3})$, despite the presence of Markovian damping terms. The results highlight the potential of DDPG and reinforcement learning techniques for addressing continuous control challenges in quantum computing applications.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•0 citations

Quantum State Generation Via Deep Reinforcement Learning

Shaojun Wu, Shan Jin, Xiaoting Wang

The quantum state generation problem is a major research goal of quantum control and quantum variational algorithms, which use iterative optimization methods to evolve the initial state to the target state. The Twin Delayed Deep Deterministic Policy (TD3) algorithm in reinforcement learning achieves high learning efficiency and better stability in continuous control tasks. Here, using the TD3, we propose a new quantum state preparation method that does not require case-by-case optimization, to find a suitable evolution path to obtain the desired state. Specifically, we input the initial state into the trained actor-network, which can output the parameters of the unitary gates step by step, thus gradually evolving the initial state to the fixed quantum state. According to the reversibility of the unitary transformation, we can obtain a sequence of unitary gates to evolve the fixed state to the desired state. To verify the effectiveness of the algorithm, we perform simulations for one-qubit, two-qubit, and four-qubit cases, and the results show that the trained actor-network can provide appropriate unitary transformations to obtain the fixed state.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•1 citation

Quantum Reinforcement Learning

Hamid D. Ismail

This chapter develops quantum reinforcement learning (QRL) as a principled extension of classical reinforcement learning (RL) into the quantum domain. Building on the standard Markov decision process framework, it shows how policies and value functions can be represented by quantum states and parameterized quantum circuits, with action probabilities arising from Born-rule measurement statistics and value estimates obtained from expectation values of observables. In this formulation, RL is preserved at the algorithmic level while its representational substrate is shifted from classical vector spaces to Hilbert space. The chapter emphasizes variational quantum circuits as the practical backbone of near-term QRL and analyzes how encoding strategies, entanglement structure, and measurement choices shape expressive power and trainability. It also examines quantum-enhanced exploration, showing how stochasticity emerges intrinsically from measurement rather than from externally injected noise. Policy-gradient and value-based approaches are compared, with particular attention to measurement cost, optimization stability, and near-term hardware suitability. This chapter further discusses quantum environments, sample-efficiency considerations, and hands-on implementations such as the quantum multi-armed bandit and quantum grid-world. Together, these topics establish QRL as a rigorous and practically grounded framework for studying sequential decision-making, control, and exploration in the quantum setting.

CRC Press eBooks•Nov 4, 2026•0 citations

Quantum-Enhanced Reinforcement Learning (QRL) for Robotics

Jincheng Zhang

This paper explores the application of Quantum-Enhanced Reinforcement Learning (QRL) to robotics, aiming to accelerate learning speeds and enable more intelligent decision-making in complex environments. Traditional reinforcement learning algorithms often struggle with scalability and convergence, particularly when dealing with high-dimensional state spaces and complex environments. Quantum computing offers the potential to overcome these limitations by leveraging quantum mechanics to perform computations that are intractable for classical computers. We propose a novel QRL framework that utilizes quantum-enhanced reinforcement learning to improve exploration, adaptation, and overall performance. The core mechanism involves employing quantum circuits to represent and manipulate the state space, facilitating more efficient learning and potentially uncovering novel strategies. This research investigates the feasibility and potential impact of QRL on robotic control, focusing on scenarios requiring rapid adaptation and robust decision-making in dynamic environments. We present preliminary results demonstrating the efficacy of our approach in a simulated robotic task involving navigation and obstacle avoidance.

•Sep 3, 2026•0 citations
Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.
Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.
One of the multiple facets of quantum reinforcement learning (QRL) is enhancing reinforcement learning (RL) algorithms with quantum submodules, namely with variational quantum circuits (VQC) as function approx-imators. QRL solutions are empirically proven to require fewer training iterations or adjustable parameters than their classical counterparts, but are usually restricted to applications that have a discrete action space and thus limited industrial relevance. We propose a hybrid quantum-classical (HQC) deep deterministic policy gradient (DDPG) approach for a robot to navigate through a maze using continuous states, continuous actions and using local observations from the robot’s LiDAR sensors. We show that this HQC method can lead to models of comparable test results to the neural network (NN)-based DDPG algorithm, that need around 200 times fewer weights. We also study the scalability of our solution with respect to the number of VQC layers and qubits, and find that in general results improve as the layer and qubit counts increase. The best rewards among all similarly sized HQC and classical DDPG methods correspond to a VQC of 8 qubits and 5 layers with no other NN. This work is another step towards continuous QRL, where literature has been sparse.
We present a gate synthesis application of the Deep Deterministic Policy Gradient (DDPG) algorithm for continuous control in a noisy quantum environment. Gate synthesis plays a crucial role in quantum computing, and optimizing control strategies for gate synthesis is of significant interest. The proposed approach leverages a DNN-based actor-critic architecture to optimize gate synthesis performance. The effectiveness of the algorithm is evaluated through simulations in a noiseless and noisy quantum environment, demonstrating its ability to achieve high-fidelity gate synthesis,O(10−3)\mathcal{O}(10^{-3})O(10−3), despite the presence of Markovian damping terms. The results highlight the potential of DDPG and reinforcement learning techniques for addressing continuous control challenges in quantum computing applications.
The quantum state generation problem is a major research goal of quantum control and quantum variational algorithms, which use iterative optimization methods to evolve the initial state to the target state. The Twin Delayed Deep Deterministic Policy (TD3) algorithm in reinforcement learning achieves high learning efficiency and better stability in continuous control tasks. Here, using the TD3, we propose a new quantum state preparation method that does not require case-by-case optimization, to find a suitable evolution path to obtain the desired state. Specifically, we input the initial state into the trained actor-network, which can output the parameters of the unitary gates step by step, thus gradually evolving the initial state to the fixed quantum state. According to the reversibility of the unitary transformation, we can obtain a sequence of unitary gates to evolve the fixed state to the desired state. To verify the effectiveness of the algorithm, we perform simulations for one-qubit, two-qubit, and four-qubit cases, and the results show that the trained actor-network can provide appropriate unitary transformations to obtain the fixed state.
This chapter develops quantum reinforcement learning (QRL) as a principled extension of classical reinforcement learning (RL) into the quantum domain. Building on the standard Markov decision process framework, it shows how policies and value functions can be represented by quantum states and parameterized quantum circuits, with action probabilities arising from Born-rule measurement statistics and value estimates obtained from expectation values of observables. In this formulation, RL is preserved at the algorithmic level while its representational substrate is shifted from classical vector spaces to Hilbert space. The chapter emphasizes variational quantum circuits as the practical backbone of near-term QRL and analyzes how encoding strategies, entanglement structure, and measurement choices shape expressive power and trainability. It also examines quantum-enhanced exploration, showing how stochasticity emerges intrinsically from measurement rather than from externally injected noise. Policy-gradient and value-based approaches are compared, with particular attention to measurement cost, optimization stability, and near-term hardware suitability. This chapter further discusses quantum environments, sample-efficiency considerations, and hands-on implementations such as the quantum multi-armed bandit and quantum grid-world. Together, these topics establish QRL as a rigorous and practically grounded framework for studying sequential decision-making, control, and exploration in the quantum setting.
This paper explores the application of Quantum-Enhanced Reinforcement Learning (QRL) to robotics, aiming to accelerate learning speeds and enable more intelligent decision-making in complex environments. Traditional reinforcement learning algorithms often struggle with scalability and convergence, particularly when dealing with high-dimensional state spaces and complex environments. Quantum computing offers the potential to overcome these limitations by leveraging quantum mechanics to perform computations that are intractable for classical computers. We propose a novel QRL framework that utilizes quantum-enhanced reinforcement learning to improve exploration, adaptation, and overall performance. The core mechanism involves employing quantum circuits to represent and manipulate the state space, facilitating more efficient learning and potentially uncovering novel strategies. This research investigates the feasibility and potential impact of QRL on robotic control, focusing on scenarios requiring rapid adaptation and robust decision-making in dynamic environments. We present preliminary results demonstrating the efficacy of our approach in a simulated robotic task involving navigation and obstacle avoidance.