Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

A framework for deep energy-based reinforcement learning with quantum speed-up

Sofiène Jerbi, Hendrik Poulsen Nautrup, Lea M. Trenkwalder, Hans J. Briegel, Vedran Dunjko

Published Oct 28, 2019• 4 citations
OpenAlex

Abstract

In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.

Related papers

On the convergence of projective-simulation–based reinforcement learning in Markov decision processes

W. L. Boyajian, J. Clausen, L. M. Trenkwalder +2 more

In recent years, the interest in leveraging quantum effects for enhancing machine learning tasks has significantly increased. Many algorithms speeding up supervised and unsupervised learning were established. The first framework in which ways to exploit quantum resources specifically for the broader context of reinforcement learning were found is projective simulation. Projective simulation presents an agent-based reinforcement learning approach designed in a manner which may support quantum walk-based speedups. Although classical variants of projective simulation have been benchmarked against common reinforcement learning algorithms, very few formal theoretical analyses have been provided for its performance in standard learning scenarios. In this paper, we provide a detailed formal discussion of the properties of this model. Specifically, we prove that one version of the projective simulation model, understood as a reinforcement learning approach, converges to optimal behavior in a large class of Markov decision processes. This proof shows that a physically inspired approach to reinforcement learning can guarantee to converge.

arXiv:1910.11914•Quantum Machine Intelligence•Nov 5, 2020•22 citations

Survey Paper on Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Shruti Doiphode, Prof. Mukul Jagtap

Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation

International Journal for Research in Applied Science and Engineering Technology•Oct 18, 2024•0 citations

Reinforcement Learning with Deep Quantum Neural Networks

Wei Hu, James Lee Hu

The advantage of quantum computers over classical computers fuels the recent trend of developing machine learning algorithms on quantum computers, which can potentially lead to breakthroughs and new learning models in this area. The aim of our study is to explore deep quantum reinforcement learning (RL) on photonic quantum computers, which can process information stored in the quantum states of light. These quantum computers can naturally represent continuous variables, making them an ideal platform to create quantum versions of neural networks. Using quantum photonic circuits, we implement Q learning and actor-critic algorithms with multilayer quantum neural networks and test them in the grid world environment. Our experiments show that 1) these quantum algorithms can solve the RL problem and 2) compared to one layer, using three layer quantum networks improves the learning of both algorithms in terms of rewards collected. In summary, our findings suggest that having more layers in deep quantum RL can enhance the learning outcome.

Journal of Quantum Information Science•Jan 1, 2019•13 citations

Control of Continuous Quantum Systems with Many Degrees of Freedom based on Convergent Reinforcement Learning

智康 王

With advances in digital technology in recent years, parallel computation utilizing GPUs has achieved remarkable efficiency, which has made it possible to use large-scale machine learning algorithms with large datasets, and deep learning, which is machine learning utilizing deep neural networks trained with millions to billions of data and parameters, has become increasingly popular and found its use for various tasks including facial recognition, language translation and combinatorial problems, etc., achieving considerably many record-breaking results.As a powerful generic tool to solve problems, deep learning has been applied to physical problems as well, and has become an important alternative tool for scientists to solve a number of important problems.With the development of experimental quantum technology over the past several decades, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems, which have opened up a new regime of complex quantum systems that can be engineered.Quantum control has found its use in controlled quantum-chemical processes and artificial quantum systems including quantum dots, superconducting qubits, trapped ions, and cavity optomechanical systems, etc., which are of considerable importance for future technology as candidates for sensors and quantum-computational devices.However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and generic numerical algorithms which search for proper control protocols are adopted.Therefore, deep learning, especially deep reinforcement learning, is a promising generic candidate solution for the control problems.Although there have already been a few successful examples of applications of deep reinforcement learning to quantum control problems, most of the existing reinforcement learning algorithms intrinsically suffer from instabilities and unsatisfactory reproducibility, and as a consequence, they typically require a large amount of fine-tuning and a large computational budget, both of which limit their applicability to quantum control problems and require expertise in machine learning.To resolve the issue of instabilities of the reinforcement learning algorithms, in this thesis, we first investigate the non-convergence issue of Q-learning, which is one of the most efficient reinforcement learning strategies.Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm.We prove the convergence of the C-DQN algorithm, and since the algorithm is scalable and computationally efficient, we apply it to a standard reinforcement learning benchmark, the Atari 2600 benchmark, to demonstrate its effectiveness.We show that when the DQN algorithm fail, the C-DQN algorithm still learns successfully.Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum-mechanical quartic oscillator i

•Mar 23, 2023•0 citations

On the Use of Quantum Reinforcement Learning in Energy-Efficiency Scenarios

Eva Andrés, Manuel Pegalájar Cuéllar, Gabriel Navarro

In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.

Energies•Aug 19, 2022•28 citations

Achieving Reinforcement Learning Speedup via Quantum Kernel Estimation

John Gilvarry, Vincent Chimaobi Emeakaroha

Reinforcement learning is playing a crucial role in sustainable developments and enabling wide applications in our daily lives ranging from self-driving cars, robotics, automation, personalised product suggestions, user notifications and many more. Kernel-based reinforcement learning holds theoretical guarantees of convergence to a unique solution for decision function approximation. This has motivated the development of implementations that exploit the advantage of kernel methods whilst overcoming the ‘curse of dimensionality’. The theoretical properties of kernel-based methods also support the combination of learning agents in loosely coupled systems into a single combined model for composable learning. Learning speedup is achievable by introducing expert knowledge to the algorithm through the kernel, which reduces variance by increasing bias and thus learning speed. Despite the theoretical promise of kernel based reinforcement learning methods, and continued evolution of more efficient algorithms, practical implementation are not prevalent. Research in quantum machine learning has successfully applied quantum algorithms to achieve computational speed-ups for various machine-learning tasks, and has also identified potential quantum advantage through the identification of patterns not accessible to classical data mapping. To date, this has not been applied to kernel based reinforcement learning. Recent work in the field of quantum learning suggests an important duality between quantum machine learning and kernel machine learning methods. Therefore, this paper proposes a framework to complement kernel-based reinforcement learning with quantum generated feature maps for function approximation to improve learning speed. This has positive consequences for optimisation of reinforcement learning techniques already in use to support sustainability targets in applications such as industry innovation and infrastructure, power generation management, data centre cooling and self-driving cars. Our established framework enables the adoption of emerging quantum kernel methods for reinforcement learning applications. The achieved results show over 20 % improvement and positive impacts of quantum map selection on model performance for different problems.

IEEE International Symposium on Technology and Society (ISTAS)•Sep 13, 2023•2 citations
In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.
In recent years, the interest in leveraging quantum effects for enhancing machine learning tasks has significantly increased. Many algorithms speeding up supervised and unsupervised learning were established. The first framework in which ways to exploit quantum resources specifically for the broader context of reinforcement learning were found is projective simulation. Projective simulation presents an agent-based reinforcement learning approach designed in a manner which may support quantum walk-based speedups. Although classical variants of projective simulation have been benchmarked against common reinforcement learning algorithms, very few formal theoretical analyses have been provided for its performance in standard learning scenarios. In this paper, we provide a detailed formal discussion of the properties of this model. Specifically, we prove that one version of the projective simulation model, understood as a reinforcement learning approach, converges to optimal behavior in a large class of Markov decision processes. This proof shows that a physically inspired approach to reinforcement learning can guarantee to converge.
Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation
The advantage of quantum computers over classical computers fuels the recent trend of developing machine learning algorithms on quantum computers, which can potentially lead to breakthroughs and new learning models in this area. The aim of our study is to explore deep quantum reinforcement learning (RL) on photonic quantum computers, which can process information stored in the quantum states of light. These quantum computers can naturally represent continuous variables, making them an ideal platform to create quantum versions of neural networks. Using quantum photonic circuits, we implement Q learning and actor-critic algorithms with multilayer quantum neural networks and test them in the grid world environment. Our experiments show that 1) these quantum algorithms can solve the RL problem and 2) compared to one layer, using three layer quantum networks improves the learning of both algorithms in terms of rewards collected. In summary, our findings suggest that having more layers in deep quantum RL can enhance the learning outcome.
With advances in digital technology in recent years, parallel computation utilizing GPUs has achieved remarkable efficiency, which has made it possible to use large-scale machine learning algorithms with large datasets, and deep learning, which is machine learning utilizing deep neural networks trained with millions to billions of data and parameters, has become increasingly popular and found its use for various tasks including facial recognition, language translation and combinatorial problems, etc., achieving considerably many record-breaking results.As a powerful generic tool to solve problems, deep learning has been applied to physical problems as well, and has become an important alternative tool for scientists to solve a number of important problems.With the development of experimental quantum technology over the past several decades, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems, which have opened up a new regime of complex quantum systems that can be engineered.Quantum control has found its use in controlled quantum-chemical processes and artificial quantum systems including quantum dots, superconducting qubits, trapped ions, and cavity optomechanical systems, etc., which are of considerable importance for future technology as candidates for sensors and quantum-computational devices.However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and generic numerical algorithms which search for proper control protocols are adopted.Therefore, deep learning, especially deep reinforcement learning, is a promising generic candidate solution for the control problems.Although there have already been a few successful examples of applications of deep reinforcement learning to quantum control problems, most of the existing reinforcement learning algorithms intrinsically suffer from instabilities and unsatisfactory reproducibility, and as a consequence, they typically require a large amount of fine-tuning and a large computational budget, both of which limit their applicability to quantum control problems and require expertise in machine learning.To resolve the issue of instabilities of the reinforcement learning algorithms, in this thesis, we first investigate the non-convergence issue of Q-learning, which is one of the most efficient reinforcement learning strategies.Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm.We prove the convergence of the C-DQN algorithm, and since the algorithm is scalable and computationally efficient, we apply it to a standard reinforcement learning benchmark, the Atari 2600 benchmark, to demonstrate its effectiveness.We show that when the DQN algorithm fail, the C-DQN algorithm still learns successfully.Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum-mechanical quartic oscillator i
In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.
Reinforcement learning is playing a crucial role in sustainable developments and enabling wide applications in our daily lives ranging from self-driving cars, robotics, automation, personalised product suggestions, user notifications and many more. Kernel-based reinforcement learning holds theoretical guarantees of convergence to a unique solution for decision function approximation. This has motivated the development of implementations that exploit the advantage of kernel methods whilst overcoming the ‘curse of dimensionality’. The theoretical properties of kernel-based methods also support the combination of learning agents in loosely coupled systems into a single combined model for composable learning. Learning speedup is achievable by introducing expert knowledge to the algorithm through the kernel, which reduces variance by increasing bias and thus learning speed. Despite the theoretical promise of kernel based reinforcement learning methods, and continued evolution of more efficient algorithms, practical implementation are not prevalent. Research in quantum machine learning has successfully applied quantum algorithms to achieve computational speed-ups for various machine-learning tasks, and has also identified potential quantum advantage through the identification of patterns not accessible to classical data mapping. To date, this has not been applied to kernel based reinforcement learning. Recent work in the field of quantum learning suggests an important duality between quantum machine learning and kernel machine learning methods. Therefore, this paper proposes a framework to complement kernel-based reinforcement learning with quantum generated feature maps for function approximation to improve learning speed. This has positive consequences for optimisation of reinforcement learning techniques already in use to support sustainability targets in applications such as industry innovation and infrastructure, power generation management, data centre cooling and self-driving cars. Our established framework enables the adoption of emerging quantum kernel methods for reinforcement learning applications. The achieved results show over 20 % improvement and positive impacts of quantum map selection on model performance for different problems.