Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Energies

On the Use of Quantum Reinforcement Learning in Energy-Efficiency Scenarios

Eva Andrés, Manuel Pegalájar Cuéllar, Gabriel Navarro

Published Aug 19, 2022• 28 citations
Publisher

Abstract

In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.

DOI
10.3390/en15166034

Related papers

Reinforcement Learning with Deep Quantum Neural Networks

Wei Hu, James Lee Hu

The advantage of quantum computers over classical computers fuels the recent trend of developing machine learning algorithms on quantum computers, which can potentially lead to breakthroughs and new learning models in this area. The aim of our study is to explore deep quantum reinforcement learning (RL) on photonic quantum computers, which can process information stored in the quantum states of light. These quantum computers can naturally represent continuous variables, making them an ideal platform to create quantum versions of neural networks. Using quantum photonic circuits, we implement Q learning and actor-critic algorithms with multilayer quantum neural networks and test them in the grid world environment. Our experiments show that 1) these quantum algorithms can solve the RL problem and 2) compared to one layer, using three layer quantum networks improves the learning of both algorithms in terms of rewards collected. In summary, our findings suggest that having more layers in deep quantum RL can enhance the learning outcome.

Journal of Quantum Information Science•Jan 1, 2019•13 citations

A framework for deep energy-based reinforcement learning with quantum speed-up

Sofiène Jerbi, Hendrik Poulsen Nautrup, Lea M. Trenkwalder +2 more

In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.

•Oct 28, 2019•4 citations

From Classical to Quantum: A Review of Recent Progress in Reinforcement Learning

Kunal Kashyap, Lalit, Daksh Shah +1 more

Standard machine learning algorithms have taken shape well due to increase in computational power. There is a special focus on reinforcement learning, which offers solutions to concepts like autonomous vehicles and robotics, among others. On the other hand, research in the fields of quantum computing and associated machine learning has catapulted in the last few years. This work analytically compares and contrasts between the classical and quantum approaches to two reinforcement learning problems, and tries to put focus on future works to advance both fields further.

International Conference for Emerging Technology (INCET)•May 21, 2021•5 citations

Control of Qubit Dynamics Using Reinforcement Learning

Dimitris Koutromanos, Dionisis Stefanatos, Emmanuel Paspalakis

The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.

Information•May 11, 2024•13 citations

Efficient Dimensionality Reduction Strategies for Quantum Reinforcement Learning

Eva Andrés, Manuel Pegalájar Cuéllar, Gabriel Navarro

Quantum neural networks constitute one of the most promising applications of Quantum Machine Learning, as they leverage both the capabilities of classical neural networks and the unique advantages of quantum mechanics. Moreover, quantum mechanics has demonstrated its ability to detect atypical patterns in data that are challenging for classical approaches to recognize. However, despite their potential, there are still open questions such as barren plateau phenomenon and the challenges of scalability and the curse of dimensionality, which become particularly relevant in Reinforcement Learning (RL) when working in environments with high-dimensional state and action spaces. This study delves into the critical realm of representing classical data as quantum states, a topic of keen interest across the scientific community. The aim is to construct streamlined circuits for efficient execution on quantum computers and simulators using minimal qubits and entanglement gates to evade barren plateau phenomena and reducing computational times. Our investigation examines and validates the efficacy of three strategies for data management and dimensionality reduction in real-world, large-scale environments for Quantum Reinforcement Learning, particularly in energy efficiency scenarios. The techniques encompass amplitude encoding, linear layer preprocessing, and data reuploading, supplemented by trainable parameters. This research sheds light on the potential of quantum machine learning in enhancing real-world environments, including energy efficiency scenarios and showcases the capabilities of quantum neural networks in the reinforcement learning landscape.

IEEE Access•Jan 1, 2023•27 citations

Control of Continuous Quantum Systems with Many Degrees of Freedom based on Convergent Reinforcement Learning

智康 王

With advances in digital technology in recent years, parallel computation utilizing GPUs has achieved remarkable efficiency, which has made it possible to use large-scale machine learning algorithms with large datasets, and deep learning, which is machine learning utilizing deep neural networks trained with millions to billions of data and parameters, has become increasingly popular and found its use for various tasks including facial recognition, language translation and combinatorial problems, etc., achieving considerably many record-breaking results.As a powerful generic tool to solve problems, deep learning has been applied to physical problems as well, and has become an important alternative tool for scientists to solve a number of important problems.With the development of experimental quantum technology over the past several decades, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems, which have opened up a new regime of complex quantum systems that can be engineered.Quantum control has found its use in controlled quantum-chemical processes and artificial quantum systems including quantum dots, superconducting qubits, trapped ions, and cavity optomechanical systems, etc., which are of considerable importance for future technology as candidates for sensors and quantum-computational devices.However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and generic numerical algorithms which search for proper control protocols are adopted.Therefore, deep learning, especially deep reinforcement learning, is a promising generic candidate solution for the control problems.Although there have already been a few successful examples of applications of deep reinforcement learning to quantum control problems, most of the existing reinforcement learning algorithms intrinsically suffer from instabilities and unsatisfactory reproducibility, and as a consequence, they typically require a large amount of fine-tuning and a large computational budget, both of which limit their applicability to quantum control problems and require expertise in machine learning.To resolve the issue of instabilities of the reinforcement learning algorithms, in this thesis, we first investigate the non-convergence issue of Q-learning, which is one of the most efficient reinforcement learning strategies.Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm.We prove the convergence of the C-DQN algorithm, and since the algorithm is scalable and computationally efficient, we apply it to a standard reinforcement learning benchmark, the Atari 2600 benchmark, to demonstrate its effectiveness.We show that when the DQN algorithm fail, the C-DQN algorithm still learns successfully.Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum-mechanical quartic oscillator i

•Mar 23, 2023•0 citations
In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.
The advantage of quantum computers over classical computers fuels the recent trend of developing machine learning algorithms on quantum computers, which can potentially lead to breakthroughs and new learning models in this area. The aim of our study is to explore deep quantum reinforcement learning (RL) on photonic quantum computers, which can process information stored in the quantum states of light. These quantum computers can naturally represent continuous variables, making them an ideal platform to create quantum versions of neural networks. Using quantum photonic circuits, we implement Q learning and actor-critic algorithms with multilayer quantum neural networks and test them in the grid world environment. Our experiments show that 1) these quantum algorithms can solve the RL problem and 2) compared to one layer, using three layer quantum networks improves the learning of both algorithms in terms of rewards collected. In summary, our findings suggest that having more layers in deep quantum RL can enhance the learning outcome.
In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.
Standard machine learning algorithms have taken shape well due to increase in computational power. There is a special focus on reinforcement learning, which offers solutions to concepts like autonomous vehicles and robotics, among others. On the other hand, research in the fields of quantum computing and associated machine learning has catapulted in the last few years. This work analytically compares and contrasts between the classical and quantum approaches to two reinforcement learning problems, and tries to put focus on future works to advance both fields further.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.
Quantum neural networks constitute one of the most promising applications of Quantum Machine Learning, as they leverage both the capabilities of classical neural networks and the unique advantages of quantum mechanics. Moreover, quantum mechanics has demonstrated its ability to detect atypical patterns in data that are challenging for classical approaches to recognize. However, despite their potential, there are still open questions such as barren plateau phenomenon and the challenges of scalability and the curse of dimensionality, which become particularly relevant in Reinforcement Learning (RL) when working in environments with high-dimensional state and action spaces. This study delves into the critical realm of representing classical data as quantum states, a topic of keen interest across the scientific community. The aim is to construct streamlined circuits for efficient execution on quantum computers and simulators using minimal qubits and entanglement gates to evade barren plateau phenomena and reducing computational times. Our investigation examines and validates the efficacy of three strategies for data management and dimensionality reduction in real-world, large-scale environments for Quantum Reinforcement Learning, particularly in energy efficiency scenarios. The techniques encompass amplitude encoding, linear layer preprocessing, and data reuploading, supplemented by trainable parameters. This research sheds light on the potential of quantum machine learning in enhancing real-world environments, including energy efficiency scenarios and showcases the capabilities of quantum neural networks in the reinforcement learning landscape.
With advances in digital technology in recent years, parallel computation utilizing GPUs has achieved remarkable efficiency, which has made it possible to use large-scale machine learning algorithms with large datasets, and deep learning, which is machine learning utilizing deep neural networks trained with millions to billions of data and parameters, has become increasingly popular and found its use for various tasks including facial recognition, language translation and combinatorial problems, etc., achieving considerably many record-breaking results.As a powerful generic tool to solve problems, deep learning has been applied to physical problems as well, and has become an important alternative tool for scientists to solve a number of important problems.With the development of experimental quantum technology over the past several decades, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems, which have opened up a new regime of complex quantum systems that can be engineered.Quantum control has found its use in controlled quantum-chemical processes and artificial quantum systems including quantum dots, superconducting qubits, trapped ions, and cavity optomechanical systems, etc., which are of considerable importance for future technology as candidates for sensors and quantum-computational devices.However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and generic numerical algorithms which search for proper control protocols are adopted.Therefore, deep learning, especially deep reinforcement learning, is a promising generic candidate solution for the control problems.Although there have already been a few successful examples of applications of deep reinforcement learning to quantum control problems, most of the existing reinforcement learning algorithms intrinsically suffer from instabilities and unsatisfactory reproducibility, and as a consequence, they typically require a large amount of fine-tuning and a large computational budget, both of which limit their applicability to quantum control problems and require expertise in machine learning.To resolve the issue of instabilities of the reinforcement learning algorithms, in this thesis, we first investigate the non-convergence issue of Q-learning, which is one of the most efficient reinforcement learning strategies.Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm.We prove the convergence of the C-DQN algorithm, and since the algorithm is scalable and computationally efficient, we apply it to a standard reinforcement learning benchmark, the Atari 2600 benchmark, to demonstrate its effectiveness.We show that when the DQN algorithm fail, the C-DQN algorithm still learns successfully.Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum-mechanical quartic oscillator i