Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE Journal on Selected Areas in Communications

Quantum-Enhanced DRL Optimization for DoA Estimation and Task Offloading in ISAC Systems

Anal Paul, Keshav Kumar Singh, Aryan Kaushik, Chih–Peng Li, Octavia A. Dobre, Marco Di Renzo, Trung Q. Duong

Published Sep 13, 2024• 40 citations
Publisher

Abstract

This work proposes a quantum-aided deep reinforcement learning (DRL) framework designed to enhance the accuracy of direction-of-arrival (DoA) estimation and the efficiency of computational task offloading in integrated sensing and communication systems. Traditional DRL approaches face challenges in handling high-dimensional state spaces and ensuring convergence to optimal policies within complex operational environments. The proposed quantum-aided DRL framework that operates in a military surveillance system exploits quantum computing’s parallel processing capabilities to encode operational states and actions into quantum states, significantly reducing the dimensionality of the decision space. For the very first time in literature, we propose a quantum-enhanced actor-critic method, utilizing quantum circuits for policy representation and optimization. Through comprehensive simulations, we demonstrate that our framework improves DoA estimation accuracy by 91.66% and 82.61% over existing DRL algorithms with faster convergence rate, and effectively manages the trade-off between sensing and communication and by optimizing task offloading decisions under stringent ultra-reliable low-latency communication requirements. Comparative analysis also reveals that our approach reduces the overall task offloading latency by 43.09% and 32.35% compared to the DRL-based deep deterministic policy gradient and proximal policy optimization algorithms, respectively.

DOI
10.1109/jsac.2024.3460061

Related papers

Survey Paper on Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Shruti Doiphode, Prof. Mukul Jagtap

Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation

International Journal for Research in Applied Science and Engineering Technology•Oct 18, 2024•0 citations

Quantum Deep Reinforcement Learning for 6G Mobile Edge Computing-based IoT Systems

James Adu Ansere, Trung Q. Duong, Saeed R. Khosravirad +3 more

This paper exploits a quantum-empowered machine learning algorithm to enhance computation learning speed. Under stochastic behaviours and quantum uncertainty, we examine the offloading problem to maximize the computational task processing efficiency, considering the computation latency, energy consumption, and quantum network adaptability. From the Markov decision process, the paper proposes a novel quantumempowered deep reinforcement learning (Qe-DRL) approach, combining quantum computing theory and machine learning to achieve exploration and exploitation trade-off via quantum parallelism significantly. Furthermore, we develop a modified Grover’s algorithm with exponential convergence speed to provide a searching strategy for transition quantum states probabilities. Simulation results establish the effectiveness of the proposed QeDRL algorithm and its superior computational learning speed.

International Wireless Communications and Mobile Computing Conference (IWCMC)•Jun 19, 2023•16 citations

Quantum Deep Reinforcement Learning for Dynamic Resource Allocation in Mobile Edge Computing-Based IoT Systems

James Adu Ansere, Eric Gyamfi, Vishal Sharma +3 more

This paper exploits a quantum-empowered machine learning algorithm to enhance computation learning speed. We leverage quantum phenomena such as superposition and entanglement to work on large-scale multi-dimensional data represented by quantum states. Under stochastic behaviors and quantum uncertainty, we examine the offloading problem to maximize the computational task processing efficiency, considering the computation latency, energy consumption, and quantum network adaptability. From the Markov decision process, the paper proposes a novel quantum-empowered deep reinforcement learning (Qe-DRL) approach, combining quantum computing theory and machine learning to achieve exploration and exploitation trade-off via quantum parallelism significantly. Furthermore, we develop a modified Grover’s algorithm with exponential convergence speed to provide a searching strategy for transition quantum states probabilities. Simulation results establish the effectiveness of the proposed Qe-DRL algorithm and its superior computational learning speed. Our proposed Qe-DRL algorithm outperforms other benchmarks in terms of energy efficiency performance.

IEEE Transactions on Wireless Communications•Nov 14, 2023•59 citations

Efficient Quantum Soft Actor-Critic Model for Dynamic Spectrum Sharing in Intelligent O-RAN

Vu-Hai Nguyen, Yared Abera Ergu, Ren‐Hung Hwang +2 more

The Open Radio Access Network (O-RAN) architecture enables intelligent Dynamic Spectrum Sharing (DSS), although state-of-the-art deep reinforcement learning (DRL) methods have demonstrated remarkable practical results, their integration with quantum computing brings greater potential for enhancements in performance. In this study, we present Quantum Soft Actor-Critic (QSAC), a novel hybrid quantum-classical methodology aimed at tackling barren plateaus and demonstrating enhanced performance compared to conventional DRL benchmarks, including Soft Actor-Critic (SAC) and Twin Delayed Deep Deterministic Policy Gradient (TD3), achieving a higher demand satisfaction ratio and markedly improved ϵ-band compliance rates under demanding conditions. Additionally, we present a comprehensive architectural design for the effective integration of our framework into the O-RAN non-real-time and near-real-time RICs. Our quantum-enhanced actor significantly reduces the number of trainable actor parameters by over 99% compared to its conventional SAC equivalent (from around 73,000 to merely 377), indicating an important improvement in model efficiency. To promote repeatability and facilitate future research, we offer the open-source implementation of our study.

International Conference on Communications•May 24, 2026•0 citations

Quantum Architecture Search via Deep Reinforcement Learning

En-Jui Kuo, Yao-Lung L. Fang, Samuel Yen-Chi Chen

Recent advances in quantum computing have drawn considerable attention to building realistic application for and using quantum computers. However, designing a suitable quantum circuit architecture requires expert knowledge. For example, it is non-trivial to design a quantum gate sequence for generating a particular quantum state with as fewer gates as possible. We propose a quantum architecture search framework with the power of deep reinforcement learning (DRL) to address this challenge. In the proposed framework, the DRL agent can only access the Pauli-$X$, $Y$, $Z$ expectation values and a predefined set of quantum operations for learning the target quantum state, and is optimized by the advantage actor-critic (A2C) and proximal policy optimization (PPO) algorithms. We demonstrate a successful generation of quantum gate sequences for multi-qubit GHZ states without encoding any knowledge of quantum physics in the agent. The design of our framework is rather general and can be employed with other DRL architectures or optimization methods to study gate synthesis and compilation for many quantum states.

arXiv:2104.07715•Apr 15, 2021•144 citations

Quantum-Enhanced Reinforcement Learning for Adaptive Decision-Making in Dynamic Environments

Priyanka Singh, Sonam Dubey

Many challenges that occur with traditional reinforcement learning in uncertain environments, such as slow convergence and scalability, can be addressed with Quantum-Enhanced Reinforcement Learning (QERL). Specifically, this research is for adaptive decision-making in environments with rapid change and great uncertainty. A QERL framework that combines variational quantum circuits and deep Q-networks is introduced. This framework optimizes decision-making processes in real-time and is applicable in areas such as autonomous systems, smart manufacturing, and intelligent resource allocation. Training for the framework took place in simulated dynamic environments during 50,000 training episodes and was then evaluated alongside Deep Q-Learning and Proximal Policy Optimization models. The QERL self-model outperformed the baseline in policy accuracy (+18.6%) and cumulative reward loss (-21.4%). Decision latency decreased 16.2%, and convergence improved 23.8% with the use of a hybrid quantum architecture. The proposed analysis demonstrated improved average reward (78.4 to 94.7) with 95% CI, indicating model consistency. Ultimately, QERL improves decision-making in complex scenarios and provides robust adaptive learning. Also noted, QERL frameworks with hybrid quantum architecture are useful for real-world systems. Furthermore, the learning framework proved to be more stable and adaptable to rapid and stochastic environmental changes compared to classical agents, in environments with incomplete dynamic and rapidly changing environments.

•Aug 1, 2026•0 citations
This work proposes a quantum-aided deep reinforcement learning (DRL) framework designed to enhance the accuracy of direction-of-arrival (DoA) estimation and the efficiency of computational task offloading in integrated sensing and communication systems. Traditional DRL approaches face challenges in handling high-dimensional state spaces and ensuring convergence to optimal policies within complex operational environments. The proposed quantum-aided DRL framework that operates in a military surveillance system exploits quantum computing’s parallel processing capabilities to encode operational states and actions into quantum states, significantly reducing the dimensionality of the decision space. For the very first time in literature, we propose a quantum-enhanced actor-critic method, utilizing quantum circuits for policy representation and optimization. Through comprehensive simulations, we demonstrate that our framework improves DoA estimation accuracy by 91.66% and 82.61% over existing DRL algorithms with faster convergence rate, and effectively manages the trade-off between sensing and communication and by optimizing task offloading decisions under stringent ultra-reliable low-latency communication requirements. Comparative analysis also reveals that our approach reduces the overall task offloading latency by 43.09% and 32.35% compared to the DRL-based deep deterministic policy gradient and proximal policy optimization algorithms, respectively.
Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation
This paper exploits a quantum-empowered machine learning algorithm to enhance computation learning speed. Under stochastic behaviours and quantum uncertainty, we examine the offloading problem to maximize the computational task processing efficiency, considering the computation latency, energy consumption, and quantum network adaptability. From the Markov decision process, the paper proposes a novel quantumempowered deep reinforcement learning (Qe-DRL) approach, combining quantum computing theory and machine learning to achieve exploration and exploitation trade-off via quantum parallelism significantly. Furthermore, we develop a modified Grover’s algorithm with exponential convergence speed to provide a searching strategy for transition quantum states probabilities. Simulation results establish the effectiveness of the proposed QeDRL algorithm and its superior computational learning speed.
This paper exploits a quantum-empowered machine learning algorithm to enhance computation learning speed. We leverage quantum phenomena such as superposition and entanglement to work on large-scale multi-dimensional data represented by quantum states. Under stochastic behaviors and quantum uncertainty, we examine the offloading problem to maximize the computational task processing efficiency, considering the computation latency, energy consumption, and quantum network adaptability. From the Markov decision process, the paper proposes a novel quantum-empowered deep reinforcement learning (Qe-DRL) approach, combining quantum computing theory and machine learning to achieve exploration and exploitation trade-off via quantum parallelism significantly. Furthermore, we develop a modified Grover’s algorithm with exponential convergence speed to provide a searching strategy for transition quantum states probabilities. Simulation results establish the effectiveness of the proposed Qe-DRL algorithm and its superior computational learning speed. Our proposed Qe-DRL algorithm outperforms other benchmarks in terms of energy efficiency performance.
The Open Radio Access Network (O-RAN) architecture enables intelligent Dynamic Spectrum Sharing (DSS), although state-of-the-art deep reinforcement learning (DRL) methods have demonstrated remarkable practical results, their integration with quantum computing brings greater potential for enhancements in performance. In this study, we present Quantum Soft Actor-Critic (QSAC), a novel hybrid quantum-classical methodology aimed at tackling barren plateaus and demonstrating enhanced performance compared to conventional DRL benchmarks, including Soft Actor-Critic (SAC) and Twin Delayed Deep Deterministic Policy Gradient (TD3), achieving a higher demand satisfaction ratio and markedly improved ϵ-band compliance rates under demanding conditions. Additionally, we present a comprehensive architectural design for the effective integration of our framework into the O-RAN non-real-time and near-real-time RICs. Our quantum-enhanced actor significantly reduces the number of trainable actor parameters by over 99% compared to its conventional SAC equivalent (from around 73,000 to merely 377), indicating an important improvement in model efficiency. To promote repeatability and facilitate future research, we offer the open-source implementation of our study.
Recent advances in quantum computing have drawn considerable attention to building realistic application for and using quantum computers. However, designing a suitable quantum circuit architecture requires expert knowledge. For example, it is non-trivial to design a quantum gate sequence for generating a particular quantum state with as fewer gates as possible. We propose a quantum architecture search framework with the power of deep reinforcement learning (DRL) to address this challenge. In the proposed framework, the DRL agent can only access the Pauli-XXX, YYY, ZZZ expectation values and a predefined set of quantum operations for learning the target quantum state, and is optimized by the advantage actor-critic (A2C) and proximal policy optimization (PPO) algorithms. We demonstrate a successful generation of quantum gate sequences for multi-qubit GHZ states without encoding any knowledge of quantum physics in the agent. The design of our framework is rather general and can be employed with other DRL architectures or optimization methods to study gate synthesis and compilation for many quantum states.
Many challenges that occur with traditional reinforcement learning in uncertain environments, such as slow convergence and scalability, can be addressed with Quantum-Enhanced Reinforcement Learning (QERL). Specifically, this research is for adaptive decision-making in environments with rapid change and great uncertainty. A QERL framework that combines variational quantum circuits and deep Q-networks is introduced. This framework optimizes decision-making processes in real-time and is applicable in areas such as autonomous systems, smart manufacturing, and intelligent resource allocation. Training for the framework took place in simulated dynamic environments during 50,000 training episodes and was then evaluated alongside Deep Q-Learning and Proximal Policy Optimization models. The QERL self-model outperformed the baseline in policy accuracy (+18.6%) and cumulative reward loss (-21.4%). Decision latency decreased 16.2%, and convergence improved 23.8% with the use of a hybrid quantum architecture. The proposed analysis demonstrated improved average reward (78.4 to 94.7) with 95% CI, indicating model consistency. Ultimately, QERL improves decision-making in complex scenarios and provides robust adaptive learning. Also noted, QERL frameworks with hybrid quantum architecture are useful for real-world systems. Furthermore, the learning framework proved to be more stable and adaptable to rapid and stochastic environmental changes compared to classical agents, in environments with incomplete dynamic and rapidly changing environments.