Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Conference on Communications

Efficient Quantum Soft Actor-Critic Model for Dynamic Spectrum Sharing in Intelligent O-RAN

Vu-Hai Nguyen, Yared Abera Ergu, Ren‐Hung Hwang, Trung Q. Duong, Van-Linh Nguyen

Published May 24, 2026• 0 citations
Publisher

Abstract

The Open Radio Access Network (O-RAN) architecture enables intelligent Dynamic Spectrum Sharing (DSS), although state-of-the-art deep reinforcement learning (DRL) methods have demonstrated remarkable practical results, their integration with quantum computing brings greater potential for enhancements in performance. In this study, we present Quantum Soft Actor-Critic (QSAC), a novel hybrid quantum-classical methodology aimed at tackling barren plateaus and demonstrating enhanced performance compared to conventional DRL benchmarks, including Soft Actor-Critic (SAC) and Twin Delayed Deep Deterministic Policy Gradient (TD3), achieving a higher demand satisfaction ratio and markedly improved ϵ-band compliance rates under demanding conditions. Additionally, we present a comprehensive architectural design for the effective integration of our framework into the O-RAN non-real-time and near-real-time RICs. Our quantum-enhanced actor significantly reduces the number of trainable actor parameters by over 99% compared to its conventional SAC equivalent (from around 73,000 to merely 377), indicating an important improvement in model efficiency. To promote repeatability and facilitate future research, we offer the open-source implementation of our study.

DOI
10.1109/icc59461.2026.11587335

Related papers

Variational Quantum Soft Actor-Critic

Qingfeng Lan

Quantum computing has a superior advantage in tackling specific problems, such as integer factorization and Simon's problem. For more general tasks in machine learning, by applying variational quantum circuits, more and more quantum algorithms have been proposed recently, especially in supervised learning and unsupervised learning. However, little work has been done in reinforcement learning, arguably more important and challenging. Previous work in quantum reinforcement learning mainly focuses on discrete control tasks where the action space is discrete. In this work, we develop a quantum reinforcement learning algorithm based on soft actor-critic -- one of the state-of-the-art methods for continuous control. Specifically, we use a hybrid quantum-classical policy network consisting of a variational quantum circuit and a classical artificial neural network. Tested in a standard reinforcement learning benchmark, we show that this quantum version of soft actor-critic is comparable with the original soft actor-critic, using much less adjustable parameters. Furthermore, we analyze the effect of different hyper-parameters and policy network architectures, pointing out the importance of architecture design for quantum reinforcement learning.

arXiv:2112.11921•Dec 20, 2021•39 citations

Quantum-Enhanced DRL Optimization for DoA Estimation and Task Offloading in ISAC Systems

Anal Paul, Keshav Kumar Singh, Aryan Kaushik +4 more

This work proposes a quantum-aided deep reinforcement learning (DRL) framework designed to enhance the accuracy of direction-of-arrival (DoA) estimation and the efficiency of computational task offloading in integrated sensing and communication systems. Traditional DRL approaches face challenges in handling high-dimensional state spaces and ensuring convergence to optimal policies within complex operational environments. The proposed quantum-aided DRL framework that operates in a military surveillance system exploits quantum computing’s parallel processing capabilities to encode operational states and actions into quantum states, significantly reducing the dimensionality of the decision space. For the very first time in literature, we propose a quantum-enhanced actor-critic method, utilizing quantum circuits for policy representation and optimization. Through comprehensive simulations, we demonstrate that our framework improves DoA estimation accuracy by 91.66% and 82.61% over existing DRL algorithms with faster convergence rate, and effectively manages the trade-off between sensing and communication and by optimizing task offloading decisions under stringent ultra-reliable low-latency communication requirements. Comparative analysis also reveals that our approach reduces the overall task offloading latency by 43.09% and 32.35% compared to the DRL-based deep deterministic policy gradient and proximal policy optimization algorithms, respectively.

IEEE Journal on Selected Areas in Communications•Sep 13, 2024•40 citations

Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization

Haftu W. Fentaw, Steve Campbell, Simon Caton

We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.

quant-pharXiv:2605.26925•May 27, 2026•0 citations

A Sample-Efficient Quantum-Classical Reinforcement Learning Framework for Transmission Switching

Lin Ding, Jianhui Wang, Huan-Hsin Tseng +3 more

This paper presents a sample-efficient hybrid quantum-classical reinforcement learning (QCRL) framework for multi-objective transmission switching (MO-TS) in power systems. We introduce a Chunk-Based Data Reuploading Variational Quantum Circuit (CBDR-VQC) architecture, which is designed to enhance policy expressivity without increasing circuit depth or width. Unlike conventional VQC approaches that scale circuit size to improve learning capacity, often at the cost of hardware feasibility and trainability, our method encodes high-dimensional input features sequentially into a fixed-width, shallow quantum circuit with a learnable measurement scheme. This design maintains compatibility with near-term quantum devices while avoiding issues such as noise accumulation and barren plateaus. Embedded within a quantum-classical Soft Actor-Critic (QCSAC) framework, the proposed model achieves competitive performance for MO-TS on the IEEE 118-bus system. It significantly reduces the numbers of both required qubits and trainable parameters compared to both classical and wider VQC-based baselines, demonstrating its promise for resourceconstrained QCRL in real-world power system optimization.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Aug 30, 2025•1 citation

Quantum deep reinforcement learning for humanoid robot navigation task

Romerik Lokossou, Birhanu Shimelis Girma, O.K. Tonguz +1 more

Classical reinforcement learning (RL) methods often struggle in complex, high-dimensional environments because of their extensive parameter requirements and challenges posed by stochastic, non-deterministic settings. This study introduces quantum deep reinforcement learning (QDRL) to train humanoid agents efficiently. While previous quantum RL models focused on smaller environments, such as wheeled robots and robotic arms, our work pioneers the application of QDRL to humanoid robotics, specifically in environments with substantial observation and action spaces, such as MuJoCo's Humanoid-v4 and Walker2d-v4. Using parameterized quantum circuits, we explored a hybrid quantum-classical setup to directly navigate high-dimensional state spaces, bypassing traditional mapping and planning. By integrating quantum computing with deep RL, we aim to develop models that can efficiently learn complex navigation tasks in humanoid robots. We evaluated the performance of the Soft Actor-Critic (SAC) in classical RL against its quantum implementation. The results show that the quantum SAC achieves an 8% higher average return (246.40) than the classical SAC (228.36) after 92% fewer steps, highlighting the accelerated learning potential of quantum computing in RL tasks.

arXiv:2509.11388•Sep 14, 2025•3 citations

Securing SIM-Assisted Wireless Networks via Quantum Reinforcement Learning

Le-Hung Hoang, Quang-Trung Luu, Dinh Thai Hoang +2 more

Stacked intelligent metasurfaces (SIMs) have recently emerged as a powerful wave-domain technology that enables multi-stage manipulation of electromagnetic signals through multilayer programmable architectures. While SIMs offer unprecedented degrees of freedom for enhancing physical-layer security, their extremely large number of meta-atoms leads to a high-dimensional and strongly coupled optimization space, making conventional design approaches inefficient and difficult to scale. Moreover, existing deep reinforcement learning (DRL) techniques suffer from slow convergence and performance degradation in dynamic wireless environments with imperfect knowledge of passive eavesdroppers. To address these challenges, we propose a hybrid quantum proximal policy optimization (QPPO) framework for SIM-assisted secure communications that jointly optimizes transmit power allocation and SIM phase shifts to maximize the average secrecy rate under power and quality-of-service constraints. Specifically, a parameterized quantum circuit is embedded into the actor network, forming a hybrid classical-quantum policy architecture that enhances policy representation capability and exploration efficiency in high-dimensional continuous action spaces. Extensive simulations demonstrate that the proposed Q-PPO scheme consistently outperforms DRL baselines, achieving approximately 15% higher secrecy rates and 30% faster convergence under imperfect eavesdropper channel state information. These results establish Q-PPO as a powerful optimization paradigm for SIM-enabled secure wireless networks.

arXiv:2602.13238•Jan 29, 2026•0 citations
The Open Radio Access Network (O-RAN) architecture enables intelligent Dynamic Spectrum Sharing (DSS), although state-of-the-art deep reinforcement learning (DRL) methods have demonstrated remarkable practical results, their integration with quantum computing brings greater potential for enhancements in performance. In this study, we present Quantum Soft Actor-Critic (QSAC), a novel hybrid quantum-classical methodology aimed at tackling barren plateaus and demonstrating enhanced performance compared to conventional DRL benchmarks, including Soft Actor-Critic (SAC) and Twin Delayed Deep Deterministic Policy Gradient (TD3), achieving a higher demand satisfaction ratio and markedly improved ϵ-band compliance rates under demanding conditions. Additionally, we present a comprehensive architectural design for the effective integration of our framework into the O-RAN non-real-time and near-real-time RICs. Our quantum-enhanced actor significantly reduces the number of trainable actor parameters by over 99% compared to its conventional SAC equivalent (from around 73,000 to merely 377), indicating an important improvement in model efficiency. To promote repeatability and facilitate future research, we offer the open-source implementation of our study.
Quantum computing has a superior advantage in tackling specific problems, such as integer factorization and Simon's problem. For more general tasks in machine learning, by applying variational quantum circuits, more and more quantum algorithms have been proposed recently, especially in supervised learning and unsupervised learning. However, little work has been done in reinforcement learning, arguably more important and challenging. Previous work in quantum reinforcement learning mainly focuses on discrete control tasks where the action space is discrete. In this work, we develop a quantum reinforcement learning algorithm based on soft actor-critic -- one of the state-of-the-art methods for continuous control. Specifically, we use a hybrid quantum-classical policy network consisting of a variational quantum circuit and a classical artificial neural network. Tested in a standard reinforcement learning benchmark, we show that this quantum version of soft actor-critic is comparable with the original soft actor-critic, using much less adjustable parameters. Furthermore, we analyze the effect of different hyper-parameters and policy network architectures, pointing out the importance of architecture design for quantum reinforcement learning.
This work proposes a quantum-aided deep reinforcement learning (DRL) framework designed to enhance the accuracy of direction-of-arrival (DoA) estimation and the efficiency of computational task offloading in integrated sensing and communication systems. Traditional DRL approaches face challenges in handling high-dimensional state spaces and ensuring convergence to optimal policies within complex operational environments. The proposed quantum-aided DRL framework that operates in a military surveillance system exploits quantum computing’s parallel processing capabilities to encode operational states and actions into quantum states, significantly reducing the dimensionality of the decision space. For the very first time in literature, we propose a quantum-enhanced actor-critic method, utilizing quantum circuits for policy representation and optimization. Through comprehensive simulations, we demonstrate that our framework improves DoA estimation accuracy by 91.66% and 82.61% over existing DRL algorithms with faster convergence rate, and effectively manages the trade-off between sensing and communication and by optimizing task offloading decisions under stringent ultra-reliable low-latency communication requirements. Comparative analysis also reveals that our approach reduces the overall task offloading latency by 43.09% and 32.35% compared to the DRL-based deep deterministic policy gradient and proximal policy optimization algorithms, respectively.
We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.
This paper presents a sample-efficient hybrid quantum-classical reinforcement learning (QCRL) framework for multi-objective transmission switching (MO-TS) in power systems. We introduce a Chunk-Based Data Reuploading Variational Quantum Circuit (CBDR-VQC) architecture, which is designed to enhance policy expressivity without increasing circuit depth or width. Unlike conventional VQC approaches that scale circuit size to improve learning capacity, often at the cost of hardware feasibility and trainability, our method encodes high-dimensional input features sequentially into a fixed-width, shallow quantum circuit with a learnable measurement scheme. This design maintains compatibility with near-term quantum devices while avoiding issues such as noise accumulation and barren plateaus. Embedded within a quantum-classical Soft Actor-Critic (QCSAC) framework, the proposed model achieves competitive performance for MO-TS on the IEEE 118-bus system. It significantly reduces the numbers of both required qubits and trainable parameters compared to both classical and wider VQC-based baselines, demonstrating its promise for resourceconstrained QCRL in real-world power system optimization.
Classical reinforcement learning (RL) methods often struggle in complex, high-dimensional environments because of their extensive parameter requirements and challenges posed by stochastic, non-deterministic settings. This study introduces quantum deep reinforcement learning (QDRL) to train humanoid agents efficiently. While previous quantum RL models focused on smaller environments, such as wheeled robots and robotic arms, our work pioneers the application of QDRL to humanoid robotics, specifically in environments with substantial observation and action spaces, such as MuJoCo's Humanoid-v4 and Walker2d-v4. Using parameterized quantum circuits, we explored a hybrid quantum-classical setup to directly navigate high-dimensional state spaces, bypassing traditional mapping and planning. By integrating quantum computing with deep RL, we aim to develop models that can efficiently learn complex navigation tasks in humanoid robots. We evaluated the performance of the Soft Actor-Critic (SAC) in classical RL against its quantum implementation. The results show that the quantum SAC achieves an 8% higher average return (246.40) than the classical SAC (228.36) after 92% fewer steps, highlighting the accelerated learning potential of quantum computing in RL tasks.
Stacked intelligent metasurfaces (SIMs) have recently emerged as a powerful wave-domain technology that enables multi-stage manipulation of electromagnetic signals through multilayer programmable architectures. While SIMs offer unprecedented degrees of freedom for enhancing physical-layer security, their extremely large number of meta-atoms leads to a high-dimensional and strongly coupled optimization space, making conventional design approaches inefficient and difficult to scale. Moreover, existing deep reinforcement learning (DRL) techniques suffer from slow convergence and performance degradation in dynamic wireless environments with imperfect knowledge of passive eavesdroppers. To address these challenges, we propose a hybrid quantum proximal policy optimization (QPPO) framework for SIM-assisted secure communications that jointly optimizes transmit power allocation and SIM phase shifts to maximize the average secrecy rate under power and quality-of-service constraints. Specifically, a parameterized quantum circuit is embedded into the actor network, forming a hybrid classical-quantum policy architecture that enhances policy representation capability and exploration efficiency in high-dimensional continuous action spaces. Extensive simulations demonstrate that the proposed Q-PPO scheme consistently outperforms DRL baselines, achieving approximately 15% higher secrecy rates and 30% faster convergence under imperfect eavesdropper channel state information. These results establish Q-PPO as a powerful optimization paradigm for SIM-enabled secure wireless networks.