Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE International Conference on Quantum Computing and Engineering (QCE)

A Sample-Efficient Quantum-Classical Reinforcement Learning Framework for Transmission Switching

Lin Ding, Jianhui Wang, Huan-Hsin Tseng, Shinjae Yoo, Tianqiao Zhao, Meng Yue

Published Aug 30, 2025• 1 citation
Publisher

Abstract

This paper presents a sample-efficient hybrid quantum-classical reinforcement learning (QCRL) framework for multi-objective transmission switching (MO-TS) in power systems. We introduce a Chunk-Based Data Reuploading Variational Quantum Circuit (CBDR-VQC) architecture, which is designed to enhance policy expressivity without increasing circuit depth or width. Unlike conventional VQC approaches that scale circuit size to improve learning capacity, often at the cost of hardware feasibility and trainability, our method encodes high-dimensional input features sequentially into a fixed-width, shallow quantum circuit with a learnable measurement scheme. This design maintains compatibility with near-term quantum devices while avoiding issues such as noise accumulation and barren plateaus. Embedded within a quantum-classical Soft Actor-Critic (QCSAC) framework, the proposed model achieves competitive performance for MO-TS on the IEEE 118-bus system. It significantly reduces the numbers of both required qubits and trainable parameters compared to both classical and wider VQC-based baselines, demonstrating its promise for resourceconstrained QCRL in real-world power system optimization.

DOI
10.1109/qce65121.2025.10319

Related papers

Variational quantum reinforcement learning via evolutionary optimization

Samuel Yen-Chi Chen, Chih-Min Huang, Chia-Wei Hsing +2 more

Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

arXiv:2109.00540•Machine Learning Science and Technology•Dec 21, 2021•121 citations

A Study on Optimization Techniques for Variational Quantum Circuits in Reinforcement Learning

Michael Kölle, Timo Witter, Tobias Rohe +3 more

Quantum Computing aims to streamline machine learning, making it more effective with fewer trainable parameters. This reduction of parameters can speed up the learning process and reduce the use of computational resources. However, in the current phase of quantum computing development, known as the noisy intermediate-scale quantum era (NISQ), learning is difficult due to a limited number of qubits and widespread quantum noise. To overcome these challenges, researchers are focusing on variational quantum circuits (VQCs). VQCs are hybrid algorithms that merge a quantum circuit, which can be adjusted through parameters, with traditional classical optimization techniques. These circuits require only few qubits for effective learning. Recent studies have presented new ways of applying VQCs to reinforcement learning, showing promising results that warrant further exploration. This study investigates the effects of various techniques — data re-uploading, input scaling, output scaling — and introduces exponential learning rate decay in the quantum proximal policy optimization algorithm’s actor-VQC. We assess these methods in the popular Frozen Lake and Cart Pole environments. Our focus is on their ability to reduce the number of parameters in the VQC without losing effectiveness. Our findings indicate that data re-uploading and an exponential learning rate decay significantly enhance hyperparameter stability and overall performance. While input scaling does not improve parameter efficiency, output scaling effectively manages greediness, leading to increased learning speed and robustness.

arXiv:2405.12354•IEEE International Conference on Quantum Software (QSW)•Jul 7, 2024•14 citations

Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components

Javier Lazaro, Juan-Ignacio Vazquez, Pablo Garcia-Bringas

Parameterised quantum circuit (PQC) based Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning (RL). By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain due to training instabilities, barren plateaus (BPs), and the difficulty of isolating the contribution of individual pipeline components. In this work, we dissect PQC based QRL architectures through a systematic experimental evaluation of three aspects recurrently identified as critical: (i) data embedding strategies, with Data Reuploading (DR) as an advanced approach; (ii) ansatz design, particularly the role of entanglement; and (iii) post-processing blocks after quantum measurement, with a focus on the underexplored Output Reuse (OR) technique. Using a unified PPO-CartPole framework, we perform controlled comparisons between hybrid and classical agents under identical conditions. Our results show that OR, though purely classical, exhibits distinct behaviour in hybrid pipelines, that DR improves trainability and stability, and that stronger entanglement can degrade optimisation, offsetting classical gains. Together, these findings provide controlled empirical evidence of the interplay between quantum and classical contributions, and establish a reproducible framework for systematic benchmarking and component-wise analysis in QRL.

quant-pharXiv:2511.17112•Nov 24, 2025•1 citation

Hybrid Quantum-Classical Policy Gradient for Adaptive Control of Cyber-Physical Systems: A Comparative Study of VQC vs. MLP

Aueaphum Aueawatthanaphisut, Nyi Wunna Tun

The comparative evaluation between classical and quantum reinforcement learning (QRL) paradigms was conducted to investigate their convergence behavior, robustness under observational noise, and computational efficiency in a benchmark control environment. The study employed a multilayer perceptron (MLP) agent as a classical baseline and a parameterized variational quantum circuit (VQC) as a quantum counterpart, both trained on the CartPole-v1 environment over 500 episodes. Empirical results demonstrated that the classical MLP achieved near-optimal policy convergence with a mean return of 498.7 +/- 3.2, maintaining stable equilibrium throughout training. In contrast, the VQC exhibited limited learning capability, with an average return of 14.6 +/- 4.8, primarily constrained by circuit depth and qubit connectivity. Noise robustness analysis further revealed that the MLP policy deteriorated gracefully under Gaussian perturbations, while the VQC displayed higher sensitivity at equivalent noise levels. Despite the lower asymptotic performance, the VQC exhibited significantly lower parameter count and marginally increased training time, highlighting its potential scalability for low-resource quantum processors. The results suggest that while classical neural policies remain dominant in current control benchmarks, quantum-enhanced architectures could offer promising efficiency advantages once hardware noise and expressivity limitations are mitigated.

arXiv:2510.06010•Oct 7, 2025•0 citations

Quantum-Inspired Reinforcement Learning Controller for DC–DC Power Converters Supplying Constant Power Loads

Ali Masoudian, Uffe Jakobsen, Mohammad Hassan Khooban

This paper proposes a quantum neural network (QNN)-based reinforcement learning (RL) controller for the real-time control of DC–DC buck converters supplying constant power loads (CPLs). Unlike conventional RL approaches that rely on classical neural networks, the proposed method employs a parameterized quantum circuit to represent the control policy, enabling a compact and expressive function approximator. The QNN is implemented as a classically simulated variational quantum circuit; no physical quantum hardware or quantum computational speedup is claimed. It is trained offline using a deterministic policy gradient framework and subsequently deployed in a real-time control loop under strict sampling constraints. The proposed 36-parameter QNN actor reduces trainable parameters by 85% versus a computation-matched classical RL baseline (245 parameters), while achieving sub-millisecond settling times and near-zero steady-state error across reference tracking, load disturbance, and supply variation scenarios validated on a dSPACE DS1202 hardware platform at 100-kHz switching frequency. Small-signal stability analysis at three tested operating points confirms phase margins exceeding 30° and a gain margin that is theoretically infinite at those points, as the linearized phase does not cross −180°. The results highlight the ability of the QNN to capture complex system dynamics with fewer trainable parameters while maintaining stable closed-loop operation, providing, to the best of the authors’ knowledge, a first step toward integrating quantum machine learning concepts into real-time power electronics control.

IEEE Access•Jan 1, 2026•1 citation

Quantum Reinforcement Learning

Hamid D. Ismail

This chapter develops quantum reinforcement learning (QRL) as a principled extension of classical reinforcement learning (RL) into the quantum domain. Building on the standard Markov decision process framework, it shows how policies and value functions can be represented by quantum states and parameterized quantum circuits, with action probabilities arising from Born-rule measurement statistics and value estimates obtained from expectation values of observables. In this formulation, RL is preserved at the algorithmic level while its representational substrate is shifted from classical vector spaces to Hilbert space. The chapter emphasizes variational quantum circuits as the practical backbone of near-term QRL and analyzes how encoding strategies, entanglement structure, and measurement choices shape expressive power and trainability. It also examines quantum-enhanced exploration, showing how stochasticity emerges intrinsically from measurement rather than from externally injected noise. Policy-gradient and value-based approaches are compared, with particular attention to measurement cost, optimization stability, and near-term hardware suitability. This chapter further discusses quantum environments, sample-efficiency considerations, and hands-on implementations such as the quantum multi-armed bandit and quantum grid-world. Together, these topics establish QRL as a rigorous and practically grounded framework for studying sequential decision-making, control, and exploration in the quantum setting.

CRC Press eBooks•Nov 4, 2026•0 citations
This paper presents a sample-efficient hybrid quantum-classical reinforcement learning (QCRL) framework for multi-objective transmission switching (MO-TS) in power systems. We introduce a Chunk-Based Data Reuploading Variational Quantum Circuit (CBDR-VQC) architecture, which is designed to enhance policy expressivity without increasing circuit depth or width. Unlike conventional VQC approaches that scale circuit size to improve learning capacity, often at the cost of hardware feasibility and trainability, our method encodes high-dimensional input features sequentially into a fixed-width, shallow quantum circuit with a learnable measurement scheme. This design maintains compatibility with near-term quantum devices while avoiding issues such as noise accumulation and barren plateaus. Embedded within a quantum-classical Soft Actor-Critic (QCSAC) framework, the proposed model achieves competitive performance for MO-TS on the IEEE 118-bus system. It significantly reduces the numbers of both required qubits and trainable parameters compared to both classical and wider VQC-based baselines, demonstrating its promise for resourceconstrained QCRL in real-world power system optimization.
Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.
Quantum Computing aims to streamline machine learning, making it more effective with fewer trainable parameters. This reduction of parameters can speed up the learning process and reduce the use of computational resources. However, in the current phase of quantum computing development, known as the noisy intermediate-scale quantum era (NISQ), learning is difficult due to a limited number of qubits and widespread quantum noise. To overcome these challenges, researchers are focusing on variational quantum circuits (VQCs). VQCs are hybrid algorithms that merge a quantum circuit, which can be adjusted through parameters, with traditional classical optimization techniques. These circuits require only few qubits for effective learning. Recent studies have presented new ways of applying VQCs to reinforcement learning, showing promising results that warrant further exploration. This study investigates the effects of various techniques — data re-uploading, input scaling, output scaling — and introduces exponential learning rate decay in the quantum proximal policy optimization algorithm’s actor-VQC. We assess these methods in the popular Frozen Lake and Cart Pole environments. Our focus is on their ability to reduce the number of parameters in the VQC without losing effectiveness. Our findings indicate that data re-uploading and an exponential learning rate decay significantly enhance hyperparameter stability and overall performance. While input scaling does not improve parameter efficiency, output scaling effectively manages greediness, leading to increased learning speed and robustness.
Parameterised quantum circuit (PQC) based Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning (RL). By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain due to training instabilities, barren plateaus (BPs), and the difficulty of isolating the contribution of individual pipeline components. In this work, we dissect PQC based QRL architectures through a systematic experimental evaluation of three aspects recurrently identified as critical: (i) data embedding strategies, with Data Reuploading (DR) as an advanced approach; (ii) ansatz design, particularly the role of entanglement; and (iii) post-processing blocks after quantum measurement, with a focus on the underexplored Output Reuse (OR) technique. Using a unified PPO-CartPole framework, we perform controlled comparisons between hybrid and classical agents under identical conditions. Our results show that OR, though purely classical, exhibits distinct behaviour in hybrid pipelines, that DR improves trainability and stability, and that stronger entanglement can degrade optimisation, offsetting classical gains. Together, these findings provide controlled empirical evidence of the interplay between quantum and classical contributions, and establish a reproducible framework for systematic benchmarking and component-wise analysis in QRL.
The comparative evaluation between classical and quantum reinforcement learning (QRL) paradigms was conducted to investigate their convergence behavior, robustness under observational noise, and computational efficiency in a benchmark control environment. The study employed a multilayer perceptron (MLP) agent as a classical baseline and a parameterized variational quantum circuit (VQC) as a quantum counterpart, both trained on the CartPole-v1 environment over 500 episodes. Empirical results demonstrated that the classical MLP achieved near-optimal policy convergence with a mean return of 498.7 +/- 3.2, maintaining stable equilibrium throughout training. In contrast, the VQC exhibited limited learning capability, with an average return of 14.6 +/- 4.8, primarily constrained by circuit depth and qubit connectivity. Noise robustness analysis further revealed that the MLP policy deteriorated gracefully under Gaussian perturbations, while the VQC displayed higher sensitivity at equivalent noise levels. Despite the lower asymptotic performance, the VQC exhibited significantly lower parameter count and marginally increased training time, highlighting its potential scalability for low-resource quantum processors. The results suggest that while classical neural policies remain dominant in current control benchmarks, quantum-enhanced architectures could offer promising efficiency advantages once hardware noise and expressivity limitations are mitigated.
This paper proposes a quantum neural network (QNN)-based reinforcement learning (RL) controller for the real-time control of DC–DC buck converters supplying constant power loads (CPLs). Unlike conventional RL approaches that rely on classical neural networks, the proposed method employs a parameterized quantum circuit to represent the control policy, enabling a compact and expressive function approximator. The QNN is implemented as a classically simulated variational quantum circuit; no physical quantum hardware or quantum computational speedup is claimed. It is trained offline using a deterministic policy gradient framework and subsequently deployed in a real-time control loop under strict sampling constraints. The proposed 36-parameter QNN actor reduces trainable parameters by 85% versus a computation-matched classical RL baseline (245 parameters), while achieving sub-millisecond settling times and near-zero steady-state error across reference tracking, load disturbance, and supply variation scenarios validated on a dSPACE DS1202 hardware platform at 100-kHz switching frequency. Small-signal stability analysis at three tested operating points confirms phase margins exceeding 30° and a gain margin that is theoretically infinite at those points, as the linearized phase does not cross −180°. The results highlight the ability of the QNN to capture complex system dynamics with fewer trainable parameters while maintaining stable closed-loop operation, providing, to the best of the authors’ knowledge, a first step toward integrating quantum machine learning concepts into real-time power electronics control.
This chapter develops quantum reinforcement learning (QRL) as a principled extension of classical reinforcement learning (RL) into the quantum domain. Building on the standard Markov decision process framework, it shows how policies and value functions can be represented by quantum states and parameterized quantum circuits, with action probabilities arising from Born-rule measurement statistics and value estimates obtained from expectation values of observables. In this formulation, RL is preserved at the algorithmic level while its representational substrate is shifted from classical vector spaces to Hilbert space. The chapter emphasizes variational quantum circuits as the practical backbone of near-term QRL and analyzes how encoding strategies, entanglement structure, and measurement choices shape expressive power and trainability. It also examines quantum-enhanced exploration, showing how stochasticity emerges intrinsically from measurement rather than from externally injected noise. Policy-gradient and value-based approaches are compared, with particular attention to measurement cost, optimization stability, and near-term hardware suitability. This chapter further discusses quantum environments, sample-efficiency considerations, and hands-on implementations such as the quantum multi-armed bandit and quantum grid-world. Together, these topics establish QRL as a rigorous and practically grounded framework for studying sequential decision-making, control, and exploration in the quantum setting.