Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2507.16641 · Quantum Machine Intelligence

Hybrid reward-driven reinforcement learning for efficient quantum circuit synthesis

Sara Giordano, Kornikar Sen, Miguel Angel Martin-Delgado

Published Feb 3, 2026• 4 citations
arXivHTMLPDF

Abstract

A reinforcement learning (RL) framework is introduced for the efficient synthesis of quantum circuits that generate specified target quantum states from a fixed initial state, addressing a central challenge in both the Noisy Intermediate-Scale Quantum (NISQ) era and future fault-tolerant quantum computing. The approach utilizes tabular Q-learning, based on action sequences, within a discretized quantum state space, to effectively manage the exponential growth of the space dimension. The framework introduces a hybrid reward mechanism, combining a static, domain-informed reward that guides the agent toward the target state with customizable dynamic penalties that discourage inefficient circuit structures such as gate congestion and redundant state revisits. This is a circuit-aware reward, in contrast to the current trend of works on this topic, which are primarily fidelity-based. By leveraging sparse matrix representations and state-space discretization, the method enables practical navigation of high-dimensional environments while minimizing computational overhead. Benchmarking on graph-state preparation tasks for up to seven qubits, we demonstrate that the algorithm consistently discovers minimal-depth circuits with optimized gate counts. Moreover, extending the framework to a universal gate set still yields low depth circuits, highlighting the algorithm’s robustness and adaptability. The results confirm that this RL-driven approach, with our completely circuit-aware method, efficiently explores the complex quantum state space and synthesizes near-optimal quantum circuits, providing a resource-efficient foundation for quantum circuit optimization.

DOI
10.1007/s42484-026-00359-8

Related papers

Adaptive Quantum Circuit Compilation with Reinforcement Learning

Jincheng Zhang

This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.

•Sep 5, 2026•0 citations

Rethinking Quantum Circuit Optimization as a Reinforcement Learning Problem

Arjun Mehta, Sven Richter

Quantum circuit optimization is a foundational challenge in realizing practical quantum computation, requiring the reduction of gate counts and circuit depth while preserving logical equivalence across noisy intermediate-scale quantum (NISQ) devices. Conventional rule-based and heuristic compilation approaches demonstrate diminishing effectiveness as circuit complexity grows, motivating the search for adaptive, data-driven alternatives. This paper reformulates quantum circuit optimization as a Markov decision process (MDP), enabling deep reinforcement learning (RL) agents to autonomously discover efficient transformation strategies. A graph neural network (GNN)-based actor-critic architecture is developed to encode circuit topology and guide transformation selection. Drawing on empirical evidence from phase-structured quantum control tasks, neural feedback architectures, and convergence behavior in active learning agents, we demonstrate that the proposed framework achieves an average two-qubit gate count reduction of 31.4% and a circuit depth reduction of 27.8% across standard benchmark circuits, outperforming established compilation baselines. This work provides a principled theoretical foundation for the RL-as-optimizer paradigm and contributes an extensible experimental platform suited to NISQ-era constraints.

American Journal Of Big Data•Apr 1, 2026•0 citations

Structure-Aware Quantum Circuit Partitioning via Reinforcement Learning for Efficient Re-Synthesis

Mohammad Walid Charrwi, C. Rasmussen, Ed Younis +2 more

The advancement of quantum computing into the utility scale requires compilation frameworks that can effectively manage the discrepancy between high-level algorithmic intent and the low-level physical constraints of contemporary hardware. Quantum circuit partitioning is a pivotal stage in this compilation pipeline, particularly when leveraging high-performance synthesis tools that are computationally bounded by the number of qubits. Existing partitioning approaches, such as ScanPartitioner [19] and QuickPartitioner [21], while effective, do not leverage structural patterns in circuits, limiting their ability to make globally informed local partitioning decisions. To address this gap, we propose a novel structural-aware quantum circuit partitioning method using a reinforcement learning (RL) framework that harnesses global circuit knowledge to guide local partitioning decisions, enabling more optimization opportunities at the sub-circuit level. Experimental results on benchmark circuits transpiled to satisfy IBM quantum hardware constraints show that our approach reduces the native two-qubit gate count compared to existing quantum circuit partitioners (ScanPartitioner and QuickPartitioner) with an average two-qubit gate reduction of 18.55% over baselines. This research establishes a scalable and robust methodology for partitioning quantum circuits, bridging the gap between exact and approximate synthesis in the Noisy Intermediate-Scale Quantum (NISQ) era and beyond.

ACM Great Lakes Symposium on VLSI (GLSVLSI)•Jun 18, 2026•0 citations

Using Topology-Aware Reinforcement Learning to Synthesize Quantum Linear Reversible Circuits

Giovanni Acampora, Allegra Cuzzocrea, Marco Lapegna +2 more

In the Noisy Intermediate-Scale Quantum (NISQ) era, efficient quantum circuit synthesis is essential for optimizing limited qubit resources and mitigating noise effects. Quantum Linear Reversible Circuits (QLRCs) are a fundamental class of circuits with applications in quantum compilation or quantum error correction. They are composed exclusively of CX and SWAP gates, which are among the noisiest operations on current quantum hardware. Therefore, their efficient synthesis is pivotal for executing QLRCs on NISQ processors, which have restricted connectivity between qubits. In this direction, we introduce a Reinforcement Learning (RL)-based synthesis approach using Proximal Policy Optimization (PPO) to synthesize QLRCs on generic topologies of up to five qubits. This work extends previous work that did not handle different topologies of quantum processors. Our method dynamically adapts to hardware constraints, learning optimal or near-optimal gate decompositions. We compare our approach against Qiskit’s synthesis methods for QLRCs, demonstrating that our RL-based synthesizer consistently achieves lower CX gate depth. These results highlight the potential of RL-driven quantum circuit synthesis as a powerful alternative to traditional heuristic-based techniques, paving the way for more efficient AI-based quantum compilation strategies in the NISQ era.

International Joint Conference on Neural Networks (IJCNN)•Jun 30, 2025•0 citations

General-purpose quantum architecture search based on deep reinforcement learning

Xiao-Yu Bi, Y. Yu, Ye‐Hong Chen +1 more

Reinforcement learning (RL) shows promise for automated quantum circuit design but often stalls due to a “fidelity trap”: by optimizing only state fidelity, agents overlook entanglement structure and become stranded in suboptimal, overly complex circuits, resulting in significantly reduced search efficiency. In this paper, we propose a scheme that overcomes this barrier by implementing an entanglement-aware learning framework and enhancing the agent's reward function with a direct, quantitative measure of entanglement. This approach offers a more comprehensive physical description of the state space. We demonstrate the efficacy of this principle on three- and four-qubit state-synthesis tasks within an expanded gate set. For this problem, where the fidelity-driven agent systematically fails to discover the minimal-depth circuit, our entanglement-aware agent consistently succeeds. This transformative result is highly robust against variations in initial random seeds and extends to multiqubit systems even in the presence of noise. Our findings establish a generalizable principle that incorporating entanglement as an auxiliary reward can significantly enhance RL-based solutions for a broad class of fidelity-centric tasks in quantum physics and pave the way for scalable, automated discovery on near-term quantum devices.

Physical Review A•Sep 26, 2025•10 citations

Quantum circuit optimization using deep reinforcement learning: Applications across multiple gate sets

Khoa Dang Tao, Sumin Jin, Muhammad Raza +1 more

The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.

quant-pharXiv:2608.19103•Aug 20, 2026•0 citations
A reinforcement learning (RL) framework is introduced for the efficient synthesis of quantum circuits that generate specified target quantum states from a fixed initial state, addressing a central challenge in both the Noisy Intermediate-Scale Quantum (NISQ) era and future fault-tolerant quantum computing. The approach utilizes tabular Q-learning, based on action sequences, within a discretized quantum state space, to effectively manage the exponential growth of the space dimension. The framework introduces a hybrid reward mechanism, combining a static, domain-informed reward that guides the agent toward the target state with customizable dynamic penalties that discourage inefficient circuit structures such as gate congestion and redundant state revisits. This is a circuit-aware reward, in contrast to the current trend of works on this topic, which are primarily fidelity-based. By leveraging sparse matrix representations and state-space discretization, the method enables practical navigation of high-dimensional environments while minimizing computational overhead. Benchmarking on graph-state preparation tasks for up to seven qubits, we demonstrate that the algorithm consistently discovers minimal-depth circuits with optimized gate counts. Moreover, extending the framework to a universal gate set still yields low depth circuits, highlighting the algorithm’s robustness and adaptability. The results confirm that this RL-driven approach, with our completely circuit-aware method, efficiently explores the complex quantum state space and synthesizes near-optimal quantum circuits, providing a resource-efficient foundation for quantum circuit optimization.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
Quantum circuit optimization is a foundational challenge in realizing practical quantum computation, requiring the reduction of gate counts and circuit depth while preserving logical equivalence across noisy intermediate-scale quantum (NISQ) devices. Conventional rule-based and heuristic compilation approaches demonstrate diminishing effectiveness as circuit complexity grows, motivating the search for adaptive, data-driven alternatives. This paper reformulates quantum circuit optimization as a Markov decision process (MDP), enabling deep reinforcement learning (RL) agents to autonomously discover efficient transformation strategies. A graph neural network (GNN)-based actor-critic architecture is developed to encode circuit topology and guide transformation selection. Drawing on empirical evidence from phase-structured quantum control tasks, neural feedback architectures, and convergence behavior in active learning agents, we demonstrate that the proposed framework achieves an average two-qubit gate count reduction of 31.4% and a circuit depth reduction of 27.8% across standard benchmark circuits, outperforming established compilation baselines. This work provides a principled theoretical foundation for the RL-as-optimizer paradigm and contributes an extensible experimental platform suited to NISQ-era constraints.
The advancement of quantum computing into the utility scale requires compilation frameworks that can effectively manage the discrepancy between high-level algorithmic intent and the low-level physical constraints of contemporary hardware. Quantum circuit partitioning is a pivotal stage in this compilation pipeline, particularly when leveraging high-performance synthesis tools that are computationally bounded by the number of qubits. Existing partitioning approaches, such as ScanPartitioner [19] and QuickPartitioner [21], while effective, do not leverage structural patterns in circuits, limiting their ability to make globally informed local partitioning decisions. To address this gap, we propose a novel structural-aware quantum circuit partitioning method using a reinforcement learning (RL) framework that harnesses global circuit knowledge to guide local partitioning decisions, enabling more optimization opportunities at the sub-circuit level. Experimental results on benchmark circuits transpiled to satisfy IBM quantum hardware constraints show that our approach reduces the native two-qubit gate count compared to existing quantum circuit partitioners (ScanPartitioner and QuickPartitioner) with an average two-qubit gate reduction of 18.55% over baselines. This research establishes a scalable and robust methodology for partitioning quantum circuits, bridging the gap between exact and approximate synthesis in the Noisy Intermediate-Scale Quantum (NISQ) era and beyond.
In the Noisy Intermediate-Scale Quantum (NISQ) era, efficient quantum circuit synthesis is essential for optimizing limited qubit resources and mitigating noise effects. Quantum Linear Reversible Circuits (QLRCs) are a fundamental class of circuits with applications in quantum compilation or quantum error correction. They are composed exclusively of CX and SWAP gates, which are among the noisiest operations on current quantum hardware. Therefore, their efficient synthesis is pivotal for executing QLRCs on NISQ processors, which have restricted connectivity between qubits. In this direction, we introduce a Reinforcement Learning (RL)-based synthesis approach using Proximal Policy Optimization (PPO) to synthesize QLRCs on generic topologies of up to five qubits. This work extends previous work that did not handle different topologies of quantum processors. Our method dynamically adapts to hardware constraints, learning optimal or near-optimal gate decompositions. We compare our approach against Qiskit’s synthesis methods for QLRCs, demonstrating that our RL-based synthesizer consistently achieves lower CX gate depth. These results highlight the potential of RL-driven quantum circuit synthesis as a powerful alternative to traditional heuristic-based techniques, paving the way for more efficient AI-based quantum compilation strategies in the NISQ era.
Reinforcement learning (RL) shows promise for automated quantum circuit design but often stalls due to a “fidelity trap”: by optimizing only state fidelity, agents overlook entanglement structure and become stranded in suboptimal, overly complex circuits, resulting in significantly reduced search efficiency. In this paper, we propose a scheme that overcomes this barrier by implementing an entanglement-aware learning framework and enhancing the agent's reward function with a direct, quantitative measure of entanglement. This approach offers a more comprehensive physical description of the state space. We demonstrate the efficacy of this principle on three- and four-qubit state-synthesis tasks within an expanded gate set. For this problem, where the fidelity-driven agent systematically fails to discover the minimal-depth circuit, our entanglement-aware agent consistently succeeds. This transformative result is highly robust against variations in initial random seeds and extends to multiqubit systems even in the presence of noise. Our findings establish a generalizable principle that incorporating entanglement as an auxiliary reward can significantly enhance RL-based solutions for a broad class of fidelity-centric tasks in quantum physics and pave the way for scalable, automated discovery on near-term quantum devices.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.