Quantum computing holds the potential to revolutionize numerous fields, yet the practical execution of quantum circuits depends on an efficient compilation process, including the placement of logical qubits on a quantum chip. This placement is a hard combinatorial problem that often defeats traditional heuristics and manual optimization. We present QAgent, a novel multi-agent reinforcement learning (RL) framework that autonomously optimizes logical qubit layouts on quantum processor. In QAgent, one agent is assigned to each logical qubit, and the agents jointly learn placement policies that minimize circuit execution cost. To address the challenges of sparse rewards and reward hacking, we propose the Breakthrough Return Bonus (BRB), a dynamic reward shaping mechanism that encourages meaningful layout improvements and accelerates convergence. Extensive experiments on diverse quantum circuit benchmarks show that QAgent reduces execution costs by up to 53.9% compared to leading baselines, and significantly enhances circuit success rates. Ablation studies confirm that BRB is essential for stable training and effective policy optimization. These results demonstrate the promise of AI-driven, workload-aware placement for advancing fault-tolerant quantum computing.
Quantum computing holds the potential to revolutionize numerous fields, yet the practical execution of quantum circuits depends on an efficient compilation process, including the placement of logical qubits on a quantum chip. This placement is a hard combinatorial problem that often defeats traditional heuristics and manual optimization. We present QAgent, a novel multi-agent reinforcement learning (RL) framework that autonomously optimizes logical qubit layouts on quantum processor. In QAgent, one agent is assigned to each logical qubit, and the agents jointly learn placement policies that minimize circuit execution cost. To address the challenges of sparse rewards and reward hacking, we propose the Breakthrough Return Bonus (BRB), a dynamic reward shaping mechanism that encourages meaningful layout improvements and accelerates convergence. Extensive experiments on diverse quantum circuit benchmarks show that QAgent reduces execution costs by up to 53.9% compared to leading baselines, and significantly enhances circuit success rates. Ablation studies confirm that BRB is essential for stable training and effective policy optimization. These results demonstrate the promise of AI-driven, workload-aware placement for advancing fault-tolerant quantum computing.
Abstract Logical-qubit allocation is a central compilation problem for fault-tolerant quantum architectures, where circuit qubits must be assigned to feasible chip tiles while minimizing the ancilla-qubit cost (the number of extra workspace qubits consumed by the routing corridors required to execute the circuit) and preserving efficient access to magic-state resources. This hard combinatorial problem has resisted full automation: existing allocators struggle to jointly capture global circuit interactions, tile-level routing feasibility, and generalization across circuit families. We propose a graph-based reinforcement-learning framework in which a graph neural network is first pretrained to predict the ancilla-qubit cost of circuit-allocation pairs from allocation-aware circuit graphs (graphs that combine circuit structure with the current placement of each qubit), and the resulting encoder is transferred to a reinforcement learning agent that assigns tiles one at a time. As a result, the agent learns to utilize past experience to produce higher-quality solutions to new instances of the allocation problem. Evaluated on MQTBench circuits compiled to lattice-surgery instructions under leave-one-family-out cross-validation, our allocator reduces average ancilla-qubit cost by 36.7% compared to the state-of-the-art ECMAS+ baseline and achieves lower cost in 57 out of 64 qubit-size bins. The method further exhibits family-dependent scale extrapolation and responsive few-shot adaptation, establishing learned logical-qubit allocation as a scalable and transferable paradigm that outperforms ECMAS+ under the evaluated fixed-chip setting.
Quantum circuit optimization remains a significant challenge in the development of practical quantum computing. Traditional methods often rely on heuristics or exhaustive searches, which are computationally expensive and may not effectively adapt to the specific characteristics of the quantum data being processed. This paper proposes a novel approach leveraging reinforcement learning (RL) to achieve contextualized quantum circuit optimization. The core idea is to train an RL agent to dynamically modify quantum circuits in real-time, guided by a reward function that balances circuit fidelity and execution time. Crucially, the agent incorporates a learned contextual representation of the input data, allowing it to tailor the circuit design to the specific context. We demonstrate the potential of this approach by outlining the key components, the reward function design, and the overall architecture. The potential benefits include improved circuit fidelity, reduced execution time, and the ability to adapt to diverse quantum datasets. This represents a shift from static circuit design to adaptive, context-aware optimization, potentially unlocking the full potential of quantum computation.
A reinforcement learning (RL) framework is introduced for the efficient synthesis of quantum circuits that generate specified target quantum states from a fixed initial state, addressing a central challenge in both the Noisy Intermediate-Scale Quantum (NISQ) era and future fault-tolerant quantum computing. The approach utilizes tabular Q-learning, based on action sequences, within a discretized quantum state space, to effectively manage the exponential growth of the space dimension. The framework introduces a hybrid reward mechanism, combining a static, domain-informed reward that guides the agent toward the target state with customizable dynamic penalties that discourage inefficient circuit structures such as gate congestion and redundant state revisits. This is a circuit-aware reward, in contrast to the current trend of works on this topic, which are primarily fidelity-based. By leveraging sparse matrix representations and state-space discretization, the method enables practical navigation of high-dimensional environments while minimizing computational overhead. Benchmarking on graph-state preparation tasks for up to seven qubits, we demonstrate that the algorithm consistently discovers minimal-depth circuits with optimized gate counts. Moreover, extending the framework to a universal gate set still yields low depth circuits, highlighting the algorithm’s robustness and adaptability. The results confirm that this RL-driven approach, with our completely circuit-aware method, efficiently explores the complex quantum state space and synthesizes near-optimal quantum circuits, providing a resource-efficient foundation for quantum circuit optimization.
Quantum circuit compilation is a crucial step in realizing quantum algorithms on real quantum hardware. Traditionally, compilation methods are largely static, relying on pre-defined mappings and optimizations that fail to account for the inherent noise and variability present in quantum devices. This paper proposes a novel approach leveraging Reinforcement Learning (RL) to dynamically generate and optimize quantum circuits. An RL agent is trained to learn a policy for circuit design, adapting to specific hardware characteristics and real-time measurement data. The agent's goal is to minimize the overall error rate of the compiled circuit. This dynamic compilation framework offers a feedback loop between hardware performance and circuit design, resulting in adaptive and robust quantum computation. The core claim of this work is that traditional static compilation methods are insufficient, and dynamic compilation driven by RL offers a superior solution. The approach presented here represents a significant advancement towards realizing the full potential of quantum computing by mitigating the effects of hardware imperfections. The framework includes a detailed description of the RL agent architecture, the reward function design, and the exploration strategy employed. The effectiveness of the proposed method is demonstrated through simulations, highlighting its ability to outperform static compilation strategies in noisy environments.
As it becomes increasingly difficult to monolithically scale a quantum processor, distributed quantum computing (DQC) offers an alternative by distributing qubits across multiple smaller interconnected quantum processor modules. In such an architecture, the challenge of quantum circuit compilation shifts from placing and routing qubits within one module to placing, routing and using the qubits efficiently across modules. In order to optimize circuit execution time, the right state-dependent networking decisions must be found, such as when and where to generate shared remote quantum states to support remote operations.
Reinforcement learning (RL) provides a natural framework for this problem, generating a compilation policy that can generalize across different circuits. Building on the framework of Promponas et al. (2024), we introduce an agent that combines a novel action-space formulation with effective action-masking strategies. A comprehensive numerical comparison of the two approaches under different coupling constraints shows that our agent achieves improved training and inference performance with a relative reduction in the modeled execution time of up to 35\%.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.