A quantum compiler is a critical piece in the quantum computing pipeline since it allows an abstract quantum circuit to be run on a physical quantum computer. One extremely important subproblem in quantum compilation is the generation of a logical to physical qubit mapping. Typically in quantum compilers this step is either implemented as a random or a heuristic based assignment that aims to minimize additional (SWAP) gate overhead in the quantum circuit.
In this paper, we present an alternative approach to solving the qubit mapping problem. Specifically, we formulate the qubit mapping problem with a combinatorial optimization (CO) objective. We then present a method to find a solution to the CO problem by training a reinforcement learning (RL) policy. We also propose a local search based post-processing algorithm to further reduce the overhead. Our results show a dramatic improvement over conventional techniques in reducing the number of SWAPs. On different real world datasets like MQTBench and Queko circuits, our trained policy achieves a \textbf{65-85\%} reduction in SWAP overhead when compared to existing quantum compilers.
A quantum compiler is a critical piece in the quantum computing pipeline since it allows an abstract quantum circuit to be run on a physical quantum computer. One extremely important subproblem in quantum compilation is the generation of a logical to physical qubit mapping. Typically in quantum compilers this step is either implemented as a random or a heuristic based assignment that aims to minimize additional (SWAP) gate overhead in the quantum circuit.
In this paper, we present an alternative approach to solving the qubit mapping problem. Specifically, we formulate the qubit mapping problem with a combinatorial optimization (CO) objective. We then present a method to find a solution to the CO problem by training a reinforcement learning (RL) policy. We also propose a local search based post-processing algorithm to further reduce the overhead. Our results show a dramatic improvement over conventional techniques in reducing the number of SWAPs. On different real world datasets like MQTBench and Queko circuits, our trained policy achieves a \textbf{65-85%} reduction in SWAP overhead when compared to existing quantum compilers.
Quantum circuit compilation is a critical step in realizing quantum algorithms on near-term quantum hardware. The process of translating high-level quantum algorithms into low-level gate sequences, while minimizing gate count and circuit depth, is often performed using deterministic algorithms that struggle to adapt to the diverse and often imperfect characteristics of available quantum devices. This paper proposes a novel approach to quantum circuit compilation utilizing reinforcement learning (RL). An RL agent is trained to dynamically adapt to hardware constraints, such as qubit connectivity and gate decomposition costs, optimizing circuit fidelity and execution time. The core claim is that RL can significantly improve quantum circuit compilation by introducing a dynamic and adaptive compilation process. The agent learns a policy that maps high-level quantum algorithms to optimized low-level gate sequences, effectively bridging the gap between theoretical algorithms and practical hardware execution. This work demonstrates the potential of RL to overcome the limitations of traditional deterministic compilation methods and paves the way for more efficient and robust quantum algorithm implementation.
Quantum circuit placement (QCP) is the process of mapping the synthesized logical quantum programs on physical quantum machines, which introduces additional SWAP gates and affects the performance of quantum circuits. Nevertheless, determining the minimal number of SWAP gates has been demonstrated to be an NP-complete problem. Various heuristic approaches have been proposed to address QCP, but they suffer from suboptimality due to the lack of exploration. Although exact approaches can achieve higher optimality, they are not scalable for large quantum circuits due to the massive design space and expensive runtime. By formulating QCP as a bilevel optimization problem, this paper proposes a novel machine learning (ML)-based framework to tackle this challenge. To address the lower-level combinatorial optimization problem, we adopt a policy-based deep reinforcement learning (DRL) algorithm with knowledge transfer to enable the generalization ability of our framework. An evolutionary algorithm is then deployed to solve the upper-level discrete search problem, which optimizes the initial mapping with a lower SWAP cost. The proposed ML-based approach provides a new paradigm to overcome the drawbacks in both traditional heuristic and exact approaches while enabling the exploration of optimality-runtime trade-off. Compared with the leading heuristic approaches, our ML-based method significantly reduces the SWAP cost by up to 100%. In comparison with the leading exact search, our proposed algorithm achieves the same level of optimality while reducing the runtime cost by up to 40 times.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
Quantum computing is gaining more and more attention due to its huge potential and the constant progress in quantum computer development. IBM and Google have released quantum architectures with more than 50 qubits. However, in these machines, the physical qubits are not fully connected so that two-qubit interaction can only be performed between specific pairs of the physical qubits. To execute a quantum circuit, it is necessary to transform it into a functionally equivalent one that respects the constraints imposed by the target architecture. Quantum circuit transformation inevitably introduces additional gates which reduces the fidelity of the circuit. Therefore, it is important that the transformation method completes the transformation with minimal overheads. It consists of two steps, initial mapping and qubit routing. Here we propose a reinforcement learning-based model to solve the initial mapping problem. Initial mapping is formulated as sequence-to-sequence learning and self-attention network is used to extract features from a circuit. For qubit routing, a DEAR (Dynamically-Extract-and-Route) framework is proposed. The framework iteratively extracts a subcircuit and uses A* search to determine when and where to insert additional gates. It helps to preserve the lookahead ability dynamically and to provide more accurate cost estimation efficiently during A* search. The experimental results show that our RL-model generates better initial mappings than the best known algorithms with 12% fewer additional gates in the qubit routing stage. Furthermore, our DEAR-framework outperforms the state-of-the-art qubit routing approach with 8.4% and 36.3% average reduction in the number of additional gates and execution time starting from the same initial mapping.
The design of quantum algorithms typically assumes the availability of an ideal quantum computer, characterized by full connectivity, noiseless operation, and unlimited coherence time. However, Noisy Intermediate-Scale Quantum (NISQ) devices present a stark contrast, with a limited number of qubits, non-negligible quantum operation errors, and stringent constraints on the connectivity of physical qubits within a Quantum Processing Unit (QPU). This necessitates the dynamic remapping of logical qubits to physical qubits within the compiler to facilitate the execution of two-qubit gates in the algorithm. However, this introduces additional operations, consequently reducing the fidelity of the algorithm. Therefore, minimizing the number of added gates becomes crucial. Finding such an optimal routing problem is NP-hard, and the task is conventionally addressed using human-crafted heuristics to search for SWAP sequences, but these lack performance guarantees. In this study, we employ a Seq2Seq machine learning model for the qubit routing task, incorporating a Transformer neural network to learn the routing information in the gate and SWAP sequence. Compared to heuristic search-based algorithms, our approach significantly reduces the overhead of quantum computing resources required to adapt logical circuits to physical circuits executable on specific quantum backend hardware.
The scaling of quantum processors is currently limited by technical challenges such as decoherence and cross- talk. As the number of qubits grows, the interference between them adds more noise to the computations. Distributed quantum computing addresses this limitation by interconnecting smaller, easier-to-handle quantum cores, but it introduces the challenge of minimizing slow, error-prone inter-core communication. The task of distributing quantum circuits across cores to minimize communication costs is known as the Qubit Allocation problem. This work is centered on finding a deep learning approach to this problem, focusing on scalability and improving the state-of-the-art performance. Classical algorithms like the Hungarian Qubit Allocation (HQA) currently represent the state of the art, but they suffer from poor scalability with circuit size. In contrast, existing Deep Learning optimizations like Reinforcement Learning (RL) approaches often lack flexibility, requiring retraining if hardware configurations change, and are far from the solution quality of classical methods. However, these are worth exploring due to the good scalability and execution times. To overcome these limitations, this thesis proposes a flexible, transformer-based architecture capable of handling arbitrary numbers of qubits and cores without retraining. The policy is trained using a combination of REINFORCE and Group Relative Policy Optimization, two well-known RL algorithms. Furthermore, to explore the test-time compute paradigm applied to this problem, we introduce QAllocZero, an AlphaZero-inspired tree search algorithm we implemented in C++ that utilizes the trained policy as a heuristic. The results show that the trained policy achieves up to 33% improvement in allocation cost with respect to the state of the art (HQA) for the Cuccaro Adder and narrows the gap between RL and HQA for the most common circuits while maintaining scalable execution time. These findings show that learning-based approaches can effectively break the computational bottlenecks of classical algorithms, a crucial step for managing future systems with millions of qubits. All the code is open source and can be found in the porject's repository (https://github.com/Vicara12/MAI-TFM_RL_Qubit_Allocation).