Quantum walks on binary trees are used in many quantum algorithms to achieve important speedup over classical algorithms. The formulation of this kind of algorithms as quantum circuit presents the advantage of being easily readable, executable on circuit based quantum computers and simulators and optimal on the usage of resources. We propose a strategy to compose quantum circuit that performs quantum walk on binary trees following universal gate model quantum computation principles. We give a particular attention to NAND formula evaluation algorithm as it could have many applications in game theory and reinforcement learning. We therefore propose an application of this algorithm and show how it can be used to train a quantum reinforcement learning agent in a two player game environment.
Quantum walks on binary trees are used in many quantum algorithms to achieve important speedup over classical algorithms. The formulation of this kind of algorithms as quantum circuit presents the advantage of being easily readable, executable on circuit based quantum computers and simulators and optimal on the usage of resources. We propose a strategy to compose quantum circuit that performs quantum walk on binary trees following universal gate model quantum computation principles. We give a particular attention to NAND formula evaluation algorithm as it could have many applications in game theory and reinforcement learning. We therefore propose an application of this algorithm and show how it can be used to train a quantum reinforcement learning agent in a two player game environment.
Game theory problems are widely applied in many research areas such as computer science and finance, with the key issue being how to quickly make decisions. Here, we present a novel quantum algorithm for game theory problems based on a continuous quantum walk. Our algorithm exhibits quantum advantage compared to classical game algorithms. Furthermore, we exploit the analogy between the wave function of the Schrödinger equation and the voltage in Kirchhoff's law to effectively translate the design of quantum game trees into classical circuit networks. We have theoretically simulated the quantum game trees and experimentally validated the quantum functionality speedup on classical circuit networks. Due to the robust scalability and stability inherent in classical circuit networks, quantum game trees implemented within this framework hold promise for addressing more intricate application scenarios.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.
We introduce the "quantum circuit daemon" (QC-Daemon), a reinforcement learning agent for compiling quantum device operations aimed at efficient quantum hardware execution. We apply QC-Daemon to the move synthesis problem called the Atom Game, which involves orchestrating parallel circuits on reconfigurable neutral atom arrays. In our numerical simulation, the QC-Daemon is implemented by two different types of transformers with a physically motivated architecture and trained by a reinforcement learning algorithm. We observe a reduction of the logarithmic infidelity for various benchmark problems up to 100 qubits by intelligently changing the layout of atoms. Additionally, we demonstrate the transferability of our approach: a Transformer-based QC-Daemon trained on a diverse set of circuits successfully generalizes its learned strategy to previously unseen circuits.
A quantum algorithm is defined by a sequence of operations that runs on a realistic model of quantum computation. Since the first quantum algorithm proposed by David Deutsch(1985), a large number of impressive quantum algorithms have been developed. Quantum random walks on a graph, which are analogous to classical stochastic walk, form the basis for some of the recent quantum algorithms that promise to significantly outperform existing classical random walk algorithms. Though research has been done on the application of quantum random walk to important computational problems, very little work has been done on its quantum circuit design. There are two types of quantum random walk algorithms: discrete-time and continuous-time. In this paper, we propose quantum circuit designs for both types of random walk algorithms that operate on various graphs. We consider in detail two important problems to which random walk algorithms are applicable: the triangle finding problem and binary welded tree problem. Though there exist a few research works related to quantum circuit design for random walk on graphs, to the best of our knowledge, the circuit designs we present here are first of their kind. We also provide an estimate of the quantum cost of these circuits for several physical machine descriptions (PMDs)of quantum systems, based on the number of quantum operations and execution cycles.
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. This work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. This paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. This study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. The study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. This work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique. The purpose of this work is to help newcomers to this emerging field of study.
Quantum reinforcement learning (QRL) can outperform classical reinforcement learning (RL) by utilizing quantum parallel theory and quantum phenomena such as superposition and entanglement. Although some excellent work has been done on QRL, most existing works either fail to show the exponential advantage of quantum computation over classical computation in terms of performance or are too demanding on quantum devices. In this paper, we provide a novel perspective on combining quantum computing and RL with faster convergence speed and relatively relaxed demands on quantum devices. Specifically, we propose a method to construct a world model with quantum circuit that allows it to interact in a quantum way. In addition, we use Grover's algorithm to efficiently extract high-value information from the quantum world model. Extensive simulation results show that the proposed method can have superior performance compared to classical RL algorithms.