With the rapid advancement of quantum computing, hybrid quantum–classical machine learning has shown numerous potential applications at the current stage, with expectations of being achievable in the noisy intermediate-scale quantum (NISQ) era. Quantum reinforcement learning, as an indispensable study, has recently demonstrated its ability to solve standard benchmark environments with formally provable theoretical advantages over classical counterparts. However, despite the progress of quantum processors and the emergence of quantum computing clouds, implementing quantum reinforcement learning algorithms utilizing parameterized quantum circuits (PQCs) on NISQ devices remains infrequent. In this work, we take the first step towards executing benchmark quantum reinforcement problems on real devices equipped with at most 136 qubits on the BAQIS Quafu quantum computing cloud. The experimental results demonstrate that the policy agents can successfully accomplish objectives under modified conditions in both the training and inference phases. Moreover, we design hardware-efficient PQC architectures in the quantum model using a multi-objective evolutionary algorithm and develop a learning algorithm that is adaptable to quantum devices. We hope that the Quafu-RL can be a guiding example to show how to realize machine learning tasks by taking advantage of quantum computers on the quantum cloud platform.
With the rapid advancement of quantum computing, hybrid quantum–classical machine learning has shown numerous potential applications at the current stage, with expectations of being achievable in the noisy intermediate-scale quantum (NISQ) era. Quantum reinforcement learning, as an indispensable study, has recently demonstrated its ability to solve standard benchmark environments with formally provable theoretical advantages over classical counterparts. However, despite the progress of quantum processors and the emergence of quantum computing clouds, implementing quantum reinforcement learning algorithms utilizing parameterized quantum circuits (PQCs) on NISQ devices remains infrequent. In this work, we take the first step towards executing benchmark quantum reinforcement problems on real devices equipped with at most 136 qubits on the BAQIS Quafu quantum computing cloud. The experimental results demonstrate that the policy agents can successfully accomplish objectives under modified conditions in both the training and inference phases. Moreover, we design hardware-efficient PQC architectures in the quantum model using a multi-objective evolutionary algorithm and develop a learning algorithm that is adaptable to quantum devices. We hope that the Quafu-RL can be a guiding example to show how to realize machine learning tasks by taking advantage of quantum computers on the quantum cloud platform.
The rapid advancement of quantum computing (QC) and machine learning (ML) has given rise to the burgeoning field of quantum machine learning (QML), aiming to capitalize on the strengths of quantum computing to propel ML forward. Despite its promise, crafting effective QML models necessitates profound expertise to strike a delicate balance between model intricacy and feasibility on Noisy Intermediate-Scale Quantum (NISQ) devices. While complex models offer robust representation capabilities, their extensive circuit depth may impede seamless execution on extant noisy quantum platforms. In this paper, we address this quandary of QML model design by employing deep reinforcement learning to explore proficient QML model architectures tailored for designated supervised learning tasks. Specifically, our methodology involves training an RL agent to devise policies that facilitate the discovery of QML models without predetermined ansatz. Furthermore, we integrate an adaptive mechanism to dynamically adjust the learning objectives, fostering continuous improvement in the agent's learning process. Through extensive numerical simulations, we illustrate the efficacy of our approach within the realm of classification tasks. Our proposed method successfully identifies VQC architectures capable of achieving high classification accuracy while minimizing gate depth. This pioneering approach not only advances the study of AI-driven quantum circuit design but also holds significant promise for enhancing performance in the NISQ era.
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. This work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. This paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. This study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. The study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. This work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique. The purpose of this work is to help newcomers to this emerging field of study.
Recent advancements in quantum computing have shown promising computational advantages in many problem areas. As one of those areas with increasing attention, hybrid quantum-classical machine learning systems have demonstrated the capability to solve various data-driven learning tasks. Recent works show that parameterized quantum circuits (PQCs) can be used to solve challenging reinforcement learning (RL) tasks with provable learning advantages. While existing works yield potentials of PQC-based methods, the design choices of PQC architectures and their influences on the learning tasks are generally underexplored. In this work, we introduce EQAS-PQC, an evolutionary quantum architecture search framework for PQC-based models, which uses a population-based genetic algorithm to evolve PQC architectures by exploring the search space of quantum operations. Experimental results show that our method can significantly improve the performance of hybrid quantum-classical models in solving benchmark reinforcement problems. We also model the probability distributions of quantum operations in top-performing architectures to identify essential design choices that are critical to the performance.
Quantum computing (QC) in the current NISQ era is still limited in size and precision. Hybrid applications mitigating those shortcomings are prevalent to gain early insight and advantages. Hybrid quantum machine learning (QML) comprises both the application of QC to improve machine learning (ML) and ML to improve QC architectures. This work considers the latter, leveraging reinforcement learning (RL) to improve quantum circuit design (QCD), which we formalize by a set of generic objectives. Furthermore, we propose qcd-gym, a concrete framework formalized as a Markov decision process, to enable learning policies capable of controlling a universal set of continuously parameterized quantum gates. Finally, we provide benchmark comparisons to assess the shortcomings and strengths of current state-of-the-art RL algorithms.
Recent work on hybrid quantum-classical machine learning systems has demonstrated success in utilizing parameterized quantum circuits (PQCs) to solve the challenging reinforcement learning (RL) tasks, with provable learning advantages over classical systems, e.g., deep neural networks. While existing work demonstrates and exploits the strength of PQC-based models, the design choices of PQC architectures and the interactions between different quantum circuits on learning tasks are generally underexplored. In this work, we introduce a Multi-objective Evolutionary Architecture Search framework for parameterized quantum circuits (MEAS-PQC), which uses a multi-objective genetic algorithm with quantum-specific configurations to perform efficient searching of optimal PQC architectures. Experimental results show that our method can find architectures that have superior learning performance on three benchmark RL tasks, and are also optimized for additional objectives including reductions in quantum noise and model size. Further analysis of patterns and probability distributions of quantum operations helps identify performance-critical design choices of hybrid quantum-classical learning systems.
The design of autonomous quantum systems constitutes an entirely new paradigm for next-generation computing by having machine intelligence do the engineering work to build and optimise successful quantum architectures. We address the problem of self-optimizing quantum systems that perform adaptive control, circuit synthesis and decision making in both accurate and uncertain controllable environments via a two-pronged approach: Reinforcement Learning (RL) together with Evolutionary Strategies (ES). In this work, we find that reinforcement learning provides a general framework for sequential decision-making, where agents learn optimal policies from reward signals by interacting with quantum environments. Evolutionary strategies, which are based on ideas from biological evolution, provide an alternative to RL by performing global optimization of quantum circuit parameters and architectures using the standard canonical processes of mutation, selection and recombination. Recent developments in quantum technologies show, again, that machine learning tools (ML), especially RL can be very useful to optimize unavailable closed blow-ups related with quantity functions. Moreover, evolutionary optimization based hybrids with RL have demonstrated better performance on the complex multi-objective quantum problems due to balanced exploitation and exploration. In this paper, we provide an overview of a versatile framework for autonomous quantum system design that showcases example algorithmic architectures, hybrid machine learning models and real world applications in areas such as quantum control, quantum circuit synthesis and energy optimization. In addition, we discuss well-known obstacles such as the scalability problem, the noise sensitivity issue, barren plateaus of quantum optimization and computational complexity. Comparisons to traditional methods show that reinforcement learning (RL) when combined with evolutionary strategies enables a significant improvement in efficiency, adaptability, and robustness for quantum system design as supported by experimental studies and case analyses. Future research directions are then discussed, including quantum-native learning algorithms, multi-agent quantum reinforcement learning and hardware-aware optimization techniques