As a branch of quantum machine learning, quantum reinforcement learning (QRL) aims to solve complex sequential decision-making problems more efficiently and effectively than its classical counterpart by exploiting quantum resources. However, in the noisy intermediate-scale quantum (NISQ) era, its realization is challenged by the ubiquitous noise-induced decoherence. Here, we propose a noise-resilient QRL scheme for a quantum eigensolver with a two-level system as an agent. By investigating the non-Markovian decoherence effect on the QRL for solving the eigenstates of the agent-environment interaction Hamiltonian, we find that the formation of a bound state in the energy spectrum of the total agent-noise system restores the QRL performance to that in the noiseless case. Providing a universal physical mechanism to suppress the decoherence effect on quantum machine learning, our result lays the foundation for designing NISQ algorithms and offers a guideline for their practical implementation.
As a branch of quantum machine learning, quantum reinforcement learning (QRL) aims to solve complex sequential decision-making problems more efficiently and effectively than its classical counterpart by exploiting quantum resources. However, in the noisy intermediate-scale quantum (NISQ) era, its realization is challenged by the ubiquitous noise-induced decoherence. Here, we propose a noise-resilient QRL scheme for a quantum eigensolver with a two-level system as an agent. By investigating the non-Markovian decoherence effect on the QRL for solving the eigenstates of the agent-environment interaction Hamiltonian, we find that the formation of a bound state in the energy spectrum of the total agent-noise system restores the QRL performance to that in the noiseless case. Providing a universal physical mechanism to suppress the decoherence effect on quantum machine learning, our result lays the foundation for designing NISQ algorithms and offers a guideline for their practical implementation.
Quantum reinforcement learning (QRL) is a prospective system that can solve sequential control problems with quantum control, but the problem of hardware noise and decoherence cripples its application in noisy intermediate-scale quantum (NISQ) systems. This paper introduces a framework of Zero-Noise Quantum Reinforcement Learning (ZN-QRL), which directly includes the noise-resilient learning framework of zero-noise extrapolation (ZNE) into the policy optimization procedure. The presented architecture is implemented to perform parameterized quantum policies using amplified levels of noise and interpolates reward signal on the noise-free limit of a quantum system, where the learning agent is free to optimize policies as it would in an ideal environment with a quantum noisefree system but does not need to fully correct the error. Detailed simulation performance of quantum control benchmarks, such as state transfer and gate synthesis control, indicate that ZN-QRL is always better than standard variational QRL and post-hoc error-reducing control. In particular, the proposed framework attains control fidelity up to 30% greater, convergence is faster and more stable and performance is robust over a large dynamic depolarizing noise. Moreover, ZN-QRL also exhibits higher scalability and is able to maintain a steady learning rate with greater circuit depth and qubit count, therefore illustrating its usefulness in practical use of quantum control on existing NISQ devices.
By harnessing entanglement to transmit information beyond the limits of classical communication, quantum dense coding serves as a key component in various quantum technologies. However, in the noisy intermediate-scale quantum era, noise-induced decoherence poses inevitable challenges to its practical implementation. By investigating the non-Markovian decoherence dynamics of the two categories of quantum dense coding protocols realized in discrete- and continuous-variable systems, we demonstrate a physical mechanism to suppress the decoherence in quantum dense coding. It is found that the formation of a bound state in the energy spectrum of the entire system consisting of each subsystem and its local environment of the quantum channel restores the quantum advantage of dense coding in the presence of decoherence. Refreshing our understanding on the noisy effect on quantum dense coding, our finding offers a guide toward the practical realization of noise-tolerant quantum dense coding.
Abstract Quantum machine learning (QML) is a young but rapidly growing field where quantum information meets machine learning. Here, we will introduce a new QML model generalising the classical concept of reinforcement learning to the quantum domain, i.e. quantum reinforcement learning (QRL). In particular, we apply this idea to the maze problem, where an agent has to learn the optimal set of actions in order to escape from a maze with the highest success probability. To perform the strategy optimisation, we consider a hybrid protocol where QRL is combined with classical deep neural networks. In particular, we find that the agent learns the optimal strategy in both the classical and quantum regimes, and we also investigate its behaviour in a noisy environment. It turns out that the quantum speedup does robustly allow the agent to exploit useful actions also at very short time scales, with key roles played by the quantum coherence and the external noise. This new framework has the high potential to be applied to perform different tasks (e.g. high transmission/processing rates and quantum error correction) in the new-generation noisy intermediate-scale quantum (NISQ) devices whose topology engineering is starting to become a new and crucial control knob for practical applications in real-world problems. This work is dedicated to the memory of Peter Wittek.
Quantum reinforcement learning (QRL) has shown potential significant potential for solving difficult sequence decision-making with the power of representation of quantum computing. However quantum states and their measurements are sensitive, and this implies that such systems are vulnerable to adversarial perturbation together with hardware noise. Avoiding variational quantum policy being misled by any but small, intelligible perturbations of quantum states, inputs or measurement outcomes can generate incorrect actions and unstable learning behaviour. This paper has provided an adversarially resilient quantum reinforcement learning (AR-QRL) architecture, which specifically aims to address these security and dependability challenges. It is a mixture of adversarial training, robust quantum states encoding and variance-sensitive policy optimization, which defends variational quantum policies against malicious quantumstate perturbation. Benchmarking quantum-control experiments have demonstrated improvements of 30%-40% in control fidelity to that of standard QRL, up to a 75% decreased reward variance, and a 40% acceleration of convergence under adversarial conditions.. These findings suggest the impact of adversarial robustness in improving effective and secure quantum reinforcement learning of noisy intermediate-scale quantum (NISQ) devices.
In the era of Noisy Intermediate-Scale Quantum devices, Variational Quantum Circuits in Quantum Machine Learning are gaining attention, advancing towards practical quantum computing applications on NISQ devices. Reinforcement Learning (RL), known for its humanlike, trial-and-error learning, excels in adapting to changing financial environments. In particular, classical algorithms such as Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) show promise, while emerging quantum neural networks offer potential for improved function approximation, better generalization capabilities and reduced parameters. In light of these advancements, we introduce a quantum-enhanced version of the DDPG agent (QDDPG), aiming to leverage these quantum capabilities for more efficient financial decisionmaking processes. The quantum part of the model utilizes a parameterized quantum policy, which consists of a sequential reuploading variational circuit with trainable input scaling parameters. Our objective is to explore the practicality and potential benefits of QRL in finance, aiming to realize viable quantum computing applications on NISQ devices. The QDDPG model surpasses traditional analytic strategies in portfolio allocation, achieving higher returns with fewer parameters and a favourable Sharpe ratio. In a simple configuration using a RAW-parameterized quantum policy, the model achieves an average Sharpe ratio of 1.20 across 15 trained agents, compared to 0.96 for the minimum-variance strategy and 0.81 for the buy-and-hold baseline. Furthermore we investigate the impact of quantum noise, a significant challenge for NISQ devices. We analyze various types of noise, such as general depolarization, amplitude damping, phase damping, measurement noise, and shot noise, which uniquely affect quantum computations. Our study quantifies how these noise types influence the performance, reliability, and robustness of the QRL model. We study new techniques to harness quantum noise as hyperparameter to control the exploration-exploitation balance of the agent, using noise as regularization method and to stabilize the training of the agents. This comprehensive analysis aims to not only highlight the challenges posed by quantum noise but also to explore innovative methods to mitigate these effects and enhance the applicability of quantum algorithms in real-world applications.
Quantum reinforcement learning (QRL) is a burgeoning area that lies at the convergence of quantum computing and machine learning. Based on the characteristics of quantum superposition and quantum entanglement, quantum reinforcement learning can achieve some quantum advantages on existing noisy intermediate-scale quantum devices compared to traditional reinforcement learning. Within the field of information systems, reinforcement learning techniques have been extensively explored for applications including intelligent decision support mechanisms, resource distribution strategies, and system enhancement under conditions of uncertainty. This paper analyzes the fundamental principles of quantum reinforcement learning and introduces research work and progress in related areas. This study examines the foundational concepts underlying quantum reinforcement learning and provides an overview of notable algorithmic strategies documented in contemporary research.