This chapter develops quantum reinforcement learning (QRL) as a principled extension of classical reinforcement learning (RL) into the quantum domain. Building on the standard Markov decision process framework, it shows how policies and value functions can be represented by quantum states and parameterized quantum circuits, with action probabilities arising from Born-rule measurement statistics and value estimates obtained from expectation values of observables. In this formulation, RL is preserved at the algorithmic level while its representational substrate is shifted from classical vector spaces to Hilbert space. The chapter emphasizes variational quantum circuits as the practical backbone of near-term QRL and analyzes how encoding strategies, entanglement structure, and measurement choices shape expressive power and trainability. It also examines quantum-enhanced exploration, showing how stochasticity emerges intrinsically from measurement rather than from externally injected noise. Policy-gradient and value-based approaches are compared, with particular attention to measurement cost, optimization stability, and near-term hardware suitability. This chapter further discusses quantum environments, sample-efficiency considerations, and hands-on implementations such as the quantum multi-armed bandit and quantum grid-world. Together, these topics establish QRL as a rigorous and practically grounded framework for studying sequential decision-making, control, and exploration in the quantum setting.
This chapter develops quantum reinforcement learning (QRL) as a principled extension of classical reinforcement learning (RL) into the quantum domain. Building on the standard Markov decision process framework, it shows how policies and value functions can be represented by quantum states and parameterized quantum circuits, with action probabilities arising from Born-rule measurement statistics and value estimates obtained from expectation values of observables. In this formulation, RL is preserved at the algorithmic level while its representational substrate is shifted from classical vector spaces to Hilbert space. The chapter emphasizes variational quantum circuits as the practical backbone of near-term QRL and analyzes how encoding strategies, entanglement structure, and measurement choices shape expressive power and trainability. It also examines quantum-enhanced exploration, showing how stochasticity emerges intrinsically from measurement rather than from externally injected noise. Policy-gradient and value-based approaches are compared, with particular attention to measurement cost, optimization stability, and near-term hardware suitability. This chapter further discusses quantum environments, sample-efficiency considerations, and hands-on implementations such as the quantum multi-armed bandit and quantum grid-world. Together, these topics establish QRL as a rigorous and practically grounded framework for studying sequential decision-making, control, and exploration in the quantum setting.
Quantum reinforcement learning (QRL) has emerged as a promising approach for controlling quantum systems and accelerating learning tasks with quantum resources. However, most existing QRL formulations still retain a classical reinforcement learning interface: states and actions are often encoded in computational bases and rewards are extracted through measurements. This interface is restrictive for genuinely quantum environments: choosing a measurement can discard entanglement information, while repeated intermediate measurements collapse the evolving state and require costly re-preparation. This motivates a fully QRL framework in which the agent, environment, rewards, and return accumulation are all quantum. In this paper, we take a step toward this goal and introduce a new QRL framework in which both the policy and the environment are modeled by conditional quantum channels. We then propose a quantum policy-gradient algorithm for parameterized conditional quantum channels with a trainable control basis. We also derive finite- and infinite-horizon gradient estimators using a new parameter-shift rule. This rule is needed because policy parameters are shared across repeated interaction blocks and the infinite-horizon objective is not a finite circuit; rather than shifting every occurrence and evaluating the whole trajectory, our estimator shifts only one sampled policy block. Sampling this block from a geometric distribution yields an unbiased gradient estimator under bounded rewards. Finally, we establish convergence of the resulting stochastic gradient-ascent procedure to stationary points under standard smoothness and step-size assumptions.
Quantum reinforcement learning (QRL) is a promising approach to learn effective decision strategies across several applications with stochastic environments. Instead of directly modeling the random variables that govern these environments, existing QRL architectures indirectly approximate environment behavior by estimating expected outcomes, which limits their expressive power and adaptive potential. Overcoming such challenges requires a novel QRL approach that exploits the distributional nature of quantum computers to directly model environment random variables as quantum state distributions. Hence, in this paper, a novel framework dubbed quantum-native reinforcement learning (QnRL) is proposed. QnRL is a distributional RL framework that learns conditional distributions naturally in Hilbert space via superimposed and entangled quantum states. Thus, QnRL can directly model the behavior of stochastic learning environments via the natural properties of quantum systems. QnRL accomplishes this via a novel, proposed quantum amplitude kickback (QuAK) algorithm that enables comparing the n-th power of the m-th moment of multiple superimposed distributions. It is theoretically proven that a conditional action policy distribution is distilled from the moments of a quantum generative model entirely within Hilbert space via QuAK, and optimized via QnRL. This complex distribution composition is also shown to provide extra dimensions for expressing environment correlations that are unknown to purely classical and classically-sampled quantum distributional models. Experimental results across diverse environments show that QnRL achieves up to 82.9% higher evaluation scores, with up to 94.3% fewer parameters on average, more accurately estimates the expected return for unseen observations, and better adapts to varying stochastic conditions compared to the baseline.
Exploring the convergence of quantum computing and machine learning, this paper delves into Quantum Reinforcement Learning (QRL) with a specific focus on Variational Quantum Circuits (VQC). Alongside a comprehensive examination of the field, the study presents a concrete implementation of QRL tailored to foundational gym environments. Leveraging principles from Quantum Computing, experiments reveal QRL’s potential in enhancing reinforcement learning tasks within these environments, showcasing notable space optimization for RL models. Quantitative results indicate a reduction in the number of trainable parameters by up to 90% when compared to classical approaches, significantly improving efficiency, mainly due to quantum principles of superposition and entanglement. This highlights the promising implications of integrating quantum computing techniques to address challenges and advance capabilities in fundamental reinforcement learning scenarios. Furthermore, the study discusses the adaptability of QRL algorithms to diverse problem domains, suggesting their potential for scalability and applicability beyond simple environments. Such versatility underscores the robustness and practical relevance of QRL methodologies in real-world scenarios, positioning them as valuable tools for tackling complex reinforcement learning challenges.
Quantum Reinforcement Learning (QRL) represents a groundbreaking approach at the intersection of quantum computing and artificial intelligence, promising to revolutionize decision-making processes across various domains. This chapter explores the fundamental principles of QRL and its diverse applications in critical sectors, including transportation networks, healthcare, financial decision-making, and smart energy systems. By leveraging the computational advantages of quantum algorithms, QRL enhances traditional reinforcement learning techniques, enabling more efficient data processing and improved strategic decision-making capabilities. Case studies within this chapter illustrate the transformative potential of QRL in optimizing logistics, improving medical imaging, and enhancing energy management strategies. The findings underscore the need for further exploration of quantum applications to fully harness their capabilities and address contemporary challenges. This comprehensive examination provides a solid foundation for researchers and practitioners aiming to integrate quantum technologies into real-world applications, paving the way for innovative solutions and sustainable advancements.
This chapter develops variational quantum models as the dominant framework for near-term quantum machine learning (QML), emphasizing their role as trainable quantum circuits embedded within hybrid quantum-classical optimization loops. Building on earlier chapters on encoding and quantum feature geometry, it formulates parameterized quantum circuits (PQCs) as families of unitary transformations acting on reference states, with model outputs obtained through expectation values of observables. This measurement-based structure makes variational learning inherently stochastic and closely ties model behavior to circuit architecture, observable choice, and optimization strategy. This chapter examines variational quantum circuits (VQCs) as learning models, introduces quantum neural networks (QNNs) as structured variational architectures, and analyzes layered, data-reuploading, recurrent, and hybrid classical-quantum designs. It highlights the central tension between expressivity and trainability, showing how universality and approximation power are counterbalanced by optimization barriers such as barren plateaus. The discussion further develops ansatz design strategies, supervised, fidelity-based, and energy-based losses, regularization under noise, and practical methods for gradient estimation and optimization, including the parameter-shift rule, Simultaneous Perturbation Stochastic Approximation (SPSA), and layer-wise training. Together, these ideas establish variational quantum models as a flexible yet hardware-constrained paradigm whose practical success depends on careful coordination among architecture, measurement, and optimization.
This paper presents a Quantum Reinforcement Learning (QRL) solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits. The implemented QRL approaches are quantum analogues of the classical neural-network-based Deep Deterministic Policy Gradient and Deep Q-Network algorithms. Through an empirical evaluation on real-world financial data, we show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models with several orders of magnitude more parameters. However, while quantum circuit execution is inherently fast at the hardware level, practical deployment on cloud-based quantum systems introduces substantial latency, making end-to-end runtime currently dominated by infrastructural overhead and limiting practical applicability. Taken together, our results suggest that QRL is theoretically competitive with state-of-the-art classical reinforcement learning and may become practically advantageous as deployment overheads diminish. This positions QRL as a promising paradigm for dynamic decision-making in complex, high-dimensional, and non-stationary environments such as financial markets. The complete codebase is released as open source at: https://github.com/VincentGurgul/qrl-dpo-public