Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2408.15328 · Quantum Science and Technology

Artificially intelligent Maxwell’s demon for optimal control of open quantum systems

Paolo Andrea Erdman, Robert Czupryniak, Bibek Bhandari, Andrew N. Jordan, Frank Noé, Jens Eisert, Giacomo Guarnieri

Published Mar 5, 2025• 10 citations
arXivHTMLPDF

Abstract

Abstract Feedback control of open quantum systems is of fundamental importance for practical applications in various contexts, ranging from quantum computation to quantum error correction and quantum metrology. Its use in the context of thermodynamics further enables the study of the interplay between information and energy. However, deriving optimal feedback control strategies is highly challenging, as it involves the optimal control of open quantum systems, the stochastic nature of quantum measurement, and the inclusion of policies that maximize a long-term time- and trajectory-averaged goal. In this work, we employ a reinforcement learning approach to automate and capture the role of a quantum Maxwell’s demon: the agent takes the literal role of discovering optimal feedback control strategies in qubit-based systems that maximize a trade-off between measurement-powered cooling and measurement efficiency. Considering weak or projective quantum measurements, we explore different regimes based on the ordering between the thermalization, the measurement, and the unitary feedback timescales, finding different and highly non-intuitive, yet interpretable, strategies. In the thermalization-dominated regime, we find strategies with elaborate finite-time thermalization protocols conditioned on measurement outcomes. In the measurement-dominated regime, we find that optimal strategies involve adaptively measuring different qubit observables reflecting the acquired information, and repeating multiple weak measurements until the quantum state is ‘sufficiently pure’, leading to random walks in state space. Finally, we study the case when all timescales are comparable, finding new feedback control strategies that considerably outperform more intuitive ones. We discuss a two-qubit example where we explore the role of entanglement and conclude discussing the scaling of our results to quantum many-body systems.

DOI
10.1088/2058-9565/adbccf

Related papers

Maxwell’s Demon Meets Nonequilibrium Quantum Thermodynamics

John Goold

Maxwell's demon explores the role of information in physical processes.Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics.In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion.We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system.The demon is experimentally implemented as a spin-1=2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1=2 system, through a natural interaction.Noise and imperfections in this protocol are investigated by the assessment of its effectiveness.This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.

Physics•Dec 5, 2016•1 citation

Experimental Rectification of Entropy Production by Maxwell’s Demon in a Quantum System

Patrice A. Camati, John P. S. Peterson, Tiago B. Batalhão +5 more

Maxwell's demon explores the role of information in physical processes. Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics. In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion. We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system. The demon is experimentally implemented as a spin-1/2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1/2 system, through a natural interaction. Noise and imperfections in this protocol are investigated by the assessment of its effectiveness. This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.

arXiv:1605.08821•Physical Review Letters•Dec 5, 2016•173 citations

Quantum Control based on Deep Reinforcement Learning

Zhikang Wang

In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.

arXiv:2212.07385•Dec 14, 2022•2 citations

Measurement-Based Quantum Thermal Machines with Feedback Control

Bibek Bhandari, Robert Czupryniak, Paolo Andrea Erdman +1 more

We investigated coupled-qubit-based thermal machines powered by quantum measurements and feedback. We considered two different versions of the machine: (1) a quantum Maxwell's demon, where the coupled-qubit system is connected to a detachable single shared bath, and (2) a measurement-assisted refrigerator, where the coupled-qubit system is in contact with a hot and cold bath. In the quantum Maxwell's demon case, we discuss both discrete and continuous measurements. We found that the power output from a single qubit-based device can be improved by coupling it to the second qubit. We further found that the simultaneous measurement of both qubits can produce higher net heat extraction compared to two setups operated in parallel where only single-qubit measurements are performed. In the refrigerator case, we used continuous measurement and unitary operations to power the coupled-qubit-based refrigerator. We found that the cooling power of a refrigerator operated with swap operations can be enhanced by performing suitable measurements.

arXiv:2212.01502•Entropy•Jan 20, 2023•23 citations

Impact of Information on Quantum Heat Engines

Lindsay Bassman Oftelie, Michele Campisi

The emerging field of quantum thermodynamics is beginning to reveal the intriguing role that information can play in quantum thermal engines. Information enters as a resource when considering feedback-controlled thermal machines. While both a general theory of quantum feedback control as well as specific examples of quantum feedback-controlled engines have been presented, still lacking is a general framework for such machines. Here, we present a framework for a generic, two-stroke quantum heat engine interacting with $N$ thermal baths and Maxwell's demon. The demon performs projective measurements on the engine working substance, the outcome of which is recorded in a classical memory, embedded in its own thermal bath. To perform feedback control, the demon enacts unitary operations on the working substance, conditioned on the recorded outcome. By considering the compound machine-memory as a hybrid (classical-quantum) standard thermal machine interacting with $N+1$ thermal baths, our framework puts the working substance and memory on equal footing, thereby enabling a comprehensible resolution to Maxwell's paradox and elucidating the intricate manner in which information impacts the performance of quantum engines. We illustrate the application of our framework with a two-qubit engine. A remarkable observation is that more information does not necessarily result in better thermodynamic performance: sometimes knowing less is better.

quant-pharXiv:2512.13371•Physical Review A•Dec 16, 2025•1 citation

Adaptive Quantum Control with Feedback Learning Algorithm

Jincheng Zhang

This paper presents a novel feedback learning algorithm for adaptive quantum control. The algorithm leverages reinforcement learning principles combined with quantum control theory to dynamically adjust control strategies based on the observed state and environment of a quantum system. The core mechanism involves constructing an adaptive quantum control feedback learning algorithm that enables autonomous optimization of the quantum system. The algorithm utilizes a state representation of the quantum system, a reward function reflecting the desired outcome, and a control policy updated through a reinforcement learning process. Mathematical formulations are provided to describe the system dynamics, the control policy, and the learning process. The proposed algorithm aims to overcome the limitations of traditional fixed-parameter quantum control methods by adapting to uncertainties and complex dynamics, ultimately leading to improved control performance and robustness. The theoretical framework presented here offers a foundation for developing advanced quantum control strategies in various applications, including quantum computation, quantum sensing, and quantum metrology.

•Aug 29, 2026•0 citations
Abstract Feedback control of open quantum systems is of fundamental importance for practical applications in various contexts, ranging from quantum computation to quantum error correction and quantum metrology. Its use in the context of thermodynamics further enables the study of the interplay between information and energy. However, deriving optimal feedback control strategies is highly challenging, as it involves the optimal control of open quantum systems, the stochastic nature of quantum measurement, and the inclusion of policies that maximize a long-term time- and trajectory-averaged goal. In this work, we employ a reinforcement learning approach to automate and capture the role of a quantum Maxwell’s demon: the agent takes the literal role of discovering optimal feedback control strategies in qubit-based systems that maximize a trade-off between measurement-powered cooling and measurement efficiency. Considering weak or projective quantum measurements, we explore different regimes based on the ordering between the thermalization, the measurement, and the unitary feedback timescales, finding different and highly non-intuitive, yet interpretable, strategies. In the thermalization-dominated regime, we find strategies with elaborate finite-time thermalization protocols conditioned on measurement outcomes. In the measurement-dominated regime, we find that optimal strategies involve adaptively measuring different qubit observables reflecting the acquired information, and repeating multiple weak measurements until the quantum state is ‘sufficiently pure’, leading to random walks in state space. Finally, we study the case when all timescales are comparable, finding new feedback control strategies that considerably outperform more intuitive ones. We discuss a two-qubit example where we explore the role of entanglement and conclude discussing the scaling of our results to quantum many-body systems.
Maxwell's demon explores the role of information in physical processes.Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics.In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion.We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system.The demon is experimentally implemented as a spin-1=2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1=2 system, through a natural interaction.Noise and imperfections in this protocol are investigated by the assessment of its effectiveness.This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.
Maxwell's demon explores the role of information in physical processes. Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics. In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion. We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system. The demon is experimentally implemented as a spin-1/2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1/2 system, through a natural interaction. Noise and imperfections in this protocol are investigated by the assessment of its effectiveness. This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.
In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.
We investigated coupled-qubit-based thermal machines powered by quantum measurements and feedback. We considered two different versions of the machine: (1) a quantum Maxwell's demon, where the coupled-qubit system is connected to a detachable single shared bath, and (2) a measurement-assisted refrigerator, where the coupled-qubit system is in contact with a hot and cold bath. In the quantum Maxwell's demon case, we discuss both discrete and continuous measurements. We found that the power output from a single qubit-based device can be improved by coupling it to the second qubit. We further found that the simultaneous measurement of both qubits can produce higher net heat extraction compared to two setups operated in parallel where only single-qubit measurements are performed. In the refrigerator case, we used continuous measurement and unitary operations to power the coupled-qubit-based refrigerator. We found that the cooling power of a refrigerator operated with swap operations can be enhanced by performing suitable measurements.
The emerging field of quantum thermodynamics is beginning to reveal the intriguing role that information can play in quantum thermal engines. Information enters as a resource when considering feedback-controlled thermal machines. While both a general theory of quantum feedback control as well as specific examples of quantum feedback-controlled engines have been presented, still lacking is a general framework for such machines. Here, we present a framework for a generic, two-stroke quantum heat engine interacting with NNN thermal baths and Maxwell's demon. The demon performs projective measurements on the engine working substance, the outcome of which is recorded in a classical memory, embedded in its own thermal bath. To perform feedback control, the demon enacts unitary operations on the working substance, conditioned on the recorded outcome. By considering the compound machine-memory as a hybrid (classical-quantum) standard thermal machine interacting with N+1N+1N+1 thermal baths, our framework puts the working substance and memory on equal footing, thereby enabling a comprehensible resolution to Maxwell's paradox and elucidating the intricate manner in which information impacts the performance of quantum engines. We illustrate the application of our framework with a two-qubit engine. A remarkable observation is that more information does not necessarily result in better thermodynamic performance: sometimes knowing less is better.
This paper presents a novel feedback learning algorithm for adaptive quantum control. The algorithm leverages reinforcement learning principles combined with quantum control theory to dynamically adjust control strategies based on the observed state and environment of a quantum system. The core mechanism involves constructing an adaptive quantum control feedback learning algorithm that enables autonomous optimization of the quantum system. The algorithm utilizes a state representation of the quantum system, a reward function reflecting the desired outcome, and a control policy updated through a reinforcement learning process. Mathematical formulations are provided to describe the system dynamics, the control policy, and the learning process. The proposed algorithm aims to overcome the limitations of traditional fixed-parameter quantum control methods by adapting to uncertainties and complex dynamics, ultimately leading to improved control performance and robustness. The theoretical framework presented here offers a foundation for developing advanced quantum control strategies in various applications, including quantum computation, quantum sensing, and quantum metrology.