How much work can an agent extract from a temporal sequence of quantum states when it can only operate online under causal constraints---deciding which energy extraction method to use with knowledge of what it has observed before? Here, we study this problem in the context of quantum state sequences that are potentially non-Markovian---generated by some underlying hidden Markov machine that the agent cannot observe. Using techniques from dynamic programming and computational mechanics, we present a method to identify the provably optimal agent strategy, with time complexity that scales linearly with sequence length. This motivates us to introduce the maximum work such agents can extract---time-ordered free energy(TOFE)---as a fundamental measure of free energy available in a temporally correlated quantum system subject to causal considerations.
How much work can an agent extract from a temporal sequence of quantum states when it can only operate online under causal constraints---deciding which energy extraction method to use with knowledge of what it has observed before? Here, we study this problem in the context of quantum state sequences that are potentially non-Markovian---generated by some underlying hidden Markov machine that the agent cannot observe. Using techniques from dynamic programming and computational mechanics, we present a method to identify the provably optimal agent strategy, with time complexity that scales linearly with sequence length. This motivates us to introduce the maximum work such agents can extract---time-ordered free energy(TOFE)---as a fundamental measure of free energy available in a temporally correlated quantum system subject to causal considerations.
This thesis develops a decision-theoretic framework for extracting thermodynamic work from temporal correlations in quantum systems. We model a classical agent -- lacking quantum memory -- performing adaptive work extraction through continuous inference and decision-making under uncertainty. By introducing ρ∗-ideal protocols, we demonstrate that exploiting memory effects allows adaptive strategies to surpass non-adaptive bounds. We formalize this via the Time-Ordered Free Energy (TOFE), a novel upper bound for causal, adaptive operations that reveals a thermodynamic gap linked to adaptive ordered discord. Additionally, we tackle work extraction from unknown sources using reinforcement learning. By adapting multi-armed bandit algorithms, we show an agent can simultaneously learn an unknown i.i.d. quantum state and extract work, achieving polylogarithmic cumulative dissipation that significantly outperforms standard tomography. Overall, this work lays the foundation for predictive and learning-based quantum thermodynamics.
In reinforcement learning, an agent interacts sequentially with an environment to maximize a reward, receiving only partial, probabilistic feedback. This creates a fundamental exploration-exploitation trade-off: the agent must explore to learn the hidden dynamics while exploiting this knowledge to maximize its target objective. While extensively studied classically, applying this framework to quantum systems requires dealing with hidden quantum states that evolve via unknown dynamics. We formalize this problem via a framework where the environment maintains a hidden quantum memory evolving via unknown quantum channels, and the agent intervenes sequentially using quantum instruments. For this setting, we adapt an optimistic maximum-likelihood estimation algorithm. We extend the analysis to continuous action spaces, allowing us to model general positive operator-valued measures (POVMs). By controlling the propagation of estimation errors through quantum channels and instruments, we prove that the cumulative regret of our strategy scales as O(K) over K episodes. Furthermore, via a reduction to the multi-armed quantum bandit problem, we establish information-theoretic lower bounds demonstrating that this sublinear scaling is strictly optimal up to polylogarithmic factors. As a physical application, we consider state-agnostic work extraction. When extracting free energy from a sequence of non-i.i.d. quantum states correlated by a hidden memory, any lack of knowledge about the source leads to thermodynamic dissipation. In our setting, the mathematical regret exactly quantifies this cumulative dissipation. Using our adaptive algorithm, the agent uses past energy outcomes to improve its extraction protocol on the fly, achieving sublinear cumulative dissipation, and, consequently, an asymptotically zero dissipation rate.
Evaluating the maximum amount of work extractable from a nanoscale quantum system is one of the central problems in quantum thermodynamics. Previous works identified the free energy of the input state as the optimal rate of extractable work under the crucial assumption: experimenters know the description of the given quantum state, which restricts the applicability to significantly limited settings. Here, we show that this optimal extractable work can be achieved without knowing the input states at all, removing the aforementioned fundamental operational restrictions. We achieve this by presenting the construction of a quantum channel whose description does not depend on input states but nevertheless extracts work quantified by the free energy of the unknown input state. Remarkably, our result partially encompasses the case of infinite-dimensional systems, for which optimal extractable work has not been known even for the standard state-aware setting. Our results clarify that, even though whether the description of the given state is provided at the beginning of the protocol generally makes the operational setting fundamentally different in accomplishing information-theoretic tasks, not knowing the input state does not influence the optimal performance of the asymptotic work extraction. The maximum work extracted from independent and identically distributed quantum states is characterised only in the state-aware scenario, and it remains unknown with no prior knowledge of the state. Here, the authors show that optimal work extraction can be achieved by a single operation, referred as universal work extraction.
Quantum information-processing techniques enable work extraction from a system's inherently quantum features, in addition to the classical free energy it contains. Meanwhile, the science of computational mechanics affords tools for the predictive modeling of non-Markovian classical and quantum stochastic processes. We combine tools from these two sciences to develop a technique for predictive work extraction from non-Markovian stochastic processes with quantum outputs. We demonstrate that this technique can extract more work than non-predictive quantum work extraction protocols, on the one hand, and predictive work extraction without quantum information processing, on the other. We discover a phase transition in the efficacy of memory for work extraction from quantum processes, which is without classical precedent. Our work opens up the prospect of machines that harness environmental free energy in an essentially quantum, essentially time-varying form.
We address the problem of quantifying the non-markovian character of quantum time evolutions of general systems in contact with an environment. We introduce two different measures of non-markovianity that exploit the specific traits of quantum correlations and are suitable for opposite experimental contexts. When complete tomographic knowledge about the evolution is available, our measure provides a necessary and sufficient condition to quantify strictly the non-markovianity. In the opposite case, when no information whatsoever is available, we propose a sufficient condition for non-markovianity. Remarkably, no optimization procedure underlies our derivation, which greatly enhances the practical relevance of the proposed criteria.
Maximum quantum work extraction is generally defined in terms of the ergotropy functional, no matter how experimentally complicated is the implementation of the optimal unitary allowing for it, especially in the case of multipartite systems. In this framework, we consider a quantum battery made up of many interacting subsystems and study the maximum extractable work via concurrent local unitary operations on each subsystem. We call the resulting functional parallel ergotropy. Focusing on the bipartite case, we first observe that parallel ergotropy outperforms work extraction via egoistic strategies, in which the first agent A extracts locally on its part the maximum available work and the second agent B, subsequently, extracts what is left on the other part. For the agents, this showcases the need of cooperating for an overall benefit. Second, from the informational point of view, we observe that the parallel capacity of a state can detect entanglement and compare it with the statistical entanglement witness that exploits fluctuations of stochastic work extraction. Additionally, we face the technical problem of computing parallel ergotropy. We derive analytical upper bounds for specific classes of states and Hamiltonians and provide recipes to obtain numerical upper bounds via semidefinite programming in the generic case. Finally, extending the concept of parallel ergotropy, we demonstrate that system's free-time evolution and application of local unitaries allow one to saturate the gap with the ergotropy of the whole system.