Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Memory as Bayesian inference: On the connection between memory and the second law of thermodynamics

Wade D. Hodson

Published Jan 1, 2016• 0 citations
OpenAlex

Abstract

A recent theoretical paper by Leonard Mlodinow and Todd Brun suggests that the functioning of physical records or "memories" is never accompanied by a decrease in entropy, meaning that all memories "align" with the thermodynamic arrow of time. In this thesis, we characterize a class of physical systems as memories in terms of inferences that can be made about the state of the world, given certain information about these systems. Tools from Bayesian probability theory are used to quantify the informativeness and reliability associated with such inferences. Based on consideration of two model systems, one classical and one quantum, we argue in favor of Mlodinow and Brun's claim that the functioning of memory systems is conditioned by thermodynamic constraints. For the classical model, we show that a memory which operates against the thermodynamic arrow, and thus "remembers" a relatively high-entropy state, is much less informative than a similar memory which aligns with the thermodynamic arrow. Our analysis of the quantum model, expressed in the density matrix formalism of quantum mechanics, allows us to consider the inferences that can be made when a quantum system is coupled to a simple type of quantum memory system. We ultimately show that these inferences can be expressed in terms of a probabilistic matrix completion problem.

Related papers

Growth of Knowledge and Entropy in Quantum Physics

Alan Forrester

Most attempts to argue for the second law of thermodynamics fail because (1) they use the unviable frequency theory of probability and (2) they do not explain why the arrow of time seen in experiments is aligned with the thermodynamic arrow of time. I use the decision theoretic interpretation of quantum probability from the many worlds interpretation of quantum mechanics to solve the probability problem. I then derive a correlation between the knowledge arrow of time and the entropy arrow of time using physical constraints on knowledge creation imposed by Popper's evolutionary theory of knowledge and the many worlds interpretation.

arXiv:0811.3609•Nov 21, 2008•0 citations

The Arrow of Time as a Physical Memory Constraint

yaniv riz

A physical observer experiences time through records. This paper derives the experienced arrow of time from the physical formation of memory records. Starting from an ordinary quantum state, each possible outcome carries a Born probability and a corresponding potential informational content before it is recorded. A measurement assigns one possible outcome to a physical record in the observer-apparatus system. The selected informational increment leaves potential status and appears as a stable record together with a dispersed thermodynamic term. The record-formation balance is written as a local relation between potential information, recorded information, and dispersed information. The dispersed term is expressed through environmental entropy, and Landauer’s principle bounds the irreversible component of record stabilization. In rate form, the same relation becomes a power balance between record-writing rate and dissipated thermal power. Under complete reversal of the observer-record-environment system, the record and the dispersed term reverse together. The recorded increment returns to potential status, and the observer’s memory of that increment is removed. Therefore a physical observer does not experience reversed time as memory of the future. The experienced temporal direction is the direction in which records are physically available to the observer. Entropy supplies the thermodynamic cost of record formation; memory supplies the physical access to temporal direction.

•May 15, 2026•0 citations

The Backwards Arrow of Time of the Coherently Bayesian Statistical Mechanic

Cosma Rohilla Shalizi

Many physicists think that the maximum entropy formalism is a straightforward application of Bayesian statistical ideas to statistical mechanics. Some even say that statistical mechanics is just the general Bayesian logic of inductive inference applied to large mechanical systems. This approach identifies thermodynamic entropy with the information-theoretic uncertainty of an (ideal) observer's subjective distribution over a system's microstates. In this brief note, I show that this postulate, plus the standard Bayesian procedure for updating probabilities, implies that the entropy of a classical system is monotonically non-increasing on the average -- the Bayesian statistical mechanic's arrow of time points backwards. Avoiding this unphysical conclusion requires rejecting the ordinary equations of motion, or practicing an incoherent form of statistical inference, or rejecting the identification of uncertainty and thermodynamic entropy.

arXiv:cond-mat/0410063•Oct 4, 2004•7 citations

A Quantum of Thermodynamics

Henrik Wilming

To understand in detail the relation between unitary quantum theory that describes our world at the microscopic scale and thermodynamics, which was long believed only to apply to macroscopic objects, is one of the most interesting and long-standing problems in physics. Recently, this problem has received renewed attention, in particular from the community of quantum information theory, but also from the field of statistical mechanics, inspired from stochastic thermodynamics. These results suggest that thermodynamics is also relevant for individual quantum systems, provided that they can be brought into contact with thermal baths. In this thesis, I use recently developed tools to provide new results both on fundamental questions, but also on questions which are of practical relevance for potential miniaturized thermal machines. In terms of fundamental questions, the results in this thesis contribute to understanding how the basic laws of thermodynamics and statistical mechanics can be understood directly from unitary quantum mechanics. In particular, I discuss and answer the following questions: i) How can we quantify the third law of thermodynamics using information theoretic methods? ii) How can we quantify the "thermodynamic value" of a state-transition in quantum systems? iii) How can we axiomatically characterize the non-equilibrium free energy and relative entropy? iv) How can we justify statistical ensembles from an operational perspective, without having to introduce either some probability measures or an information theoretic entropy measure? v) How can we understand the equilibration of closed quantum systems, how long does it take and how difficult is it to avoid? In the second part of the thesis I discuss in detail how experimental restrictions, which become important at the quantum scale, influence the ultimate thermodynamic bounds for thermal machines. In particular, the results in this thesis provide thermodynamic bounds on work-extraction and efficiencies of thermal machines in situations where 1.) an experimenter only has limited field strengths available, 2.) an experimenter cannot control the interactions between particles, but external fields arbitrarily well, and 3.) situations in which a small quantum system can only be strongly coupled to heat baths. These bounds are tight and I provide explicit examples illustrating the different behaviours. Finally, I come back to a classic problem in statistical physics: The emergence of spontaneous symmetry breaking. Here, I provide general and rigorous new results that show how symmetry-breaking stationary states emerge from fluctuations in order parameters in dissipative lattice models.

•Jan 1, 2018•0 citations

Probabilistic Foundations of Statistical Mechanics: A Bayesian Approach

Brian H Buck, A. C. Merchant

We examine the fundamental aspects of statistical mechanics, dividing the problem into a discussion purely about probability, which we analyse from a Bayesian standpoint. We argue that the existence of a unique maximising probability distribution $\{p(j\vert K)\}$ for states labelled by $j$ given data $K$ implies that the corresponding maximal value of the information entropy $σ(\{(p_j\vert K)\}) = -\sum_j (p_j \vert K)\ln{(p_j\vert K)}$ depends explicitly on the data at equilibrium and on the Hamiltonian of the system. As such, it is a direct measure of our uncertainty about the exact state of the body and can be identified with the traditional thermodynamic entropy. We consider the well known microcanonical, canonical and grand canonical methods and ensure that the fluctuations about mean values are generally minuscule for macroscopic systems before identifying these mean values with experimental observables and thereby connecting to many standard results from thermodynamics. Unexpectedly, we find that it is not generally possible for a quantum process to be both isentropic and reversibly adiabatic. This is in sharp contrast to traditional thermodynamics where it is assumed that isentropic, reversible adiabatic processes can be summoned up on demand and easily realised. By contrast, we find that linear relations between pressures $P_j$ and energies $E_j$ are necessary and sufficient conditions for a quasi-static and adiabatic change to be isentropic, but, of course, this relationship only holds for a few especially simple systems, such as the perfect gas, and is not generally true for more complicated systems. By considering the associated entropy increases up to second order in small volume changes we argue that the consequences are in practice negligible.

arXiv:1512.01368•Dec 4, 2015•2 citations

Thermodynamics and the Structure of Quantum Theory as a Generalized Probabilistic Theory

Marius Krumm

This thesis investigates the connection between quantum theory, thermodynamics and information theory. Theories with structure similar to that of quantum theory are considered, mathematically described by the framework of "Generalized Probabilistic Theories". For these theories, a thought experiment by von Neumann is adapted to obtain a natural thermodynamic entropy definition, following a proposal by J. Barrett. Mathematical properties of this entropy are compared to physical consequences of the thought experiment. The validity of the second law of thermodynamics is investigated. In that context, observables and projective measurements are generalized to prove an entropy increase for projective measurements of ensembles. Information-theoretically motivated definitions of the entropy are compared to the entropy from the thermodynamic thought experiment. The conditions for the thermodynamic entropy to be well-defined are considered in greater detail. Several further properties of the theories under consideration (e.g. whether there is higher order interference, Pfister's state discrimination principle) and their relation to entropy are investigated.

arXiv:1508.03299•Aug 13, 2015•11 citations
A recent theoretical paper by Leonard Mlodinow and Todd Brun suggests that the functioning of physical records or "memories" is never accompanied by a decrease in entropy, meaning that all memories "align" with the thermodynamic arrow of time. In this thesis, we characterize a class of physical systems as memories in terms of inferences that can be made about the state of the world, given certain information about these systems. Tools from Bayesian probability theory are used to quantify the informativeness and reliability associated with such inferences. Based on consideration of two model systems, one classical and one quantum, we argue in favor of Mlodinow and Brun's claim that the functioning of memory systems is conditioned by thermodynamic constraints. For the classical model, we show that a memory which operates against the thermodynamic arrow, and thus "remembers" a relatively high-entropy state, is much less informative than a similar memory which aligns with the thermodynamic arrow. Our analysis of the quantum model, expressed in the density matrix formalism of quantum mechanics, allows us to consider the inferences that can be made when a quantum system is coupled to a simple type of quantum memory system. We ultimately show that these inferences can be expressed in terms of a probabilistic matrix completion problem.
Most attempts to argue for the second law of thermodynamics fail because (1) they use the unviable frequency theory of probability and (2) they do not explain why the arrow of time seen in experiments is aligned with the thermodynamic arrow of time. I use the decision theoretic interpretation of quantum probability from the many worlds interpretation of quantum mechanics to solve the probability problem. I then derive a correlation between the knowledge arrow of time and the entropy arrow of time using physical constraints on knowledge creation imposed by Popper's evolutionary theory of knowledge and the many worlds interpretation.
A physical observer experiences time through records. This paper derives the experienced arrow of time from the physical formation of memory records. Starting from an ordinary quantum state, each possible outcome carries a Born probability and a corresponding potential informational content before it is recorded. A measurement assigns one possible outcome to a physical record in the observer-apparatus system. The selected informational increment leaves potential status and appears as a stable record together with a dispersed thermodynamic term. The record-formation balance is written as a local relation between potential information, recorded information, and dispersed information. The dispersed term is expressed through environmental entropy, and Landauer’s principle bounds the irreversible component of record stabilization. In rate form, the same relation becomes a power balance between record-writing rate and dissipated thermal power. Under complete reversal of the observer-record-environment system, the record and the dispersed term reverse together. The recorded increment returns to potential status, and the observer’s memory of that increment is removed. Therefore a physical observer does not experience reversed time as memory of the future. The experienced temporal direction is the direction in which records are physically available to the observer. Entropy supplies the thermodynamic cost of record formation; memory supplies the physical access to temporal direction.
Many physicists think that the maximum entropy formalism is a straightforward application of Bayesian statistical ideas to statistical mechanics. Some even say that statistical mechanics is just the general Bayesian logic of inductive inference applied to large mechanical systems. This approach identifies thermodynamic entropy with the information-theoretic uncertainty of an (ideal) observer's subjective distribution over a system's microstates. In this brief note, I show that this postulate, plus the standard Bayesian procedure for updating probabilities, implies that the entropy of a classical system is monotonically non-increasing on the average -- the Bayesian statistical mechanic's arrow of time points backwards. Avoiding this unphysical conclusion requires rejecting the ordinary equations of motion, or practicing an incoherent form of statistical inference, or rejecting the identification of uncertainty and thermodynamic entropy.
To understand in detail the relation between unitary quantum theory that describes our world at the microscopic scale and thermodynamics, which was long believed only to apply to macroscopic objects, is one of the most interesting and long-standing problems in physics. Recently, this problem has received renewed attention, in particular from the community of quantum information theory, but also from the field of statistical mechanics, inspired from stochastic thermodynamics. These results suggest that thermodynamics is also relevant for individual quantum systems, provided that they can be brought into contact with thermal baths. In this thesis, I use recently developed tools to provide new results both on fundamental questions, but also on questions which are of practical relevance for potential miniaturized thermal machines. In terms of fundamental questions, the results in this thesis contribute to understanding how the basic laws of thermodynamics and statistical mechanics can be understood directly from unitary quantum mechanics. In particular, I discuss and answer the following questions: i) How can we quantify the third law of thermodynamics using information theoretic methods? ii) How can we quantify the "thermodynamic value" of a state-transition in quantum systems? iii) How can we axiomatically characterize the non-equilibrium free energy and relative entropy? iv) How can we justify statistical ensembles from an operational perspective, without having to introduce either some probability measures or an information theoretic entropy measure? v) How can we understand the equilibration of closed quantum systems, how long does it take and how difficult is it to avoid? In the second part of the thesis I discuss in detail how experimental restrictions, which become important at the quantum scale, influence the ultimate thermodynamic bounds for thermal machines. In particular, the results in this thesis provide thermodynamic bounds on work-extraction and efficiencies of thermal machines in situations where 1.) an experimenter only has limited field strengths available, 2.) an experimenter cannot control the interactions between particles, but external fields arbitrarily well, and 3.) situations in which a small quantum system can only be strongly coupled to heat baths. These bounds are tight and I provide explicit examples illustrating the different behaviours. Finally, I come back to a classic problem in statistical physics: The emergence of spontaneous symmetry breaking. Here, I provide general and rigorous new results that show how symmetry-breaking stationary states emerge from fluctuations in order parameters in dissipative lattice models.
We examine the fundamental aspects of statistical mechanics, dividing the problem into a discussion purely about probability, which we analyse from a Bayesian standpoint. We argue that the existence of a unique maximising probability distribution {p(j∣K)}\{p(j\vert K)\}{p(j∣K)} for states labelled by jjj given data KKK implies that the corresponding maximal value of the information entropy σ({(pj∣K)})=−∑j(pj∣K)ln⁡(pj∣K)σ(\{(p_j\vert K)\}) = -\sum_j (p_j \vert K)\ln{(p_j\vert K)}σ({(pj​∣K)})=−∑j​(pj​∣K)ln(pj​∣K) depends explicitly on the data at equilibrium and on the Hamiltonian of the system. As such, it is a direct measure of our uncertainty about the exact state of the body and can be identified with the traditional thermodynamic entropy. We consider the well known microcanonical, canonical and grand canonical methods and ensure that the fluctuations about mean values are generally minuscule for macroscopic systems before identifying these mean values with experimental observables and thereby connecting to many standard results from thermodynamics. Unexpectedly, we find that it is not generally possible for a quantum process to be both isentropic and reversibly adiabatic. This is in sharp contrast to traditional thermodynamics where it is assumed that isentropic, reversible adiabatic processes can be summoned up on demand and easily realised. By contrast, we find that linear relations between pressures PjP_jPj​ and energies EjE_jEj​ are necessary and sufficient conditions for a quasi-static and adiabatic change to be isentropic, but, of course, this relationship only holds for a few especially simple systems, such as the perfect gas, and is not generally true for more complicated systems. By considering the associated entropy increases up to second order in small volume changes we argue that the consequences are in practice negligible.
This thesis investigates the connection between quantum theory, thermodynamics and information theory. Theories with structure similar to that of quantum theory are considered, mathematically described by the framework of "Generalized Probabilistic Theories". For these theories, a thought experiment by von Neumann is adapted to obtain a natural thermodynamic entropy definition, following a proposal by J. Barrett. Mathematical properties of this entropy are compared to physical consequences of the thought experiment. The validity of the second law of thermodynamics is investigated. In that context, observables and projective measurements are generalized to prove an entropy increase for projective measurements of ensembles. Information-theoretically motivated definitions of the entropy are compared to the entropy from the thermodynamic thought experiment. The conditions for the thermodynamic entropy to be well-defined are considered in greater detail. Several further properties of the theories under consideration (e.g. whether there is higher order interference, Pfister's state discrimination principle) and their relation to entropy are investigated.