Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [E. H. Lieb and J. Yngvason Proc. R. Soc. 469, 20130408 (2013)]. In particular, we obtain a direct relation between the Clausius entropy and the Shannon entropy, or its generalization to quantum systems, the von Neumann entropy. More generally, we find that entropy measures relevant in nonequilibrium thermodynamics correspond to entropies used in one-shot information theory.
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [E. H. Lieb and J. Yngvason Proc. R. Soc. 469, 20130408 (2013)]. In particular, we obtain a direct relation between the Clausius entropy and the Shannon entropy, or its generalization to quantum systems, the von Neumann entropy. More generally, we find that entropy measures relevant in nonequilibrium thermodynamics correspond to entropies used in one-shot information theory.
The notion of entropy was introduced by Clausius in order to discuss the thermal behavior of physical systems. After the work of Clausius, Boltzmann, Gibbs and others, the beginning of the exact treatment of entropy was due to von Neumann and Shannon. Their starting point was very di erent, von Neumann was motivated by quantum mechanics and Shannon founded communication theory. The entropy of a state describing a physical system is a quantity expressing the uncertainty or randomness of the system. Shannon regarded this uncertainty attached to a physical system as the amount of information carried by the system, so that the entropy of a state can be read as the information carried by the state. His idea comes from the following consideration: If a physical system has a large uncertainty and one receives information on the system by some procedure, then the so obtained information is more valuable than that received from a system having less uncertainty. It is understood in probability theory that the notion of (Shannon or measure theoretic) entropy has successful applications in a variety of subjects because it determines the asymptotic behavior of certain probabilities in the course of independent trials. First we will discuss this phenomena for nite quantum systems. By a nite quantum system we mean an algebra of matrices which is stable under taking adjoint. (In other words, a nite quantum system is a nite dimensional C*algebra.) If A is such an algebra then there is a linear functional Tr which takes the value 1 at each minimal projection. It is "tracial " in the sense that Tr ab = Tr ba (a; b 2 A) Every functional! on A is determined by a density operator D! 2 A in the form!(a) = Tr D! a (a 2 A): The entropy S(!) of a functional! is de ned by means of its density operator as S(!) = Tr (D!): This notion was introduced by von Neumann in 1927 and we term it von Neumann's entropy or shortly entropy (cf. [1]). In fact, von Neumann de ned the entropy such
Quantum information-theoretic approach had been identified as a way to understand the foundations of quantum mechanics as early as 1950 due to Shannon. However, there hasn’t been enough advancement or rigorous development of the subject. In the following paper we try to find the relationship between a general quantum mechanical observable and von Neumann entropy. We find that the expectation values and the uncertainties of the observables have bounds which depend on the entropy. The results also show that von Neumann entropy is not just the uncertainty of the state but also encompasses the information about expectation values and uncertainties of any observable which depend on the observers choice for a particular measurement. Also a reverse uncertainty relation is derived for [Formula: see text] quantum mechanical observables.
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality (IC) recently proposed by Pawlowski et al (2009 arXiv:0905.2292). We consider two entropic quantities, which we term measurement and mixing entropy. In the context of classical and quantum theory, these coincide, being given by the Shannon and von Neumann entropies, respectively; in general, however, they are very different. In particular, while measurement entropy is easily seen to be concave, mixing entropy need not be. In fact, as we show, mixing entropy is not concave whenever the state space is a non-simplicial polytope. Thus, the condition that measurement and mixing entropies coincide is a strong constraint on possible theories. We call theories with this property monoentropic . Measurement entropy is subadditive, but not in general strongly subadditive. Equivalently, if we define the mutual information between two systems A and B by the usual formula I ( A : B )= H ( A )+ H ( B )- H ( AB ), where H denotes the measurement entropy and AB is a non-signaling composite of A and B , then it can happen that I ( A : BC )< I ( A : B ). This is relevant to IC in the sense of Pawlowski et al : we show that any monoentropic non-signaling theory in which measurement entropy is strongly subadditive, and also satisfies a version of the Holevo bound, is informationally causal, and on the other hand we observe that Popescu–Rohrlich boxes, which violate IC, also violate strong subadditivity. We also explore the interplay between measurement and mixing entropy and various natural conditions on theories that arise in quantum axiomatics.
The concept of entropy rst arose in the study of thermodynamics and statistical me- chanics. In information theory the so-called Shannon entropy plays a signi cant role, which measures how much information we gain, on average, when we learn the value of a random variable. An alternative interpretation is that the entropy of the random variable quanti es the amount of uncertainty about the random variable before we learn its value. The concept was introduced by Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication". In the rst part of the present work we consider this fundamental measure of information and show some of its properties, such as non-negativity. The relative entropy or the Kullback-Leibler divergence is a very useful measure of the distance between two prob- ability distributions. After introducing its de nition and other notions (joint entropy, conditional entropy or mutual information) we can prove additional assertions about the Shannon entropy and present statements about the relations between the above men- tioned quantities. In quantum information theory, there is a quantity, namely von Neumann entropy, which corresponds to Shannon entropy and originates from it. In the section Von Neu- mann entropy we will introduce this quantity which is a continuous functional on the states, or on their mathematical representatives, the density operators, and it ful ls in- equalities such as subadditivity and strong subaddititvity. Just as with the Shannon entropy, it is extremely useful to de ne a quantum version of the relative entropy, which is one of the most important numerical quantities appearing in quantum information theory. It is used as a measure of distinguishability between quantum states. Among others its properties are jointly convexity, monotonicity which are speci cally convenient. Transformations on structures which preserve operations, quantities or relations among elements appear in many areas of mathematics. These are the so-called preservers which show up also in physics and even in chemistry where they are usually called symmetries. Therefore, the study of preserver transformations is an important area of research. The third section of this work is based on a paper, in which Lajos Molnár described the structure every bijective map on the space of all density operators on a given Hilbert space, which preserves the relative entropy. This result is closely related to Wigner's famous theorem on the form of quantum mechanical symmetry transformations. The above theorem of Molnár can be extended into two directions. On the one hand, the result holds also in the case when the transformation under consideration acts on the space of all positive semide nite operators not only on density operators (i.e., we omit the condition of unit trace). On the other hand the statement remains valid even if we omit the strongly restrictive condition that the transformation is bijective. The second result is a joint work with Prof. Lajos Molnár which appeared in the journal Linear Algebra and Its Applications.
Entropy has played a pivotal role in physics ever since its inception by Clausius in the early industrial age. Shannon ushered the concept into the information age by using it to describe fundamental limits of information processing, founding the field of information theory. Today, a fruitful interaction between information theory and physics is pushing our understanding of entropy forward. It has revealed, in particular, that no single notion of entropy suffices to capture all relevant properties of physical systems, making it necessary to consider a variety of entropic measures of quantum information, each with distinct features and uses. This review surveys these recent advances in a structured way, covering a comprehensive array of information measures, including Rényi and smooth entropies, together with their interrelations. Starting with a systematic characterization of entropies that motivates their definitions from natural physical principles, we discuss their applications ranging from quantum communication and cryptography to resource theories, thermodynamics, and many-body physics.
We derive the quantum Fisher information for entropy estimation in a Gibbs state and show that it equals the inverse of the heat capacity, which is dual to the temperature Fisher information given by the heat capacity divided by the square of the temperature. Their product is independent of the Hamiltonian and depends only on the temperature, leading to a metrological uncertainty relation between the variances of entropy and temperature estimators in which all system-specific quantities cancel. This relation arises from the dually-flat structure of the Gibbs exponential family expressed in thermodynamic coordinates, and holds for all standard thermodynamically conjugate pairs. We identify energy measurement as the optimal protocol for entropy estimation, analyse critical-point scaling where the entropy Fisher information vanishes, and connect it to the Ruppeiner metric in entropy coordinates. We lastly examine the distinguished role of the von Neumann entropy within the Rényi family. Generalisations to the grand canonical and generalised Gibbs ensembles are given.