Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Symposium Honoring Mathematical Physicist Jean-Pierre Vigier

Information, Entropy, and the Classical Ideal Gas

David Sands, Jeremy Dunning-Davies

Published Oct 16, 2013• 0 citations
Publisher

Abstract

The physical basis of the canonical and grand canonical distributions is questioned. In particular, we question the usual methods by which these distributions are derived, namely that fluctuations in entropy around energy and particle number are assumed to occur when the entropy depends only on variables which cannot themselves fluctuate. We show, starting from the Maxwellian velocity distribution, that the probability that a classical ideal gas at a fixed temperature occupies a given energy state corresponds not to the canonical ensemble of classical statistical mechanics but to the Gamma distribution. Computer simulations of a hard-sphere fluid demonstrate the principles. The analysis is extended to open systems in which the number of particles fluctuates and we show that for a system connected to a particle reservoir the Poisson distribution governs the probability of finding a given number of particles. The resulting probability distributions are used to calculate the Shannon information entropy which is then compared with the thermodynamic entropy. We argue that information theoretic entropy and thermodynamic entropy, whilst related, are not necessarily identical and that the information entropy contains non-thermodynamic components.

DOI
10.1142/9789814504782_0024

Related papers

Probabilistic Foundations of Statistical Mechanics: A Bayesian Approach

Brian H Buck, A. C. Merchant

We examine the fundamental aspects of statistical mechanics, dividing the problem into a discussion purely about probability, which we analyse from a Bayesian standpoint. We argue that the existence of a unique maximising probability distribution $\{p(j\vert K)\}$ for states labelled by $j$ given data $K$ implies that the corresponding maximal value of the information entropy $σ(\{(p_j\vert K)\}) = -\sum_j (p_j \vert K)\ln{(p_j\vert K)}$ depends explicitly on the data at equilibrium and on the Hamiltonian of the system. As such, it is a direct measure of our uncertainty about the exact state of the body and can be identified with the traditional thermodynamic entropy. We consider the well known microcanonical, canonical and grand canonical methods and ensure that the fluctuations about mean values are generally minuscule for macroscopic systems before identifying these mean values with experimental observables and thereby connecting to many standard results from thermodynamics. Unexpectedly, we find that it is not generally possible for a quantum process to be both isentropic and reversibly adiabatic. This is in sharp contrast to traditional thermodynamics where it is assumed that isentropic, reversible adiabatic processes can be summoned up on demand and easily realised. By contrast, we find that linear relations between pressures $P_j$ and energies $E_j$ are necessary and sufficient conditions for a quasi-static and adiabatic change to be isentropic, but, of course, this relationship only holds for a few especially simple systems, such as the perfect gas, and is not generally true for more complicated systems. By considering the associated entropy increases up to second order in small volume changes we argue that the consequences are in practice negligible.

arXiv:1512.01368•Dec 4, 2015•2 citations

Occupancy Statistics and Entropy in Bose Systems

A. Spalvieri

In this work, we compare three formulations of thermodynamic entropy for a non-interacting bosonic gas: (i) the grand-canonical Bose-Einstein entropy, (ii) the finite-N canonical entropy obtained from the exact partition function (Ziff, Uhlenbeck, and Kac construction), and (iii) the entropy of a multinomial distribution with Boltzmann categorical probabilities and temperature determined from Clausius' equation. It is well-known that the grand-canonical Bose-Einstein systematically overestimates entropy of canonical systems in regimes where particle-number fluctuations are significant. The exact canonical entropy correctly enforces the particle-number constraint, but recent experimental results suggest that it also overestimates particle-number fluctuations below the crossover temperature. The multinomial distribution is less common in thermodynamics. It addresses in a mathematically exact way the puzzle of the famous -log(N!) term introduced by Gibbs as a deus ex machina in discussions of thermodynamic entropy. One remarkable consequence is that the entropy of the multinomial distribution overcomes the issue of negative entropy at low temperature that affects the Gibbs and the Sackur-Tetrode entropies at low temperature. The analysis presented in the paper shows that the multinomial distribution, equipped with a categorical distribution calibrated in such a way that the resulting multinomial entropy fits Clausius' equation, provides accurate approximations to the canonical entropy in the classical regime, while it is smaller than the canonical entropy below the crossover temperature. One feature of the multinomial distribution is that it predicts lower peak variance of the number of particles in the ground state than the canonical distribution. This is in agreement with recent experimental results; hence, this paper identifies the thermodynamically calibrated multinomial distribution as a candidate alternative to the canonical distribution for thermodynamic bosonic entropy in finite systems.

Entropy•Aug 13, 2026•0 citations

The Exact Entropy Formula of the Ideal Gas and its Information-Theoretic Interpretation

A. Spalvieri

The paper analyzes the entropy of a system composed by non-interacting and indistinguishable particles whose quantum state numbers are modelled as independent and identically distributed classical random variables. The crucial observation is that, under this assumption, whichever is the number of particles that constitute the system, the occupancy numbers of system's quantum (micro)states are multinomially distributed. This observation leads to an entropy formula for the physical system, which is nothing else than the entropy formula of the multinomial distribution, for which we claim novelty, in the sense that it is proposed here for the first time that the entropy of the multinomial distribution is the entropy of the physical system. The entropy formula of the multinomial distribution unveils yet unexplored connections between information theory and statistical mechanics, among which we mention the connection between conditional entropy of the random microstate given the random occupancy numbers and the Boltzmann-Planck entropy $\log(W)$ and between these two and the Gibbs correction term $\log(N!)$, thermalization and communication-theoretic preparation of a thermal state, accessible information of the thermal state and physical entropy of the thermalized system. A noticeable specific result that descends from our approach is the exact quantum correction to the textbook Sackur-Tetrode formula for the entropy of an ideal gas at the thermal equilibrium in a container.

arXiv:2305.04652•May 8, 2023•1 citation

Statistical Mechanics from a Quantum Perspective

Roland Kjellander

An energy level consists of all microscopic states of a system with the same energy. The energy levels of a system depend on the number of particles and the volume. At each instance in time, many different quantum states are represented in the various systems in the ensemble. A microcanonical ensemble describes an isolated system which has a fixed energy, volume and number of particles. The chapter describes certain relationships between fluctuations in thermodynamical variables and some other thermodynamical entities. One case of particular interest is when each subsystem is composed of one single particle, for instance very weakly interacting particles adsorbed on a surface or the particles in an ideal gas. The chapter presents more strict derivations of the probability expressions in the canonical, isobaric-isothermal and grand canonical ensembles. Volume and number of particles can also be considered as being mechanical variables. Entropy and temperature are examples of nonmechanical variables.

CRC Press eBooks•Jul 30, 2019•0 citations

Gases in two dimensions: universal thermodynamics and its consequences

Dragoş-Victor Anghel

I discuss ideal and interacting quantum gases obeying general fractional exclusion statistics. For systems with constant density of single-particle states, described in the mean field approximation, the entropy depends neither on the microscopic exclusion statistics, nor on the interaction. Such systems are called thermodynamically equivalent and I show that the microscopic reason for this equivalence is a one-to-one correspondence between the excited states of these systems. This provides a method, different from the bosonization technique, of transforming between systems of different exclusion statistics. In the last section the macroscopic aspects of this method are discussed. In appendix A, I calculate the fluctuation of the ground-state population of a condensed Bose gas in a grand-canonical ensemble and mean field approximation, while in appendix B I show a situation where although the system exhibits fractional exclusion properties on microscopic energy intervals, a rigorous calculation of the population of single-particle states reveals a condensation phenomenon. This also implies a malfunction of the usual and simplified calculation technique of the most probable statistical distributions.

arXiv:cond-mat/0105089•Journal of Physics A Mathematical and General•Aug 13, 2002•28 citations

Direct Link Between Energy and Information via Fisher’s Measure

F. Pennini, Angelo Plastino

Energy and information are fundamental concepts that play crucial roles in the foundations of science, each contributing to different aspects of our understanding of the natural world. Here we encounter that Fisher' measure is able to link them. We also investigate some interesting relations between Fisher's information, Poisson's distribution, and thermal quantifiers in a grand canonical ensemble. These connections permit one to visualize interesting novel artifacts in statistical mechanics. We clearly see them emerge in discussing the ideal gas. For example, we encounter that the Poisson distribution is intrinsically involved in the physics of the ideal gas, and we are able to give a Fisher-estimation of the difference in informational content between the grand canonical and canonical ensembles.

•Feb 7, 2024•0 citations
The physical basis of the canonical and grand canonical distributions is questioned. In particular, we question the usual methods by which these distributions are derived, namely that fluctuations in entropy around energy and particle number are assumed to occur when the entropy depends only on variables which cannot themselves fluctuate. We show, starting from the Maxwellian velocity distribution, that the probability that a classical ideal gas at a fixed temperature occupies a given energy state corresponds not to the canonical ensemble of classical statistical mechanics but to the Gamma distribution. Computer simulations of a hard-sphere fluid demonstrate the principles. The analysis is extended to open systems in which the number of particles fluctuates and we show that for a system connected to a particle reservoir the Poisson distribution governs the probability of finding a given number of particles. The resulting probability distributions are used to calculate the Shannon information entropy which is then compared with the thermodynamic entropy. We argue that information theoretic entropy and thermodynamic entropy, whilst related, are not necessarily identical and that the information entropy contains non-thermodynamic components.
We examine the fundamental aspects of statistical mechanics, dividing the problem into a discussion purely about probability, which we analyse from a Bayesian standpoint. We argue that the existence of a unique maximising probability distribution {p(j∣K)}\{p(j\vert K)\}{p(j∣K)} for states labelled by jjj given data KKK implies that the corresponding maximal value of the information entropy σ({(pj∣K)})=−∑j(pj∣K)ln⁡(pj∣K)σ(\{(p_j\vert K)\}) = -\sum_j (p_j \vert K)\ln{(p_j\vert K)}σ({(pj​∣K)})=−∑j​(pj​∣K)ln(pj​∣K) depends explicitly on the data at equilibrium and on the Hamiltonian of the system. As such, it is a direct measure of our uncertainty about the exact state of the body and can be identified with the traditional thermodynamic entropy. We consider the well known microcanonical, canonical and grand canonical methods and ensure that the fluctuations about mean values are generally minuscule for macroscopic systems before identifying these mean values with experimental observables and thereby connecting to many standard results from thermodynamics. Unexpectedly, we find that it is not generally possible for a quantum process to be both isentropic and reversibly adiabatic. This is in sharp contrast to traditional thermodynamics where it is assumed that isentropic, reversible adiabatic processes can be summoned up on demand and easily realised. By contrast, we find that linear relations between pressures PjP_jPj​ and energies EjE_jEj​ are necessary and sufficient conditions for a quasi-static and adiabatic change to be isentropic, but, of course, this relationship only holds for a few especially simple systems, such as the perfect gas, and is not generally true for more complicated systems. By considering the associated entropy increases up to second order in small volume changes we argue that the consequences are in practice negligible.
In this work, we compare three formulations of thermodynamic entropy for a non-interacting bosonic gas: (i) the grand-canonical Bose-Einstein entropy, (ii) the finite-N canonical entropy obtained from the exact partition function (Ziff, Uhlenbeck, and Kac construction), and (iii) the entropy of a multinomial distribution with Boltzmann categorical probabilities and temperature determined from Clausius' equation. It is well-known that the grand-canonical Bose-Einstein systematically overestimates entropy of canonical systems in regimes where particle-number fluctuations are significant. The exact canonical entropy correctly enforces the particle-number constraint, but recent experimental results suggest that it also overestimates particle-number fluctuations below the crossover temperature. The multinomial distribution is less common in thermodynamics. It addresses in a mathematically exact way the puzzle of the famous -log(N!) term introduced by Gibbs as a deus ex machina in discussions of thermodynamic entropy. One remarkable consequence is that the entropy of the multinomial distribution overcomes the issue of negative entropy at low temperature that affects the Gibbs and the Sackur-Tetrode entropies at low temperature. The analysis presented in the paper shows that the multinomial distribution, equipped with a categorical distribution calibrated in such a way that the resulting multinomial entropy fits Clausius' equation, provides accurate approximations to the canonical entropy in the classical regime, while it is smaller than the canonical entropy below the crossover temperature. One feature of the multinomial distribution is that it predicts lower peak variance of the number of particles in the ground state than the canonical distribution. This is in agreement with recent experimental results; hence, this paper identifies the thermodynamically calibrated multinomial distribution as a candidate alternative to the canonical distribution for thermodynamic bosonic entropy in finite systems.
The paper analyzes the entropy of a system composed by non-interacting and indistinguishable particles whose quantum state numbers are modelled as independent and identically distributed classical random variables. The crucial observation is that, under this assumption, whichever is the number of particles that constitute the system, the occupancy numbers of system's quantum (micro)states are multinomially distributed. This observation leads to an entropy formula for the physical system, which is nothing else than the entropy formula of the multinomial distribution, for which we claim novelty, in the sense that it is proposed here for the first time that the entropy of the multinomial distribution is the entropy of the physical system. The entropy formula of the multinomial distribution unveils yet unexplored connections between information theory and statistical mechanics, among which we mention the connection between conditional entropy of the random microstate given the random occupancy numbers and the Boltzmann-Planck entropy log⁡(W)\log(W)log(W) and between these two and the Gibbs correction term log⁡(N!)\log(N!)log(N!), thermalization and communication-theoretic preparation of a thermal state, accessible information of the thermal state and physical entropy of the thermalized system. A noticeable specific result that descends from our approach is the exact quantum correction to the textbook Sackur-Tetrode formula for the entropy of an ideal gas at the thermal equilibrium in a container.
An energy level consists of all microscopic states of a system with the same energy. The energy levels of a system depend on the number of particles and the volume. At each instance in time, many different quantum states are represented in the various systems in the ensemble. A microcanonical ensemble describes an isolated system which has a fixed energy, volume and number of particles. The chapter describes certain relationships between fluctuations in thermodynamical variables and some other thermodynamical entities. One case of particular interest is when each subsystem is composed of one single particle, for instance very weakly interacting particles adsorbed on a surface or the particles in an ideal gas. The chapter presents more strict derivations of the probability expressions in the canonical, isobaric-isothermal and grand canonical ensembles. Volume and number of particles can also be considered as being mechanical variables. Entropy and temperature are examples of nonmechanical variables.
I discuss ideal and interacting quantum gases obeying general fractional exclusion statistics. For systems with constant density of single-particle states, described in the mean field approximation, the entropy depends neither on the microscopic exclusion statistics, nor on the interaction. Such systems are called thermodynamically equivalent and I show that the microscopic reason for this equivalence is a one-to-one correspondence between the excited states of these systems. This provides a method, different from the bosonization technique, of transforming between systems of different exclusion statistics. In the last section the macroscopic aspects of this method are discussed. In appendix A, I calculate the fluctuation of the ground-state population of a condensed Bose gas in a grand-canonical ensemble and mean field approximation, while in appendix B I show a situation where although the system exhibits fractional exclusion properties on microscopic energy intervals, a rigorous calculation of the population of single-particle states reveals a condensation phenomenon. This also implies a malfunction of the usual and simplified calculation technique of the most probable statistical distributions.
Energy and information are fundamental concepts that play crucial roles in the foundations of science, each contributing to different aspects of our understanding of the natural world. Here we encounter that Fisher' measure is able to link them. We also investigate some interesting relations between Fisher's information, Poisson's distribution, and thermal quantifiers in a grand canonical ensemble. These connections permit one to visualize interesting novel artifacts in statistical mechanics. We clearly see them emerge in discussing the ideal gas. For example, we encounter that the Poisson distribution is intrinsically involved in the physics of the ideal gas, and we are able to give a Fisher-estimation of the difference in informational content between the grand canonical and canonical ensembles.