Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Exponential Families with Incompatible Statistics and Their Entropy Distance

Stephan Weis

Published Jan 1, 2010• 7 citations
OpenAlex

Abstract

This work generalizes the interaction measure of multi-information known in probability theory to finite-level quantum systems.This is done in the more general context of the entropy distance from an exponential family.One of the most well-known measures of stochastic dependence is multi-information, applied in various fields including Neuroscience and Statistical Mechanics.Multi-information is the entropy distance from an exponential family.In statistics, an exponential family is a very familiar parametric model, it admits a simple geometric description of entropy distance and maximum-likelihood estimation by mean values.But a complete description of these concepts requires to study extensions of a family, an investigation that was started by N. N. Čencov and O. Barndorff-Nielsen and continued principally by I. Csiszár and F. Matúš.Very little is known about extensions of an exponential family for a finite-level quantum system.In this thesis we consider mean value parameters of the statistic of such a family and a suitable extension thereof.Generalizing probability theory, we prove that the parameters describe the entropy distance from the family and they parametrize its rIclosure consisting of the points approximated in relative entropy.The dimension function of a local maximizer of entropy distance is bounded by the dimension of the family.We show that the rI -closure of a Gibbs family consists of the maximum entropy ensembles.A new and generic phenomenon of a non-abelian exponential family is the appearance of non-exposed faces of the convex mean value set.We prove that mean values of the closure of an e-geodesic included in the family meet only the relative interior of exposed faces.Unlike in finite probability spaces there are examples with a discontinuous entropy distance, the continuity being equivalent to equality of rI -closure and topological closure of the family.Examples suggest that the topology of an exponential family is related to the topology of associated projector lattices and to open projections and symmetrizations of state spaces.We conclude that multi-information for a quantum system is a continuous function equal to the entropy distance from a factorizable family.For analysis of a factorizable family we supply a partial classification of convex exponential families.As a perspective to a dynamical situation we examine a measure of temporal interaction for abelian systems, which is related to multi-information.i Being able to write this thesis in the Mathematical Physics group at Friedrich-Alexander-Universität Erlangen-Nürnberg was a great opportunity.In the first place I want to thank my supervisor Prof. Andreas Knauf for the constant and friendly support in conversations about ideas and literature both in mathematics and physics and for his encouragement to a systematic problem solving.

Related papers

Analysis of generalized Gibbs states

Stephan Weis, Ilya M. Spitkovsky

An exponential family is a manifold of generalized Gibbs state of the form exp(H)/Tr(exp(H)), where H belongs to a vector space of (possibly non-commutative) hermitian matrices. Generalized Gibbs states are important in small-scale thermodynamics, they represent equilibrium states regarding several conserved quantities that admit novel operations without heat dissipation [1]. Quantum information theory and condensed matter physics consider a space of local Hamiltonians acting on spins. The entropy distance from this exponential family is a measure of many-body complexity [2,3,4]. This talk is concerned with the geometry and topology of an exponential family and its entropy distance [5]. The maximum-entropy inference map parametrizes the exponential family. This map is continuous in the interior of its domain, the joint numerical range [6]. We describe the points of discontinuity in terms of open mapping theorems and eigenvalue crossings. Because of the discontinuity, the inference map and the entropy distance cannot be approximated through interior points. Instead, it is necessary to study faces (flat portions on the boundary) of the joint numerical range. With local Hamiltonians, this requires studying the faces of the set of quantum marginals. References. [1] M. Lostaglio, D. Jennings, and T. Rudolph,New Journal of Physics 19, 043008 (2017). [2] N. Ay, E. Olbrich, N. Bertschinger, and J. Jost,Chaos 21, 037103 (2011). [3] S. Niekamp, T. Galla, M. Kleinmann, and O. Gühne, J. Phys. A-Math. Gen. 46, 125301 (2013). [4] B. Zeng, X. Chen, D.-L. Zhou, and X.-G. Wen, Quantum Information Meets Quantum Matter,New York: Springer, 2019. [5] S. Weis, Journal of Convex Analysis 21, 339-399 (2014). [6] L. Rodman, I. M. Spitkovsky, A. Szkoła, and S. Weis,Journal of Mathematical Physics 57, 015204 (2016).

Proceedings of Entropy 2021: The Scientific Tool of the 21st Century•May 5, 2021•0 citations

Entropy distance: New quantum phenomena

Stephan Weis, Andreas Knauf

We study a curve of Gibbsian families of complex 3 × 3-matrices and point out new features, absent in commutative finite-dimensional algebras: a discontinuous maximum-entropy inference, a discontinuous entropy distance, and non-exposed faces of the mean value set. We analyze these problems from various aspects including convex geometry, topology, and information geometry. This research is motivated by a theory of infomax principles, where we contribute by computing first order optimality conditions of the entropy distance.

arXiv:1007.5464•Journal of Mathematical Physics•Oct 1, 2012•30 citations

Geometric Models for Quantum Statistical Inference

Dorje C. Brody, Lane P. Hughston

Abstract Our purpose here is to draw attention to an interesting relationship that can be shown to hold between (a) information geometry, and (b) quantum geometry. By ‘information geometry’, we mean the natural geometry associated with families of probability distributions. This is a subject that goes back more than half a century to the pioneering work of Rao (1945), who showed that the Fisher information matrix associated with a parametrised family of probability dis tributions gives rise to a natural Riemannian metric on the parameter space. This then allows one to speak in a precise way of the ‘distance’ between two probability distributions, an idea that turns out to be useful in problems of statistical inference. An extensive literature has thus developed, following this line of enquiry, on applications of differential geometry to statistics (see, e.g., Amari (1985), Barndorff-Nielsen et al. (1986), Murray and Rice (1993)). By ‘quantum geometry’, on the other hand, we mean the geometry of the manifold of states associated with a given quantum mechanical system. In particular, we consider the manifold of ‘pure’ states associated with a given complex Hilbert space. This is the space of ‘rays’ through the origin in the Hilbert space, which has the structure of a complex projective space. It is known that the usual rules of quantum mechanics allow one to construct a natural metric on this space, namely the Fubini-Study metric, the specification of which is equivalent to all the familiar structure associated with ordinary quantum mechanics.

Oxford University Press eBooks•Apr 30, 1998•5 citations

Information geometrical study of quantum Boltzmann machines

Nihal Yapage

In this thesis, we consider quantum extension of the well-known stochastic neuralnetwork model called (classical) Boltzmann machine (CBM) from an informationgeometrical point of view. The new model is called quantum Boltzmann machine(QBM).We investigate some properties and geometrical aspects of QBM analogous tothose of CBM. Furthermore, we study the mean-field approximation for such a modelfrom the information geometrical point of view. This is in some sense motivated bythe application of mean-field approximation for probabilistic inference in the graphicalmodels in the classical probability theory. Although the problem tackled in the presentthesis is somewhat deviated from the major field of quantum information theory, wehave elucidated the relationships among several well-known fields such as statisticalphysics, differential geometry, information theory and statistics using the concepts ofthe new emerging subject of quantum information geometry.We first define QBMs which can be considered as a general class of quantumIsing spin models. The states we consider are assumed to have at most secondorderinteractions with arbitrary but deterministic coupling coefficients. We call sucha state a QBM for the reason that it can be regarded as a quantum extension ofthe equilibrium distribution of CBM. The totality of QBMs is then shown to forma quantum exponential family and thus can be considered as a smooth manifoldhaving similar geometrical structures to those of CBMs. The information geometricalstructure of the manifold of QBMs is discussed and the problem of approximating agiven quantum state (density operator) by a QBM is also treated.We also define a restricted class of QBMs called the strongly separable QBMs(SSQBMs). We consider the dynamics of SSQBMs and propose a new state renewalrule based on that of CBM. The geometrical structure of the totality of SSQBMs isshown to be equivalent to that of the totality of CBMs. Approximation process forSSQBMs is also studied. Finally, we briefly discuss the parameter estimation of aSSQBM.Next, we study the mean-field approximation for QBMs from an information geometricalpoint of view. We elaborate on the significance and usefulness of informationgeometrical concepts, in particular the e-(exponential) and m-(mixture) projections,in studying the naive mean-field approximation for QBMs and derive the naive meanfieldequation explicitly. We also discuss the higher-order corrections to the naivemean-field approximation based on the idea of Plefka expansion in statistical physics.We elucidate the geometrical essence of the corrections and provide the expansioncoefficients with expressions in terms of information geometrical quantities. Here,one may note this work as the information geometrical interpretation of [Ple06] andas the quantum extension of [Tan00].

•Jan 1, 2008•2 citations

Canonical Divergence for Measuring Classical and Quantum Complexity

Domenico Felice, Stefano Mancini, Nihat Ay

A new canonical divergence is put forward for generalizing an information-geometric measure of complexity for both classical and quantum systems. On the simplex of probability measures, it is proved that the new divergence coincides with the Kullback-Leibler divergence, which is used to quantify how much a probability measure deviates from the non-interacting states that are modeled by exponential families of probabilities. On the space of positive density operators, we prove that the same divergence reduces to the quantum relative entropy, which quantifies many-party correlations of a quantum state from a Gibbs family.

arXiv:1903.09797•Entropy•Apr 24, 2019•10 citations

A Nonlocal Formulation of Quantum Maximum Entropy Principle Including Fractional Exclusion Statistics

Massimo Trovato, Lino Reggiani, Viale A. Doria

In thermodynamics and statistical mechanics entropy is the fundamental physical quantity to describe the evolution of a statistical ensemble. Its microscopic definition was provided by Boltzmann through the celebrated expression S = kB ln Γ, where kB is the Boltzmann constant and Γ is the number of microstates exploiting the given macroscopic properties. In this context, it is well known that in classical mechanics the entropy: i) allows the violation of the uncertainty principle [1], [2]; ii) can be considered as a special case of the so-called Boltzmann-Gibbs-Shannon entropy that enables one to apply results of information theory to physics [1], [3]. In particular, by introducing the principle of maximum entropy (MEP) it was found possible to derive rigorous hydrodynamic (HD) models based on the moments of the distribution function to all orders of a power expansion and including appropriate closure conditions [4], [5]. Accordingly, making use of the Lagrange multipliers technique, it was found possible to construct the set of evolution equations for the macro-variables of interest. Apart from some partial attempts [3], [6], this is no longer the case in quantum mechanics. Here, the main difficulties concern with: i) the definition of a proper quantum entropy that includes particle indistinguishability; ii) the formulation of a global quantum MEP (QMEP) that allows one to obtain a quantum distribution function both for thermodynamic equilibrium and nonequilibrium configurations. From one hand, in the framework of a nonlocal quantum theory, the generalization of the corresponding Lagrange multipliers is also an open problem. Fron another hand, a rigorous formulation of quantum HD (QHD) closed models is a demanding issue for many kinds of problems in quantum systems like, interacting fermionic and bosonic gases, quantum turbulence, quantum fluids, quantized vortices, nuclear physics, confined carrier transport in semiconductor heterostrucures, phonon and electron transport in nanostructures, nanowires and thin layers. The aim of this talk is to address the above drawbacks by defining a quantum entropy in terms of the reduced density matrix, thus explicitly incorporating quantum statistics into problems involving a system of identical particles. As a further step, with respect to the uncertainty principle, starting from the Wigner representation we formulate a quantum maximum entropy principle which, in the framework of information theory, allows us to obtain a non-local theory for the system under study. As a final step, we determine a closed quantum hydrodynamic model for the macroscopic variables used as constraints in the QMEP approach.

•Jan 1, 2013•0 citations
This work generalizes the interaction measure of multi-information known in probability theory to finite-level quantum systems.This is done in the more general context of the entropy distance from an exponential family.One of the most well-known measures of stochastic dependence is multi-information, applied in various fields including Neuroscience and Statistical Mechanics.Multi-information is the entropy distance from an exponential family.In statistics, an exponential family is a very familiar parametric model, it admits a simple geometric description of entropy distance and maximum-likelihood estimation by mean values.But a complete description of these concepts requires to study extensions of a family, an investigation that was started by N. N. Čencov and O. Barndorff-Nielsen and continued principally by I. Csiszár and F. Matúš.Very little is known about extensions of an exponential family for a finite-level quantum system.In this thesis we consider mean value parameters of the statistic of such a family and a suitable extension thereof.Generalizing probability theory, we prove that the parameters describe the entropy distance from the family and they parametrize its rIclosure consisting of the points approximated in relative entropy.The dimension function of a local maximizer of entropy distance is bounded by the dimension of the family.We show that the rI -closure of a Gibbs family consists of the maximum entropy ensembles.A new and generic phenomenon of a non-abelian exponential family is the appearance of non-exposed faces of the convex mean value set.We prove that mean values of the closure of an e-geodesic included in the family meet only the relative interior of exposed faces.Unlike in finite probability spaces there are examples with a discontinuous entropy distance, the continuity being equivalent to equality of rI -closure and topological closure of the family.Examples suggest that the topology of an exponential family is related to the topology of associated projector lattices and to open projections and symmetrizations of state spaces.We conclude that multi-information for a quantum system is a continuous function equal to the entropy distance from a factorizable family.For analysis of a factorizable family we supply a partial classification of convex exponential families.As a perspective to a dynamical situation we examine a measure of temporal interaction for abelian systems, which is related to multi-information.i Being able to write this thesis in the Mathematical Physics group at Friedrich-Alexander-Universität Erlangen-Nürnberg was a great opportunity.In the first place I want to thank my supervisor Prof. Andreas Knauf for the constant and friendly support in conversations about ideas and literature both in mathematics and physics and for his encouragement to a systematic problem solving.
An exponential family is a manifold of generalized Gibbs state of the form exp(H)/Tr(exp(H)), where H belongs to a vector space of (possibly non-commutative) hermitian matrices. Generalized Gibbs states are important in small-scale thermodynamics, they represent equilibrium states regarding several conserved quantities that admit novel operations without heat dissipation [1]. Quantum information theory and condensed matter physics consider a space of local Hamiltonians acting on spins. The entropy distance from this exponential family is a measure of many-body complexity [2,3,4]. This talk is concerned with the geometry and topology of an exponential family and its entropy distance [5]. The maximum-entropy inference map parametrizes the exponential family. This map is continuous in the interior of its domain, the joint numerical range [6]. We describe the points of discontinuity in terms of open mapping theorems and eigenvalue crossings. Because of the discontinuity, the inference map and the entropy distance cannot be approximated through interior points. Instead, it is necessary to study faces (flat portions on the boundary) of the joint numerical range. With local Hamiltonians, this requires studying the faces of the set of quantum marginals. References. [1] M. Lostaglio, D. Jennings, and T. Rudolph,New Journal of Physics 19, 043008 (2017). [2] N. Ay, E. Olbrich, N. Bertschinger, and J. Jost,Chaos 21, 037103 (2011). [3] S. Niekamp, T. Galla, M. Kleinmann, and O. Gühne, J. Phys. A-Math. Gen. 46, 125301 (2013). [4] B. Zeng, X. Chen, D.-L. Zhou, and X.-G. Wen, Quantum Information Meets Quantum Matter,New York: Springer, 2019. [5] S. Weis, Journal of Convex Analysis 21, 339-399 (2014). [6] L. Rodman, I. M. Spitkovsky, A. Szkoła, and S. Weis,Journal of Mathematical Physics 57, 015204 (2016).
We study a curve of Gibbsian families of complex 3 × 3-matrices and point out new features, absent in commutative finite-dimensional algebras: a discontinuous maximum-entropy inference, a discontinuous entropy distance, and non-exposed faces of the mean value set. We analyze these problems from various aspects including convex geometry, topology, and information geometry. This research is motivated by a theory of infomax principles, where we contribute by computing first order optimality conditions of the entropy distance.
Abstract Our purpose here is to draw attention to an interesting relationship that can be shown to hold between (a) information geometry, and (b) quantum geometry. By ‘information geometry’, we mean the natural geometry associated with families of probability distributions. This is a subject that goes back more than half a century to the pioneering work of Rao (1945), who showed that the Fisher information matrix associated with a parametrised family of probability dis tributions gives rise to a natural Riemannian metric on the parameter space. This then allows one to speak in a precise way of the ‘distance’ between two probability distributions, an idea that turns out to be useful in problems of statistical inference. An extensive literature has thus developed, following this line of enquiry, on applications of differential geometry to statistics (see, e.g., Amari (1985), Barndorff-Nielsen et al. (1986), Murray and Rice (1993)). By ‘quantum geometry’, on the other hand, we mean the geometry of the manifold of states associated with a given quantum mechanical system. In particular, we consider the manifold of ‘pure’ states associated with a given complex Hilbert space. This is the space of ‘rays’ through the origin in the Hilbert space, which has the structure of a complex projective space. It is known that the usual rules of quantum mechanics allow one to construct a natural metric on this space, namely the Fubini-Study metric, the specification of which is equivalent to all the familiar structure associated with ordinary quantum mechanics.
In this thesis, we consider quantum extension of the well-known stochastic neuralnetwork model called (classical) Boltzmann machine (CBM) from an informationgeometrical point of view. The new model is called quantum Boltzmann machine(QBM).We investigate some properties and geometrical aspects of QBM analogous tothose of CBM. Furthermore, we study the mean-field approximation for such a modelfrom the information geometrical point of view. This is in some sense motivated bythe application of mean-field approximation for probabilistic inference in the graphicalmodels in the classical probability theory. Although the problem tackled in the presentthesis is somewhat deviated from the major field of quantum information theory, wehave elucidated the relationships among several well-known fields such as statisticalphysics, differential geometry, information theory and statistics using the concepts ofthe new emerging subject of quantum information geometry.We first define QBMs which can be considered as a general class of quantumIsing spin models. The states we consider are assumed to have at most secondorderinteractions with arbitrary but deterministic coupling coefficients. We call sucha state a QBM for the reason that it can be regarded as a quantum extension ofthe equilibrium distribution of CBM. The totality of QBMs is then shown to forma quantum exponential family and thus can be considered as a smooth manifoldhaving similar geometrical structures to those of CBMs. The information geometricalstructure of the manifold of QBMs is discussed and the problem of approximating agiven quantum state (density operator) by a QBM is also treated.We also define a restricted class of QBMs called the strongly separable QBMs(SSQBMs). We consider the dynamics of SSQBMs and propose a new state renewalrule based on that of CBM. The geometrical structure of the totality of SSQBMs isshown to be equivalent to that of the totality of CBMs. Approximation process forSSQBMs is also studied. Finally, we briefly discuss the parameter estimation of aSSQBM.Next, we study the mean-field approximation for QBMs from an information geometricalpoint of view. We elaborate on the significance and usefulness of informationgeometrical concepts, in particular the e-(exponential) and m-(mixture) projections,in studying the naive mean-field approximation for QBMs and derive the naive meanfieldequation explicitly. We also discuss the higher-order corrections to the naivemean-field approximation based on the idea of Plefka expansion in statistical physics.We elucidate the geometrical essence of the corrections and provide the expansioncoefficients with expressions in terms of information geometrical quantities. Here,one may note this work as the information geometrical interpretation of [Ple06] andas the quantum extension of [Tan00].
A new canonical divergence is put forward for generalizing an information-geometric measure of complexity for both classical and quantum systems. On the simplex of probability measures, it is proved that the new divergence coincides with the Kullback-Leibler divergence, which is used to quantify how much a probability measure deviates from the non-interacting states that are modeled by exponential families of probabilities. On the space of positive density operators, we prove that the same divergence reduces to the quantum relative entropy, which quantifies many-party correlations of a quantum state from a Gibbs family.
In thermodynamics and statistical mechanics entropy is the fundamental physical quantity to describe the evolution of a statistical ensemble. Its microscopic definition was provided by Boltzmann through the celebrated expression S = kB ln Γ, where kB is the Boltzmann constant and Γ is the number of microstates exploiting the given macroscopic properties. In this context, it is well known that in classical mechanics the entropy: i) allows the violation of the uncertainty principle [1], [2]; ii) can be considered as a special case of the so-called Boltzmann-Gibbs-Shannon entropy that enables one to apply results of information theory to physics [1], [3]. In particular, by introducing the principle of maximum entropy (MEP) it was found possible to derive rigorous hydrodynamic (HD) models based on the moments of the distribution function to all orders of a power expansion and including appropriate closure conditions [4], [5]. Accordingly, making use of the Lagrange multipliers technique, it was found possible to construct the set of evolution equations for the macro-variables of interest. Apart from some partial attempts [3], [6], this is no longer the case in quantum mechanics. Here, the main difficulties concern with: i) the definition of a proper quantum entropy that includes particle indistinguishability; ii) the formulation of a global quantum MEP (QMEP) that allows one to obtain a quantum distribution function both for thermodynamic equilibrium and nonequilibrium configurations. From one hand, in the framework of a nonlocal quantum theory, the generalization of the corresponding Lagrange multipliers is also an open problem. Fron another hand, a rigorous formulation of quantum HD (QHD) closed models is a demanding issue for many kinds of problems in quantum systems like, interacting fermionic and bosonic gases, quantum turbulence, quantum fluids, quantized vortices, nuclear physics, confined carrier transport in semiconductor heterostrucures, phonon and electron transport in nanostructures, nanowires and thin layers. The aim of this talk is to address the above drawbacks by defining a quantum entropy in terms of the reduced density matrix, thus explicitly incorporating quantum statistics into problems involving a system of identical particles. As a further step, with respect to the uncertainty principle, starting from the Wigner representation we formulate a quantum maximum entropy principle which, in the framework of information theory, allows us to obtain a non-local theory for the system under study. As a final step, we determine a closed quantum hydrodynamic model for the macroscopic variables used as constraints in the QMEP approach.