Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2112.03582 · Entropy

On a 2-Relative Entropy

James Fullwood

Published Dec 31, 2021• 2 citations
arXivHTMLPDF

Abstract

We construct a 2-categorical extension of the relative entropy functor of Baez and Fritz, and show that our construction is functorial with respect to vertical morphisms. Moreover, we show such a ‘2-relative entropy’ satisfies natural 2-categorial analogues of convex linearity, vanishing under optimal hypotheses, and lower semicontinuity. While relative entropy is a relative measure of information between probability distributions, we view our construction as a relative measure of information between channels.

DOI
10.3390/e24010074

Related papers

A Bayesian characterization of relative entropy

John C. Baez, Tobias Fritz

We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence.We use a number of interesting categories related to probability theory.In particular, we consider a category FinStat where an object is a finite set equipped with a probability distribution, while a morphism is a measure-preserving function f : X → Y together with a stochastic right inverse s : Y → X.The function f can be thought of as a measurement process, while s provides a hypothesis about the state of the measured system given the result of a measurement.Given this data we can define the entropy of the probability distribution on X relative to the 'prior' given by pushing the probability distribution on Y forwards along s.We say that s is 'optimal' if these distributions agree.We show that any convex linear, lower semicontinuous functor from FinStat to the additive monoid [0, ∞] which vanishes when s is optimal must be a scalar multiple of this relative entropy.Our proof is independent of all earlier characterizations, but inspired by the work of Petz.We thank Ryszard Kostecki and Rob Spekkens for discussions

arXiv:1402.3067•Theory and applications of categories•Jan 1, 2014•63 citations

A characterization of von Neumann entropy using functors

Kenji Nakahira

Baez, Fritz, and Leinster derived a method for characterizing Shannon entropy in classical systems. In this method, they considered a functor from a certain category to the monoid of non-negative real numbers with addition as a map from measure-preserving functions to non-negative real numbers, and derived Shannon entropy by imposing several simple conditions. We propose a method for characterizing von Neumann entropy by extending their results to quantum systems.

arXiv:2309.10353•Sep 19, 2023•0 citations

Categories of Brègman operations and epistemic (co)monads

Ryszard Paweł Kostecki

We construct a categorical framework for nonlinear postquantum inference, with embeddings of convex closed sets of suitable reflexive Banach spaces as objects and pullbacks of Brègman quasi-nonexpansive mappings (in particular, constrained maximisations of Brègman relative entropies) as morphisms. It provides a nonlinear convex analytic analogue of Chencov's programme of geometric study of categories of linear positive maps between spaces of states, a working model of Mielnik's nonlinear transmitters, and a setting for nonlinear resource theories (with monoids of Brègman quasi-nonexpansive maps as free operations, their asymptotic fixed point sets as free sets, and Brègman relative entropies as resource monotones). We construct a range of concrete examples for semi-finite JBW-algebras and any W*-algebras. Due to relative entropy's asymmetry, all constructions have left and right versions, with Legendre duality inducing categorical equivalence between their well-defined restrictions. Inner groupoids of these categories implement the notion of statistical equivalence. The hom-sets of a subcategory of morphisms given by entropic projections have the structure of partially ordered commutative monoids (so, they are resource theories in Fritz's sense). Further restriction of objects to affine sets turns Brègman relative entropy into a functor. Finally, following Lawvere's adjointness paradigm for deductive logic, but with a semantic twist representing Jaynes' and Chencov's views on statistical inference, we introduce a category-theoretic multi-(co)agent setting for inductive inference theories, implemented by families of monads and comonads. We show that the brègmanian approach provides some special cases of this setting.

arXiv:2103.07810•Mar 13, 2021•1 citation

Lower semicontinuity of relative entropy disturbance and its consequences

Максим Евгеньевич Широков

It is proved that the decrease of quantum relative entropy under the action of a quantum operation is a lower semicontinuous function of the pair of its arguments. This property implies, in particular, that the local discontinuity jumps of the quantum relative entropy do not increase under the action of quantum operations. It also implies the lower semicontinuity of the modulus of the joint convexity of quantum relative entropy (as a function of ensembles of quantum states). Various corollaries and applications of these results are considered. Bibliography: 42 titles.

arXiv:2302.04809•Sbornik Mathematics•Jan 1, 2024•0 citations

Postquantum Br\`{e}gman relative entropies and nonlinear resource theories

Ryszard Paweł Kostecki

We introduce the family of postquantum Bregman relative entropies, based on nonlinear embeddings into reflexive Banach spaces (with examples given by reflexive noncommutative Orlicz spaces over semi-finite W*-algebras, nonassociative L$_p$ spaces over semi-finite JBW-algebras, and noncommutative L$_p$ spaces over arbitrary W*-algebras). This allows us to define a class of geometric categories for nonlinear postquantum inference theory (providing an extension of Chencov's approach to foundations of statistical inference), with constrained maximisations of Bregman relative entropies as morphisms and nonlinear images of closed convex sets as objects. Further generalisation to a framework for nonlinear convex operational theories is developed using a larger class of morphisms, determined by Bregman nonexpansive operations (which provide a well-behaved family of Mielnik's nonlinear transmitters). As an application, we derive a range of nonlinear postquantum resource theories determined in terms of this class of operations.

•Oct 5, 2017•2 citations

A complete characterisation of conditional entropies

Roberto Rubboli, Erkka Haapasalo, Marco Tomamichel

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number $α$. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterisation of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterisation of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We show that the most general conditional entropy is a generalized convex combination of conditional entropies defined as exponential averages of Rényi entropies of the conditioned distribution, parameterized by a real number and a probability measure on the positive reals.

arXiv:2601.23213•Jan 30, 2026•4 citations
We construct a 2-categorical extension of the relative entropy functor of Baez and Fritz, and show that our construction is functorial with respect to vertical morphisms. Moreover, we show such a ‘2-relative entropy’ satisfies natural 2-categorial analogues of convex linearity, vanishing under optimal hypotheses, and lower semicontinuity. While relative entropy is a relative measure of information between probability distributions, we view our construction as a relative measure of information between channels.
We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence.We use a number of interesting categories related to probability theory.In particular, we consider a category FinStat where an object is a finite set equipped with a probability distribution, while a morphism is a measure-preserving function f : X → Y together with a stochastic right inverse s : Y → X.The function f can be thought of as a measurement process, while s provides a hypothesis about the state of the measured system given the result of a measurement.Given this data we can define the entropy of the probability distribution on X relative to the 'prior' given by pushing the probability distribution on Y forwards along s.We say that s is 'optimal' if these distributions agree.We show that any convex linear, lower semicontinuous functor from FinStat to the additive monoid [0, ∞] which vanishes when s is optimal must be a scalar multiple of this relative entropy.Our proof is independent of all earlier characterizations, but inspired by the work of Petz.We thank Ryszard Kostecki and Rob Spekkens for discussions
We construct a categorical framework for nonlinear postquantum inference, with embeddings of convex closed sets of suitable reflexive Banach spaces as objects and pullbacks of Brègman quasi-nonexpansive mappings (in particular, constrained maximisations of Brègman relative entropies) as morphisms. It provides a nonlinear convex analytic analogue of Chencov's programme of geometric study of categories of linear positive maps between spaces of states, a working model of Mielnik's nonlinear transmitters, and a setting for nonlinear resource theories (with monoids of Brègman quasi-nonexpansive maps as free operations, their asymptotic fixed point sets as free sets, and Brègman relative entropies as resource monotones). We construct a range of concrete examples for semi-finite JBW-algebras and any W*-algebras. Due to relative entropy's asymmetry, all constructions have left and right versions, with Legendre duality inducing categorical equivalence between their well-defined restrictions. Inner groupoids of these categories implement the notion of statistical equivalence. The hom-sets of a subcategory of morphisms given by entropic projections have the structure of partially ordered commutative monoids (so, they are resource theories in Fritz's sense). Further restriction of objects to affine sets turns Brègman relative entropy into a functor. Finally, following Lawvere's adjointness paradigm for deductive logic, but with a semantic twist representing Jaynes' and Chencov's views on statistical inference, we introduce a category-theoretic multi-(co)agent setting for inductive inference theories, implemented by families of monads and comonads. We show that the brègmanian approach provides some special cases of this setting.
It is proved that the decrease of quantum relative entropy under the action of a quantum operation is a lower semicontinuous function of the pair of its arguments. This property implies, in particular, that the local discontinuity jumps of the quantum relative entropy do not increase under the action of quantum operations. It also implies the lower semicontinuity of the modulus of the joint convexity of quantum relative entropy (as a function of ensembles of quantum states). Various corollaries and applications of these results are considered. Bibliography: 42 titles.
We introduce the family of postquantum Bregman relative entropies, based on nonlinear embeddings into reflexive Banach spaces (with examples given by reflexive noncommutative Orlicz spaces over semi-finite W*-algebras, nonassociative Lp_pp​ spaces over semi-finite JBW-algebras, and noncommutative Lp_pp​ spaces over arbitrary W*-algebras). This allows us to define a class of geometric categories for nonlinear postquantum inference theory (providing an extension of Chencov's approach to foundations of statistical inference), with constrained maximisations of Bregman relative entropies as morphisms and nonlinear images of closed convex sets as objects. Further generalisation to a framework for nonlinear convex operational theories is developed using a larger class of morphisms, determined by Bregman nonexpansive operations (which provide a well-behaved family of Mielnik's nonlinear transmitters). As an application, we derive a range of nonlinear postquantum resource theories determined in terms of this class of operations.
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number ααα. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterisation of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterisation of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We show that the most general conditional entropy is a generalized convex combination of conditional entropies defined as exponential averages of Rényi entropies of the conditioned distribution, parameterized by a real number and a probability measure on the positive reals.