Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Introduction to Logical Entropy and its Relationship to Shannon Entropy

David Ellerman

Published Jan 1, 2021• 8 citations
Publisher
DOI
10.2139/ssrn.3978011

Related papers

Introduction to Logical Entropy and its Relationship to Shannon Entropy

David Ellerman

We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined." Logical entropy provides that definition. Logical entropy is the natural measure of the notion of information based on distinctions, differences, distinguishability, and diversity. It is the (normalized) quantitative measure of the distinctions of a partition on a set--just as the Boole-Laplace logical probability is the normalized quantitative measure of the elements of a subset of a set. And partitions and subsets are mathematically dual concepts--so the logic of partitions is dual in that sense to the usual Boolean logic of subsets, and hence the name "logical entropy." The logical entropy of a partition has a simple interpretation as the probability that a distinction or dit (elements in different blocks) is obtained in two independent draws from the underlying set. The Shannon entropy is shown to also be based on this notion of information-as-distinctions; it is the average minimum number of binary partitions (bits) that need to be joined to make all the same distinctions of the given partition. Hence all the concepts of simple, joint, conditional, and mutual logical entropy can be transformed into the corresponding concepts of Shannon entropy by a uniform non-linear dit-bit transform. And finally logical entropy linearizes naturally to the corresponding quantum concept. The quantum logical entropy of an observable applied to a state is the probability that two different eigenvalues are obtained in two independent projective measurements of that observable on that state. Keywords: logical entropy, Shannon entropy, partitions, MaxEntropy, quantum logical entropy, von Neumann entropy

arXiv:2112.01966•Dec 3, 2021•4 citations

Shannon Entropy: From Probabilities to Particles

Thomas Wallek, Andreas Pfennig

•Feb 13, 2012•0 citations

Information Theory and Statistical Entropy

Roger Balian

Springer eBooks•Jan 1, 1991•2 citations

Introduction to logical entropy and its relationship to Shannon entropy

David Ellerman

We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an “amount of information,” but, as he pointed out, “no concept of information itself was defined.” Logical entropy provides that definition. Logical entropy is the natural measure of the notion of information based on distinctions, differences, distinguishability, and diversity. It is the (normalized) quantitative measure of the distinctions of a partition on a set-just as the Boole–Laplace logical probability is the normalized quantitative measure of the elements of a subset of a set. And partitions and subsets are mathematically dual concepts – so the logic of partitions is dual in that sense to the usual Boolean logic of subsets, and hence the name “logical entropy.” The logical entropy of a partition has a simple interpretation as the probability that a distinction or dit (elements in different blocks) is obtained in two independent draws from the underlying set. The Shannon entropy is shown to also be based on this notion of information-as-distinctions; it is the average minimum number of binary partitions (bits) that need to be joined to make all the same distinctions of the given partition. Hence all the concepts of simple, joint, conditional, and mutual logical entropy can be transformed into the corresponding concepts of Shannon entropy by a uniform non-linear dit-bit transform. And finally logical entropy linearizes naturally to the corresponding quantum concept. The quantum logical entropy of an observable applied to a state is the probability that two different eigenvalues are obtained in two independent projective measurements of that observable on that state.

4open•Jan 1, 2022•13 citations

Entropic Geometry from Logic

Bob Coecke

We establish the following equation: Quantitative Probability = Logic + Partiality of Knowledge + Entropy I.e.: 1. A finitary probability space Δn (= all probability measures on {1,…, n}) can be fully and faithfully represented by the pair consisting of the abstraction Dn (= the object up to isomorphism) of the partially ordered set (Δn, ⊑) introduced in [3], and, Shannon entropy; 2. Dn itself can be obtained via a systematic purely order-theoretic procedure (which embodies introduction of partiality of knowledge) on an (algebraic) logic. This procedure applies to any poset A; DA ≅ (Δn ,⊑) when A is the n-element powerset and DA ≅ (Ωn, ⊑), the domain of mixed quantum states also introduced in [3], when A is the lattice of subspaces of a Hilbert space.

arXiv:quant-ph/0212065•Electronic Notes in Theoretical Computer Science•Jan 1, 2003•9 citations

Basics of information theory

Jonathan A. Jones, Dieter Jaksch

Cambridge University Press eBooks•Jul 19, 2012•1 citation
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined." Logical entropy provides that definition. Logical entropy is the natural measure of the notion of information based on distinctions, differences, distinguishability, and diversity. It is the (normalized) quantitative measure of the distinctions of a partition on a set--just as the Boole-Laplace logical probability is the normalized quantitative measure of the elements of a subset of a set. And partitions and subsets are mathematically dual concepts--so the logic of partitions is dual in that sense to the usual Boolean logic of subsets, and hence the name "logical entropy." The logical entropy of a partition has a simple interpretation as the probability that a distinction or dit (elements in different blocks) is obtained in two independent draws from the underlying set. The Shannon entropy is shown to also be based on this notion of information-as-distinctions; it is the average minimum number of binary partitions (bits) that need to be joined to make all the same distinctions of the given partition. Hence all the concepts of simple, joint, conditional, and mutual logical entropy can be transformed into the corresponding concepts of Shannon entropy by a uniform non-linear dit-bit transform. And finally logical entropy linearizes naturally to the corresponding quantum concept. The quantum logical entropy of an observable applied to a state is the probability that two different eigenvalues are obtained in two independent projective measurements of that observable on that state. Keywords: logical entropy, Shannon entropy, partitions, MaxEntropy, quantum logical entropy, von Neumann entropy
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an “amount of information,” but, as he pointed out, “no concept of information itself was defined.” Logical entropy provides that definition. Logical entropy is the natural measure of the notion of information based on distinctions, differences, distinguishability, and diversity. It is the (normalized) quantitative measure of the distinctions of a partition on a set-just as the Boole–Laplace logical probability is the normalized quantitative measure of the elements of a subset of a set. And partitions and subsets are mathematically dual concepts – so the logic of partitions is dual in that sense to the usual Boolean logic of subsets, and hence the name “logical entropy.” The logical entropy of a partition has a simple interpretation as the probability that a distinction or dit (elements in different blocks) is obtained in two independent draws from the underlying set. The Shannon entropy is shown to also be based on this notion of information-as-distinctions; it is the average minimum number of binary partitions (bits) that need to be joined to make all the same distinctions of the given partition. Hence all the concepts of simple, joint, conditional, and mutual logical entropy can be transformed into the corresponding concepts of Shannon entropy by a uniform non-linear dit-bit transform. And finally logical entropy linearizes naturally to the corresponding quantum concept. The quantum logical entropy of an observable applied to a state is the probability that two different eigenvalues are obtained in two independent projective measurements of that observable on that state.
We establish the following equation: Quantitative Probability = Logic + Partiality of Knowledge + Entropy I.e.: 1. A finitary probability space Δn (= all probability measures on {1,…, n}) can be fully and faithfully represented by the pair consisting of the abstraction Dn (= the object up to isomorphism) of the partially ordered set (Δn, ⊑) introduced in [3], and, Shannon entropy; 2. Dn itself can be obtained via a systematic purely order-theoretic procedure (which embodies introduction of partiality of knowledge) on an (algebraic) logic. This procedure applies to any poset A; DA ≅ (Δn ,⊑) when A is the n-element powerset and DA ≅ (Ωn, ⊑), the domain of mixed quantum states also introduced in [3], when A is the lattice of subspaces of a Hilbert space.