Abstract Shannon information is defined for characterizing the uncertainty information of classical probabilistic distributions. As an uncertainty measure it is generally believed to be positive. This holds for any information quantity from two random variables because of the polymatroidal axioms. However, it is unknown why there is negative information for more than two random variables on finite dimensional spaces. It is first shown that the negative tripartite Shannon mutual information implies specific Bayesian network representations of its joint distribution. Then it is shown that the negative Shannon information is obtained from general tripartite Bayesian networks with quantum realizations. This provides a device‐independent witness of negative Shannon information. Finally the result is extended for general networks. The present result shows new insights in the network compatibility from non‐Shannon information inequalities.
Abstract Shannon information is defined for characterizing the uncertainty information of classical probabilistic distributions. As an uncertainty measure it is generally believed to be positive. This holds for any information quantity from two random variables because of the polymatroidal axioms. However, it is unknown why there is negative information for more than two random variables on finite dimensional spaces. It is first shown that the negative tripartite Shannon mutual information implies specific Bayesian network representations of its joint distribution. Then it is shown that the negative Shannon information is obtained from general tripartite Bayesian networks with quantum realizations. This provides a device‐independent witness of negative Shannon information. Finally the result is extended for general networks. The present result shows new insights in the network compatibility from non‐Shannon information inequalities.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.
The conditional quantum mutual information I(A; B|C) of a tripartite state ρABC is an information quantity which lies at the center of many problems in quantum information theory. Three of its main properties are that it is non-negative for any tripartite state, that it decreases under local operations applied to systems A and B, and that it obeys the duality relation I(A; B|C) = I(A; B|D) for a four-party pure state on systems ABCD. The conditional mutual information also underlies the squashed entanglement, an entanglement measure that satisfies all of the axioms desired for an entanglement measure. As such, it has been an open question to find Rényi generalizations of the conditional mutual information, that would allow for a deeper understanding of the original quantity and find applications beyond the traditional memoryless setting of quantum information theory. The present paper addresses this question, by defining different α-Rényi generalizations Iα(A; B|C) of the conditional mutual information, some of which we can prove converge to the conditional mutual information in the limit α → 1. Furthermore, we prove that many of these generalizations satisfy non-negativity, duality, and monotonicity with respect to local operations on one of the systems A or B (with it being left as an open question to prove that monotonicity holds with respect to local operations on both systems). The quantities defined here should find applications in quantum information theory and perhaps even in other areas of physics, but we leave this for future work. We also state a conjecture regarding the monotonicity of the Rényi conditional mutual informations defined here with respect to the Rényi parameter α. We prove that this conjecture is true in some special cases and when α is in a neighborhood of one.
The concept of entropy rst arose in the study of thermodynamics and statistical me- chanics. In information theory the so-called Shannon entropy plays a signi cant role, which measures how much information we gain, on average, when we learn the value of a random variable. An alternative interpretation is that the entropy of the random variable quanti es the amount of uncertainty about the random variable before we learn its value. The concept was introduced by Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication". In the rst part of the present work we consider this fundamental measure of information and show some of its properties, such as non-negativity. The relative entropy or the Kullback-Leibler divergence is a very useful measure of the distance between two prob- ability distributions. After introducing its de nition and other notions (joint entropy, conditional entropy or mutual information) we can prove additional assertions about the Shannon entropy and present statements about the relations between the above men- tioned quantities. In quantum information theory, there is a quantity, namely von Neumann entropy, which corresponds to Shannon entropy and originates from it. In the section Von Neu- mann entropy we will introduce this quantity which is a continuous functional on the states, or on their mathematical representatives, the density operators, and it ful ls in- equalities such as subadditivity and strong subaddititvity. Just as with the Shannon entropy, it is extremely useful to de ne a quantum version of the relative entropy, which is one of the most important numerical quantities appearing in quantum information theory. It is used as a measure of distinguishability between quantum states. Among others its properties are jointly convexity, monotonicity which are speci cally convenient. Transformations on structures which preserve operations, quantities or relations among elements appear in many areas of mathematics. These are the so-called preservers which show up also in physics and even in chemistry where they are usually called symmetries. Therefore, the study of preserver transformations is an important area of research. The third section of this work is based on a paper, in which Lajos Molnár described the structure every bijective map on the space of all density operators on a given Hilbert space, which preserves the relative entropy. This result is closely related to Wigner's famous theorem on the form of quantum mechanical symmetry transformations. The above theorem of Molnár can be extended into two directions. On the one hand, the result holds also in the case when the transformation under consideration acts on the space of all positive semide nite operators not only on density operators (i.e., we omit the condition of unit trace). On the other hand the statement remains valid even if we omit the strongly restrictive condition that the transformation is bijective. The second result is a joint work with Prof. Lajos Molnár which appeared in the journal Linear Algebra and Its Applications.
A causal structure is a relationship between observed variables that in general restricts the possible correlations between them. This relationship can be mediated by unobserved systems, modelled by random variables in the classical case or joint quantum systems in the quantum case. One way to differentiate between the correlations realisable by two different causal structures is to use entropy vectors, i.e., vectors whose components correspond to the entropies of each subset of the observed variables. To date, the starting point for deriving entropic constraints within causal structures are the so-called Shannon inequalities (positivity of entropy, conditional entropy and conditional mutual information). In the present work we investigate what happens when non-Shannon entropic inequalities are included as well. We show that in general these lead to tighter outer approximations of the set of realisable entropy vectors and hence enable a sharper distinction of different causal structures. Since non-Shannon inequalities can only be applied amongst classical variables, it might be expected that their use enables an entropic distinction between classical and quantum causal structures. However, this remains an open question. We also introduce techniques for deriving inner approximations to the allowed sets of entropy vectors for a given causal structure. These are useful for proving tightness of outer approximations or for finding interesting regions of entropy space. We illustrate these techniques in several scenarios, including the triangle causal structure.
We analyze the geometry of a joint distribution over a set of discrete random variables. We briefly review Shannon's entropy, conditional entropy, mutual information and conditional mutual information. We review the entropic information distance formula of Rokhlin and Rajski. We then define an analogous information area. We motivate this definition and discuss its properties. We extend this definition to higher-dimensional volumes. We briefly discuss the potential utility for these geometric measures in quantum information processing.
With the aid of a toy model, the Monty Hall Problem (MHP), the counterintuitive and theoretically problematic concept of negative information in classical systems is well understood. It is shown that, as its quantum counterpart, classical local mutual information, obtained through a measurement, can be expressed as the difference between the information gained with the evidence and the negative information generated due to the inefficiency of the measurement itself; a novel local Shannon metric, the transfer information content, is defined as this difference, which is negative if the measurement generates more disturbance than the evidence, i.e., generates a classical measurement back action. This metric is valid for both, Classical and Quantum measurements, and it is proposed as a starting point towards a general physical theory of information. This information-disturbance trade-off in classical measurements is a kind of Heisenberg principle at macroscopic scales, and it is proposed, as further work, to incorporate this result in the already existing generalized uncertainty principles in the field of quantum gravity.