Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Quantum Developments of Classical Information Theory

Mostafa Mansour

Published Feb 7, 2026• 0 citations
Publisher

Abstract

These notes (in French) provide a concise introduction to information theory, spanning its classical foundations: Shannon entropy, joint and conditional entropies, mutual information, statistical correlations, and Markov chains; as well as Shannon's source coding and channel capacity theorems, which govern the optimal compression and transmission of information through noisy channels. They also cover the quantum extensions of these concepts, including the density operator formalism, distances between quantum states, von Neumann entropy, coherent information, conditional entropy, entanglement theory, quantum discord, and quantum resource theory.

DOI
10.5281/zenodo.19945728

Related papers

Principles of Classical and Quantum Information Theory

Bouchra El Alaoui, Abdallah Slaoui, R. Ahl Laamara

Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.

CRC Press eBooks•May 15, 2026•0 citations

Quantum information theory

E. Desurvire

This chapter sets the basis of quantum information theory (QIT). The central purpose of QIT is to qualify the transmission of either classical or quantum information over quantum channels. The starting point of the QIT description is von Neumann entropy, S(ρ), which represents the quantum counterpart of Shannon's classical entropy, H(X). Such a definition rests on that of the density operator (or density matrix) of a quantum system, ρ, which plays a role similar to that of the random-events source X in Shannon's theory. As we shall see, there also exists an elegant and one-to-one correspondence between the quantum and classical definitions of the entropy variants relative entropy, joint entropy, conditional entropy, and mutual information. But such a similarity is only apparent. Indeed, one becomes rapidly convinced from a systematic analysis of the entropy's additivity rules that fundamental differences separate the two worlds. The classical notion of information correlation between two event sources for quantum states shall be referred to as quantum entanglement. We then define a quantum communication channel, which encodes and decodes classical information into or from quantum states. The analysis shows that the mutual information H(X;Y) between originator and recipient in this communication channel cannot exceed a quantity χ, called the Holevo bound, which itself satisfies χ ≤ H(X), where H(X) is the entropy of the originator's classical information source.

Cambridge University Press eBooks•Feb 19, 2009•5 citations

Outlook

Vlatko Vedral

Abstract This book has discussed the foundations of quantum information science as well as the relationship between physics and information theory in general. It has considered the quantum equivalents of the Shannon coding and channel capacity theorems. The von Neumann entropy plays a role analogous to the Shannon entropy, and the Holevo bound is the analogue of Shannon's mutual information used to quantify the capacity of a classical channel. Quantum systems can process information more efficiently than classical systems in a number of different ways. Quantum teleportation and quantum dense coding can be performed using quantum entanglement. Entanglement is an excess of correlations that can exist in quantum physics and is impossible to reproduce classically (with what is termed “separable” states). The book has also demonstrated how to discriminate entangled from separable states using entanglement witnesses, as well as how to quantify entanglement, and looked at quantum computation and quantum algorithms.

Oxford University Press eBooks•Sep 28, 2006•0 citations

Information Theory Meets Quantum Chemistry: A Review and Perspective

Yilin Zhao, Dongbo Zhao, Chunying Rong +2 more

In this survey, we begin with a concise introduction to information theory within Shannon's framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. We then demonstrate how to apply these information-theoretic tools in quantum chemistry, adopting either classical or quantum formalisms based on the choice of information carrier involved.

Entropy•Jun 16, 2025•16 citations

Classical information

Vlatko Vedral

Abstract This book deals with the basic concept of information, its importance, and the similarities and differences between classical and quantum information. First, classical information theory and its application to communication are discussed. Quantum mechanics and communication based on the laws of quantum mechanics are then considered. Quantum entanglement, a key resource in quantum communication, is analysed both from the fundamental perspective and from the perspective of its information-processing capability. This book also introduces quantum computation (in the context of the search and factorisation algorithms) and the basics of quantum error correction. Throughout this book, the connections between information theory, thermodynamics, and (quantum) physics are emphasised and discussed. Other topics covered in the book are data compression, related measures of information such as relative entropy, joint entropy, conditional entropy, and mutual information, along with the capacity of a noisy channel.

Oxford University Press eBooks•Sep 28, 2006•0 citations

Classical and quantum relative entropy

Patrícia Szokol

The concept of entropy rst arose in the study of thermodynamics and statistical me- chanics. In information theory the so-called Shannon entropy plays a signi cant role, which measures how much information we gain, on average, when we learn the value of a random variable. An alternative interpretation is that the entropy of the random variable quanti es the amount of uncertainty about the random variable before we learn its value. The concept was introduced by Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication". In the rst part of the present work we consider this fundamental measure of information and show some of its properties, such as non-negativity. The relative entropy or the Kullback-Leibler divergence is a very useful measure of the distance between two prob- ability distributions. After introducing its de nition and other notions (joint entropy, conditional entropy or mutual information) we can prove additional assertions about the Shannon entropy and present statements about the relations between the above men- tioned quantities. In quantum information theory, there is a quantity, namely von Neumann entropy, which corresponds to Shannon entropy and originates from it. In the section Von Neu- mann entropy we will introduce this quantity which is a continuous functional on the states, or on their mathematical representatives, the density operators, and it ful ls in- equalities such as subadditivity and strong subaddititvity. Just as with the Shannon entropy, it is extremely useful to de ne a quantum version of the relative entropy, which is one of the most important numerical quantities appearing in quantum information theory. It is used as a measure of distinguishability between quantum states. Among others its properties are jointly convexity, monotonicity which are speci cally convenient. Transformations on structures which preserve operations, quantities or relations among elements appear in many areas of mathematics. These are the so-called preservers which show up also in physics and even in chemistry where they are usually called symmetries. Therefore, the study of preserver transformations is an important area of research. The third section of this work is based on a paper, in which Lajos Molnár described the structure every bijective map on the space of all density operators on a given Hilbert space, which preserves the relative entropy. This result is closely related to Wigner's famous theorem on the form of quantum mechanical symmetry transformations. The above theorem of Molnár can be extended into two directions. On the one hand, the result holds also in the case when the transformation under consideration acts on the space of all positive semide nite operators not only on density operators (i.e., we omit the condition of unit trace). On the other hand the statement remains valid even if we omit the strongly restrictive condition that the transformation is bijective. The second result is a joint work with Prof. Lajos Molnár which appeared in the journal Linear Algebra and Its Applications.

•May 6, 2010•0 citations
These notes (in French) provide a concise introduction to information theory, spanning its classical foundations: Shannon entropy, joint and conditional entropies, mutual information, statistical correlations, and Markov chains; as well as Shannon's source coding and channel capacity theorems, which govern the optimal compression and transmission of information through noisy channels. They also cover the quantum extensions of these concepts, including the density operator formalism, distances between quantum states, von Neumann entropy, coherent information, conditional entropy, entanglement theory, quantum discord, and quantum resource theory.
Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.
This chapter sets the basis of quantum information theory (QIT). The central purpose of QIT is to qualify the transmission of either classical or quantum information over quantum channels. The starting point of the QIT description is von Neumann entropy, S(ρ), which represents the quantum counterpart of Shannon's classical entropy, H(X). Such a definition rests on that of the density operator (or density matrix) of a quantum system, ρ, which plays a role similar to that of the random-events source X in Shannon's theory. As we shall see, there also exists an elegant and one-to-one correspondence between the quantum and classical definitions of the entropy variants relative entropy, joint entropy, conditional entropy, and mutual information. But such a similarity is only apparent. Indeed, one becomes rapidly convinced from a systematic analysis of the entropy's additivity rules that fundamental differences separate the two worlds. The classical notion of information correlation between two event sources for quantum states shall be referred to as quantum entanglement. We then define a quantum communication channel, which encodes and decodes classical information into or from quantum states. The analysis shows that the mutual information H(X;Y) between originator and recipient in this communication channel cannot exceed a quantity χ, called the Holevo bound, which itself satisfies χ ≤ H(X), where H(X) is the entropy of the originator's classical information source.
Abstract This book has discussed the foundations of quantum information science as well as the relationship between physics and information theory in general. It has considered the quantum equivalents of the Shannon coding and channel capacity theorems. The von Neumann entropy plays a role analogous to the Shannon entropy, and the Holevo bound is the analogue of Shannon's mutual information used to quantify the capacity of a classical channel. Quantum systems can process information more efficiently than classical systems in a number of different ways. Quantum teleportation and quantum dense coding can be performed using quantum entanglement. Entanglement is an excess of correlations that can exist in quantum physics and is impossible to reproduce classically (with what is termed “separable” states). The book has also demonstrated how to discriminate entangled from separable states using entanglement witnesses, as well as how to quantify entanglement, and looked at quantum computation and quantum algorithms.
Abstract This book deals with the basic concept of information, its importance, and the similarities and differences between classical and quantum information. First, classical information theory and its application to communication are discussed. Quantum mechanics and communication based on the laws of quantum mechanics are then considered. Quantum entanglement, a key resource in quantum communication, is analysed both from the fundamental perspective and from the perspective of its information-processing capability. This book also introduces quantum computation (in the context of the search and factorisation algorithms) and the basics of quantum error correction. Throughout this book, the connections between information theory, thermodynamics, and (quantum) physics are emphasised and discussed. Other topics covered in the book are data compression, related measures of information such as relative entropy, joint entropy, conditional entropy, and mutual information, along with the capacity of a noisy channel.
The concept of entropy rst arose in the study of thermodynamics and statistical me- chanics. In information theory the so-called Shannon entropy plays a signi cant role, which measures how much information we gain, on average, when we learn the value of a random variable. An alternative interpretation is that the entropy of the random variable quanti es the amount of uncertainty about the random variable before we learn its value. The concept was introduced by Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication". In the rst part of the present work we consider this fundamental measure of information and show some of its properties, such as non-negativity. The relative entropy or the Kullback-Leibler divergence is a very useful measure of the distance between two prob- ability distributions. After introducing its de nition and other notions (joint entropy, conditional entropy or mutual information) we can prove additional assertions about the Shannon entropy and present statements about the relations between the above men- tioned quantities. In quantum information theory, there is a quantity, namely von Neumann entropy, which corresponds to Shannon entropy and originates from it. In the section Von Neu- mann entropy we will introduce this quantity which is a continuous functional on the states, or on their mathematical representatives, the density operators, and it ful ls in- equalities such as subadditivity and strong subaddititvity. Just as with the Shannon entropy, it is extremely useful to de ne a quantum version of the relative entropy, which is one of the most important numerical quantities appearing in quantum information theory. It is used as a measure of distinguishability between quantum states. Among others its properties are jointly convexity, monotonicity which are speci cally convenient. Transformations on structures which preserve operations, quantities or relations among elements appear in many areas of mathematics. These are the so-called preservers which show up also in physics and even in chemistry where they are usually called symmetries. Therefore, the study of preserver transformations is an important area of research. The third section of this work is based on a paper, in which Lajos Molnár described the structure every bijective map on the space of all density operators on a given Hilbert space, which preserves the relative entropy. This result is closely related to Wigner's famous theorem on the form of quantum mechanical symmetry transformations. The above theorem of Molnár can be extended into two directions. On the one hand, the result holds also in the case when the transformation under consideration acts on the space of all positive semide nite operators not only on density operators (i.e., we omit the condition of unit trace). On the other hand the statement remains valid even if we omit the strongly restrictive condition that the transformation is bijective. The second result is a joint work with Prof. Lajos Molnár which appeared in the journal Linear Algebra and Its Applications.