Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2102.11887

Quantum Cross Entropy and Maximum Likelihood Principle

Zhou Shangnan, Yixu Wang

Published Feb 23, 2021• 14 citations
arXivHTMLPDF

Abstract

Quantum machine learning is an emerging field at the intersection of machine learning and quantum computing. Classical cross entropy plays a central role in machine learning. We define its quantum generalization, the quantum cross entropy, prove its lower bounds, and investigate its relation to quantum fidelity. In the classical case, minimizing cross entropy is equivalent to maximizing likelihood. In the quantum case, when the quantum cross entropy is constructed from quantum data undisturbed by quantum measurements, this relation holds. Classical cross entropy is equal to negative log-likelihood. When we obtain quantum cross entropy through empirical density matrix based on measurement outcomes, the quantum cross entropy is lower-bounded by negative log-likelihood. These two different scenarios illustrate the information loss when making quantum measurements. We conclude that to achieve the goal of full quantum machine learning, it is crucial to utilize the deferred measurement principle.

DOI
10.48550/arxiv.2102.11887

Related papers

Quantum Data Compression and Quantum Cross Entropy

Zhou Shangnan

The emerging field of quantum machine learning has the potential of revolutionizing our perspectives of quantum computing and artificial intelligence. In the predominantly empirical realm of quantum machine learning, a theoretical void persists. This paper addresses the gap by highlighting the quantum cross entropy, a pivotal counterpart to the classical cross entropy. We establish quantum cross entropy's role in quantum data compression, a fundamental machine learning task, by demonstrating that it acts as the compression rate for sub-optimal quantum source coding. Our approach involves a novel, universal quantum data compression protocol based on the quantum generalization of variable-length coding and the principle of quantum strong typicality. This reveals that quantum cross entropy can effectively serve as a loss function in quantum machine learning algorithms. Furthermore, we illustrate that the minimum of quantum cross entropy aligns with the von Neumann entropy, reinforcing its role as the optimal compression rate and underscoring its significance in advancing our understanding of quantum machine learning's theoretical framework.

arXiv:2106.13823•Jun 25, 2021•13 citations

Quantum statistical learning theory: concepts, regimes, and open problems

Ferhat Özgür Çatak

Statistical learning theory provides the mathematical foundations for understanding generalization from finite data. As quantum computing and quantum machine learning continue to develop, there is increasing interest in whether analogous principles can be formulated for learning settings involving quantum data, quantum models, or quantum computation. This paper offers an introductory perspective on quantum statistical learning theory. We begin by recalling the basic ideas of classical statistical learning theory, including empirical risk minimization, hypothesis classes, generalization, and sample complexity, and then discuss how these concepts must be adapted in the quantum setting. In particular, we distinguish between learning with classical data using quantum algorithms, learning with quantum models, and learning directly from quantum data. We then develop a resource-sensitive description that separates population risk, ideal empirical risk, and finite-shot empirical risk. This distinction makes explicit the different roles of training sample size and measurement budget. For a binary measurement-based classifier, we derive a finite-shot condition relating the required measurement budget to the sample size, confidence level, and classification margin. We further discuss how finite-sample generalization, finite-measurement estimation, and hardware imperfections can be represented as distinct contributions to experimentally observed learning performance. We conclude with a discussion of open problems and directions toward a more unified theory of generalization and learnability in quantum learning.

Quantum Information Processing•Oct 5, 2026•0 citations

Quantum Information and Entropy

Mark M. Wilde

In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.

Cambridge University Press eBooks•Nov 2, 2016•0 citations

Quantum Embeddings of Classical Data for Quantum Machine Learning

Gennaro De Luca, Yinong Chen

A major area of research in the field of quantum machine learning is the analysis of the loss landscape, particularly of variational quantum algorithms. These works often provide bounds and generalizations for various ansatzes and quantum embedding strategies. These analyses include approaches such as the Hessian and Fisher information matrices as well as generalized trigonometric polynomials. However, many such reviews often rely on a rotational encoding in practice or focus on few different approaches. The goal of this work is to statistically analyze experimental results from a quantum machine learning model that employs various different quantum embedding approaches, including those covered in related work, as well as the effect of measurement basis on the model.

Asian Conference on Artificial Intelligence Technology (ACAIT)•Dec 9, 2022•7 citations

Dual representations of classical and quantum entropies: Theory and perspectives: Invited keynote

Olivier Rioul

The MAXENT (maximum entropy) principle gives a natural correspondance between (classical and quantum) statistical mechanics and information theory. Its general formulation leverages on log-concavity (or Klein's inequality) and the celebrated information inequality, which is equivalent to Gibbs' inequality. The resulting convex optimization allows one to relate entropy to the log-canonical partition function by Legendre-Fenchel duality. This can be further generalized to dual variational characterization of relative entropy or Kullback-Leibler divergence, relating Gibbs variationnal principle to Donsker-Vardhan variational representations.From an information theoretical point of view, the Gibbs principle and its associated evidence lower bound serves as the basis of modern machine learning algorithms, such as the expectation-maximization (EM) algorithm and its restriction to parameters output by a neural network. Interestingly, a large class of alternating optimization algorithms including EM and Blahut-Arimoto for computing channel capacity and source rate-distortion functions can be cast in a generic information geometric alternating projections on convex sets of measures. On the other hand, Donsker-Vardhan representations, when restricted to the output of a neural network, allows one to efficiently estimate information measures, and when applied to parametric estimators with quadratic risk, provide HCR-like and CRB-like statistical lower bounds. An alternative derivation on such lower bounds in the Bayesian setting makes use of the Weyl-Heisenberg uncertainty principle, which opens up new perspectives.In this keynote, I will select and review several of the aforementioned topics, and discuss the interactions between classical and quantum interpretations.

•Jul 1, 2026•0 citations

Quantum Fair Machine Learning

Elija Perrier

In this paper, we inaugurate the field of quantum fair machine learning. We undertake a comparative analysis of differences and similarities between classical and quantum fair machine learning algorithms, specifying how the unique features of quantum computation alter measures, metrics and remediation strategies when quantum algorithms are subject to fairness constraints. We present the first results in quantum fair machine learning by demonstrating the use of Grover's search algorithm to satisfy statistical parity constraints imposed on quantum algorithms. We provide lower-bounds on iterations needed to achieve such statistical parity within ε-tolerance. We extend canonical Lipschitz-conditioned individual fairness criteria to the quantum setting using quantum metrics. We examine the consequences for typical measures of fairness in machine learning context when quantum information processing and quantum data are involved. Finally, we propose open questions and research programmes for this new field of interest to researchers in computer science, ethics and quantum computation.

arXiv:2102.00753•Proceedings of the AAAI/ACM Conference on AI Ethics and Society•Jul 21, 2021•19 citations
Quantum machine learning is an emerging field at the intersection of machine learning and quantum computing. Classical cross entropy plays a central role in machine learning. We define its quantum generalization, the quantum cross entropy, prove its lower bounds, and investigate its relation to quantum fidelity. In the classical case, minimizing cross entropy is equivalent to maximizing likelihood. In the quantum case, when the quantum cross entropy is constructed from quantum data undisturbed by quantum measurements, this relation holds. Classical cross entropy is equal to negative log-likelihood. When we obtain quantum cross entropy through empirical density matrix based on measurement outcomes, the quantum cross entropy is lower-bounded by negative log-likelihood. These two different scenarios illustrate the information loss when making quantum measurements. We conclude that to achieve the goal of full quantum machine learning, it is crucial to utilize the deferred measurement principle.
The emerging field of quantum machine learning has the potential of revolutionizing our perspectives of quantum computing and artificial intelligence. In the predominantly empirical realm of quantum machine learning, a theoretical void persists. This paper addresses the gap by highlighting the quantum cross entropy, a pivotal counterpart to the classical cross entropy. We establish quantum cross entropy's role in quantum data compression, a fundamental machine learning task, by demonstrating that it acts as the compression rate for sub-optimal quantum source coding. Our approach involves a novel, universal quantum data compression protocol based on the quantum generalization of variable-length coding and the principle of quantum strong typicality. This reveals that quantum cross entropy can effectively serve as a loss function in quantum machine learning algorithms. Furthermore, we illustrate that the minimum of quantum cross entropy aligns with the von Neumann entropy, reinforcing its role as the optimal compression rate and underscoring its significance in advancing our understanding of quantum machine learning's theoretical framework.
Statistical learning theory provides the mathematical foundations for understanding generalization from finite data. As quantum computing and quantum machine learning continue to develop, there is increasing interest in whether analogous principles can be formulated for learning settings involving quantum data, quantum models, or quantum computation. This paper offers an introductory perspective on quantum statistical learning theory. We begin by recalling the basic ideas of classical statistical learning theory, including empirical risk minimization, hypothesis classes, generalization, and sample complexity, and then discuss how these concepts must be adapted in the quantum setting. In particular, we distinguish between learning with classical data using quantum algorithms, learning with quantum models, and learning directly from quantum data. We then develop a resource-sensitive description that separates population risk, ideal empirical risk, and finite-shot empirical risk. This distinction makes explicit the different roles of training sample size and measurement budget. For a binary measurement-based classifier, we derive a finite-shot condition relating the required measurement budget to the sample size, confidence level, and classification margin. We further discuss how finite-sample generalization, finite-measurement estimation, and hardware imperfections can be represented as distinct contributions to experimentally observed learning performance. We conclude with a discussion of open problems and directions toward a more unified theory of generalization and learnability in quantum learning.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.
A major area of research in the field of quantum machine learning is the analysis of the loss landscape, particularly of variational quantum algorithms. These works often provide bounds and generalizations for various ansatzes and quantum embedding strategies. These analyses include approaches such as the Hessian and Fisher information matrices as well as generalized trigonometric polynomials. However, many such reviews often rely on a rotational encoding in practice or focus on few different approaches. The goal of this work is to statistically analyze experimental results from a quantum machine learning model that employs various different quantum embedding approaches, including those covered in related work, as well as the effect of measurement basis on the model.
The MAXENT (maximum entropy) principle gives a natural correspondance between (classical and quantum) statistical mechanics and information theory. Its general formulation leverages on log-concavity (or Klein's inequality) and the celebrated information inequality, which is equivalent to Gibbs' inequality. The resulting convex optimization allows one to relate entropy to the log-canonical partition function by Legendre-Fenchel duality. This can be further generalized to dual variational characterization of relative entropy or Kullback-Leibler divergence, relating Gibbs variationnal principle to Donsker-Vardhan variational representations.From an information theoretical point of view, the Gibbs principle and its associated evidence lower bound serves as the basis of modern machine learning algorithms, such as the expectation-maximization (EM) algorithm and its restriction to parameters output by a neural network. Interestingly, a large class of alternating optimization algorithms including EM and Blahut-Arimoto for computing channel capacity and source rate-distortion functions can be cast in a generic information geometric alternating projections on convex sets of measures. On the other hand, Donsker-Vardhan representations, when restricted to the output of a neural network, allows one to efficiently estimate information measures, and when applied to parametric estimators with quadratic risk, provide HCR-like and CRB-like statistical lower bounds. An alternative derivation on such lower bounds in the Bayesian setting makes use of the Weyl-Heisenberg uncertainty principle, which opens up new perspectives.In this keynote, I will select and review several of the aforementioned topics, and discuss the interactions between classical and quantum interpretations.
In this paper, we inaugurate the field of quantum fair machine learning. We undertake a comparative analysis of differences and similarities between classical and quantum fair machine learning algorithms, specifying how the unique features of quantum computation alter measures, metrics and remediation strategies when quantum algorithms are subject to fairness constraints. We present the first results in quantum fair machine learning by demonstrating the use of Grover's search algorithm to satisfy statistical parity constraints imposed on quantum algorithms. We provide lower-bounds on iterations needed to achieve such statistical parity within ε-tolerance. We extend canonical Lipschitz-conditioned individual fairness criteria to the quantum setting using quantum metrics. We examine the consequences for typical measures of fairness in machine learning context when quantum information processing and quantum data are involved. Finally, we propose open questions and research programmes for this new field of interest to researchers in computer science, ethics and quantum computation.