Modeling joint probability distributions over sequences has been studied from many perspectives. The physics community developed matrix product states, a tensor-train decomposition for probabilistic modeling, motivated by the need to tractably model many-body systems. But similar models have also been studied in the stochastic processes and weighted automata literature, with little work on how these bodies of work relate to each other. We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences. We demonstrate several equivalence results between models used in these three communities: (i) uniform variants of matrix product states, Born machines and locally purified states from the quantum tensor networks literature, (ii) predictive state representations, hidden Markov models, norm-observable operator models and hidden quantum Markov models from the stochastic process literature,and (iii) stochastic weighted automata, probabilistic automata and quadratic automata from the formal languages literature. Such connections may open the door for results and methods developed in one area to be applied in another.
Modeling joint probability distributions over sequences has been studied from many perspectives. The physics community developed matrix product states, a tensor-train decomposition for probabilistic modeling, motivated by the need to tractably model many-body systems. But similar models have also been studied in the stochastic processes and weighted automata literature, with little work on how these bodies of work relate to each other. We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences. We demonstrate several equivalence results between models used in these three communities: (i) uniform variants of matrix product states, Born machines and locally purified states from the quantum tensor networks literature, (ii) predictive state representations, hidden Markov models, norm-observable operator models and hidden quantum Markov models from the stochastic process literature,and (iii) stochastic weighted automata, probabilistic automata and quadratic automata from the formal languages literature. Such connections may open the door for results and methods developed in one area to be applied in another.
Tensor-network techniques have enjoyed outstanding success in physics, and have recently attracted attention in machine learning, both as a tool for the formulation of new learning algorithms and for enhancing the mathematical understanding of existing methods. Inspired by these developments, and the natural correspondence between tensor networks and probabilistic graphical models, we provide a rigorous analysis of the expressive power of various tensor-network factorizations of discrete multivariate probability distributions. These factorizations include non-negative tensor-trains/MPS, which are in correspondence with hidden Markov models, and Born machines, which are naturally related to local quantum circuits. When used to model probability distributions, they exhibit tractable likelihoods and admit efficient learning algorithms. Interestingly, we prove that there exist probability distributions for which there are unbounded separations between the resource requirements of some of these tensor-network factorizations. Particularly surprising is the fact that using complex instead of real tensors can lead to an arbitrarily large reduction in the number of parameters of the network. Additionally, we introduce locally purified states (LPS), a new factorization inspired by techniques for the simulation of quantum systems, with provably better expressive power than all other representations considered. The ramifications of this result are explored through numerical experiments. Our findings imply that LPS should be considered over hidden Markov models, and furthermore provide guidelines for the design of local quantum circuits for probabilistic modeling.
Algorithms developed to solve many-body quantum problems, like tensor networks, can turn into powerful quantum-inspired tools to tackle issues in the classical domain. This work focuses on matrix product operators, a prominent numerical technique to study many-body quantum systems, especially in one dimension. It has been previously shown that such a tool can be used for classification, learning of deterministic sequence-to-sequence processes, and generic quantum processes. We further develop a matrix product operator algorithm to learn probabilistic sequence-to-sequence processes and apply this algorithm to probabilistic cellular automata. This new approach can accurately learn probabilistic cellular automata processes in different conditions, even when the process is a probabilistic mixture of different chaotic rules. In addition, we find that the ability to learn these dynamics is a function of the bitwise difference between the rules and whether one is much more likely than the other. Published by the American Physical Society 2024
Predicting a stochastic process' future lies at the heart of many scientific areas.A predictive model extracts information from a stochastic process' past and uses it to generate future statistics.There has been significant amount of effort expended towards finding optimal predictive models that minimize the required amount of past information.By taking advantage of quantum resources, quantum models have been shown to reduce the memory requirements beyond classical limits.Meanwhile, tensor networks are an extremely useful mathematical tool for understanding quantum many-body systems.In this thesis, we explore the connection between tensor networks and quantum predictive models.A particular class of tensor networks, matrix product states (MPS), is utilized to further improve quantum models.First, we establish a connection between matrix product states (MPS) and the optimal classical predictive models.MPS methods offer a systematic method for constructing quantum predictive models and even an improved method for computing the amount of quantum memory.Second, we show that for some families of stochastic processes our method allows us to construct quantum models with unbounded memory advantages over the optimal classical models.Third, we propose a family of divergence measures to quantify the distance between two stochastic processes.This family of divergence measures overcomes certain weaknesses of the existing measures.Moreover, we propose an efficient means of computing the divergence measure using our MPS methods.Finally, we address the problem of constructing quantum predictive models solely from the output of an observed stochastic process.We use machine learning methods to propose a systematic algorithm for learning the quantum model.This provides an alternative method for constructing quantum models with fixed amounts of amount of memory.
Tensor-network techniques have recently proven useful in machine learning, both as a tool for the formulation of new learning algorithms and for enhancing the mathematical understanding of existing methods.Inspired by these developments, and the natural correspondence between tensor networks and probabilistic graphical models, we provide a rigorous analysis of the expressive power of various tensornetwork factorizations of discrete multivariate probability distributions.These factorizations include non-negative tensor-trains/MPS, which are in correspondence with hidden Markov models, and Born machines, which are naturally related to the probabilistic interpretation of quantum circuits.When used to model probability distributions, they exhibit tractable likelihoods and admit efficient learning algorithms.Interestingly, we prove that there exist probability distributions for which there are unbounded separations between the resource requirements of some of these tensor-network factorizations.Of particular interest, using complex instead of real tensors can lead to an arbitrarily large reduction in the number of parameters of the network.Additionally, we introduce locally purified states (LPS), a new factorization inspired by techniques for the simulation of quantum systems, with provably better expressive power than all other representations considered.The ramifications of this result are explored through numerical experiments.
This chapter presents tensor networks as a unifying framework for quantum-inspired machine learning and for the analysis of near-term quantum learning models. Motivated by the challenge of high dimensionality, this chapter shows how tensor networks exploit locality, low-rank structure, and controlled correlations to represent high-order data efficiently while avoiding the exponential scaling of naïve tensor representations. Fundamental concepts including tensors, tensor contraction, and graphical network representations are introduced, followed by the development of key architectures such as Matrix Product States (Tensor Trains) and the Multi-scale Entanglement Renormalization Ansatz. Their roles in controlling expressivity, entanglement, and computational cost are emphasized throughout. This chapter further demonstrates the close correspondence between tensor networks and quantum circuits, showing how shallow quantum models can be interpreted as low-bond-dimension tensor networks and how this perspective clarifies classical simulability, representational capacity, and noisy intermediate-scale quantum (NISQ)-era limitations. Hybrid quantum-tensor network architectures are presented as a practical route for combining classical compression and representation learning with targeted quantum processing. Applications in natural language processing, biological sequence learning, and image and signal processing illustrate how tensor network methods serve as both efficient classical models and structured surrogates for quantum learning systems. Hands-on Qiskit and PennyLane examples connect these ideas to executable machine learning workflows.
Tensor networks were developed in the context of many-body physics as compressed representations of multiparticle quantum states. These representations mitigate the exponential complexity of many-body systems by capturing only the most relevant dependencies. Due to the formal similarity between quantum entanglement and statistical correlations, tensor networks have recently been integrated in machine learning, operating both as alternative learning architectures and as decompositions of components of neural networks. The expectation is that the theoretical understanding of tensor networks developed within quantum many-body physics leads to novel methods that offer advantages in terms of computational efficiency, explainability, or privacy. Here we review the use of tensor networks in the context of machine learning, providing a critical assessment of the state of the art, the potential advantages, and the challenges that must be overcome.