Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1801.00315 · Quantum Science and Technology

Learning relevant features of data with multi-scale tensor networks

E Miles Stoudenmire

Published Mar 28, 2018• 156 citations
arXivHTMLPDF

Abstract

Abstract Inspired by coarse-graining approaches used in physics, we show how similar algorithms can be adapted for data. The resulting algorithms are based on layered tree tensor networks and scale linearly with both the dimension of the input and the training set size. Computing most of the layers with an unsupervised algorithm, then optimizing just the top layer for supervised classification of the MNIST and fashion MNIST data sets gives very good results. We also discuss mixing a prior guess for supervised weights together with an unsupervised representation of the data, yielding a smaller number of features nevertheless able to give good performance.

DOI
10.1088/2058-9565/aaba1a

Related papers

Multi-scale tensor network architecture for machine learning

Justin A. Reyes, Edwin Miles Stoudenmire

Abstract We present an algorithm for supervised learning using tensor networks, employing a step of data pre-processing by coarse-graining through a sequence of wavelet transformations. These transformations are represented as a set of tensor network layers identical to those in a multi-scale entanglement renormalization ansatz tensor network. We perform supervised learning and regression tasks through a model based on a matrix product states (MPSs) acting on the coarse-grained data. Because the entire model consists of tensor contractions (apart from the initial non-linear feature map), we can adaptively fine-grain the optimized MPS model ‘backwards’ through the layers with essentially no loss in performance. The MPS itself is trained using an adaptive algorithm based on the density matrix renormalization group algorithm. We test our methods by performing a classification task on audio data and a regression task on temperature time-series data, studying the dependence of training accuracy on the number of coarse-graining layers and showing how fine-graining through the network may be used to initialize models which access finer-scale features.

arXiv:2001.08286•Machine Learning Science and Technology•May 11, 2021•69 citations

Supervised Learning with Quantum-Inspired Tensor Networks

Edwin Miles Stoudenmire, David J. Schwab

Tensor networks are efficient representations of high-dimensional tensors which have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing such networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize models for classifying images. For the MNIST data set we obtain less than 1% test set classification error. We discuss how the tensor network form imparts additional structure to the learned model and suggest a possible generative interpretation.

arXiv:1605.05775•May 18, 2016•185 citations

Supervised Learning with Tensor Networks

Edwin Miles Stoudenmire, David J. Schwab

Tensor networks are approximations of high-order tensors which are efficient to work with and have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing tensor networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize non-linear kernel learning models. For the MNIST data set we obtain less than 1% test set classification error. We discuss an interpretation of the additional structure imparted by the tensor network to the learned model.

neural information processing systems•Jan 1, 2016•245 citations

Tree-tensor-network classifiers for machine learning: From quantum inspired to quantum assisted

Michael L. Wall, Giuseppe D’Aguanno

We describe a quantum-assisted machine learning method in which multivariate data are encoded into quantum states in a Hilbert space whose dimension is exponentially large in the length of the data vector. Learning in this space occurs through applying a low-depth quantum circuit with a tree-tensor-network (TTN) topology acting as an unsupervised feature extractor to identify the most relevant quantum states in a data-driven fashion and then applying a supervised linear classifier encoding the class decision in a small-dimensional quantum register. We present tools for making TTN classifiers amenable to implementation on gate-based quantum computing devices, including an embedding map with accuracy similar to the recently defined exponential machines (A. Novikov et al., arXiv:1605.03795) but which produces valid quantum state embeddings of classical data vectors, and the use of manifold-based gradient optimization schemes to produce isometric operations mapping quantum states to a register of qubits defining a class decision. We detail methods for efficiently obtaining one-point and two-point correlation functions of the vectors defining the decision boundary of the quantum model, which can be used for model interpretability, as well as methods for obtaining classification decisions from partial data vectors. Further, we show that the use of isometric tensors can significantly aid in the human interpretability of the correlation functions extracted from the decision weights and may produce models that are less susceptible to adversarial perturbations. Finally, we discuss in detail the problem of compiling classically optimized isometric TTN models into unitary operations to be run on quantum computers and how isometric models requiring postselection on quantum hardware can be used to precondition variational Ans\"atze for models without postselection. We demonstrate our methodologies in applications utilizing the MNIST database of handwritten digits and a multivariate time-series data set of human activity recognition.

arXiv:2104.02249•Physical Review A•Oct 7, 2021•38 citations

Multi-layered tensor networks for image classification

Raghavendra Selvan, Silas Nyboe Ørting, Erik B. Dam

The recently introduced locally orderless tensor network (LoTeNet) for supervised image classification uses matrix product state (MPS) operations on grids of transformed image patches. The resulting patch representations are combined back together into the image space and aggregated hierarchically using multiple MPS blocks per layer to obtain the final decision rules. In this work, we propose a non-patch based modification to LoTeNet that performs one MPS operation per layer, instead of several patch-level operations. The spatial information in the input images to MPS blocks at each layer is squeezed into the feature dimension, similar to LoTeNet, to maximise retained spatial correlation between pixels when images are flattened into 1D vectors. The proposed multi-layered tensor network (MLTN) is capable of learning linear decision boundaries in high dimensional spaces in a multi-layered setting, which results in a reduction in the computation cost compared to LoTeNet without any degradation in performance.

arXiv:2011.06982•Nov 13, 2020•4 citations

Machine Learning With Tree Tensor Networks, CP Rank Constraints, and Tensor Dropout

Hao Chen, Thomas Barthel

Tensor networks developed in the context of condensed matter physics try to approximate order- N tensors with a reduced number of degrees of freedom that is only polynomial in N and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio b=4 reaches a test set accuracy of 90.3% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.

arXiv:2305.19440•IEEE Transactions on Pattern Analysis and Machine Intelligence•May 2, 2024•32 citations
Abstract Inspired by coarse-graining approaches used in physics, we show how similar algorithms can be adapted for data. The resulting algorithms are based on layered tree tensor networks and scale linearly with both the dimension of the input and the training set size. Computing most of the layers with an unsupervised algorithm, then optimizing just the top layer for supervised classification of the MNIST and fashion MNIST data sets gives very good results. We also discuss mixing a prior guess for supervised weights together with an unsupervised representation of the data, yielding a smaller number of features nevertheless able to give good performance.
Abstract We present an algorithm for supervised learning using tensor networks, employing a step of data pre-processing by coarse-graining through a sequence of wavelet transformations. These transformations are represented as a set of tensor network layers identical to those in a multi-scale entanglement renormalization ansatz tensor network. We perform supervised learning and regression tasks through a model based on a matrix product states (MPSs) acting on the coarse-grained data. Because the entire model consists of tensor contractions (apart from the initial non-linear feature map), we can adaptively fine-grain the optimized MPS model ‘backwards’ through the layers with essentially no loss in performance. The MPS itself is trained using an adaptive algorithm based on the density matrix renormalization group algorithm. We test our methods by performing a classification task on audio data and a regression task on temperature time-series data, studying the dependence of training accuracy on the number of coarse-graining layers and showing how fine-graining through the network may be used to initialize models which access finer-scale features.
Tensor networks are efficient representations of high-dimensional tensors which have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing such networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize models for classifying images. For the MNIST data set we obtain less than 1% test set classification error. We discuss how the tensor network form imparts additional structure to the learned model and suggest a possible generative interpretation.
Tensor networks are approximations of high-order tensors which are efficient to work with and have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing tensor networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize non-linear kernel learning models. For the MNIST data set we obtain less than 1% test set classification error. We discuss an interpretation of the additional structure imparted by the tensor network to the learned model.
We describe a quantum-assisted machine learning method in which multivariate data are encoded into quantum states in a Hilbert space whose dimension is exponentially large in the length of the data vector. Learning in this space occurs through applying a low-depth quantum circuit with a tree-tensor-network (TTN) topology acting as an unsupervised feature extractor to identify the most relevant quantum states in a data-driven fashion and then applying a supervised linear classifier encoding the class decision in a small-dimensional quantum register. We present tools for making TTN classifiers amenable to implementation on gate-based quantum computing devices, including an embedding map with accuracy similar to the recently defined exponential machines (A. Novikov et al., arXiv:1605.03795) but which produces valid quantum state embeddings of classical data vectors, and the use of manifold-based gradient optimization schemes to produce isometric operations mapping quantum states to a register of qubits defining a class decision. We detail methods for efficiently obtaining one-point and two-point correlation functions of the vectors defining the decision boundary of the quantum model, which can be used for model interpretability, as well as methods for obtaining classification decisions from partial data vectors. Further, we show that the use of isometric tensors can significantly aid in the human interpretability of the correlation functions extracted from the decision weights and may produce models that are less susceptible to adversarial perturbations. Finally, we discuss in detail the problem of compiling classically optimized isometric TTN models into unitary operations to be run on quantum computers and how isometric models requiring postselection on quantum hardware can be used to precondition variational Ans\"atze for models without postselection. We demonstrate our methodologies in applications utilizing the MNIST database of handwritten digits and a multivariate time-series data set of human activity recognition.
The recently introduced locally orderless tensor network (LoTeNet) for supervised image classification uses matrix product state (MPS) operations on grids of transformed image patches. The resulting patch representations are combined back together into the image space and aggregated hierarchically using multiple MPS blocks per layer to obtain the final decision rules. In this work, we propose a non-patch based modification to LoTeNet that performs one MPS operation per layer, instead of several patch-level operations. The spatial information in the input images to MPS blocks at each layer is squeezed into the feature dimension, similar to LoTeNet, to maximise retained spatial correlation between pixels when images are flattened into 1D vectors. The proposed multi-layered tensor network (MLTN) is capable of learning linear decision boundaries in high dimensional spaces in a multi-layered setting, which results in a reduction in the computation cost compared to LoTeNet without any degradation in performance.
Tensor networks developed in the context of condensed matter physics try to approximate order- N tensors with a reduced number of degrees of freedom that is only polynomial in N and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio b=4 reaches a test set accuracy of 90.3% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.