Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2108.08659 · IEEE Transactions on Artificial Intelligence

Residual Tensor Train: A Quantum-Inspired Approach for Learning Multiple Multilinear Correlations

Yiwei Chen, Yu Pan, Daoyi Dong

Published Jul 27, 2022• 17 citations
arXivHTMLPDF

Abstract

States of quantum many-body systems are defined in a high-dimensional Hilbert space, where rich and complex interactions among subsystems can be modeled. In machine learning, complex multiple multilinear correlations may also exist within input features. In this article, we present a quantum-inspired multilinear model, named residual tensor train (ResTT), to capture the multiple multilinear correlations of features, from low to high orders, within a single model. ResTT is able to build a robust decision boundary in a high-dimensional space for solving fitting and classification tasks. In particular, we prove that the fully connected layer and the Volterra series can be taken as special cases of ResTT. Furthermore, we derive the rule for weight initialization that stabilizes the training of ResTT based on a mean-field analysis. We prove that such a rule is much more relaxed than that of tensor train (TT), which means that ResTT can easily address the vanishing and exploding gradient problem that exists in the existing TT models. Numerical experiments demonstrate that ResTT outperforms the state-of-the-art tensor network and benchmark deep learning models on MNIST and Fashion-MNIST datasets. Moreover, ResTT achieves better performance than other statistical methods on two practical examples with limited data, which are known to have complex feature interactions.

DOI
10.1109/tai.2022.3194132

Related papers

Robust supervised learning based on tensor network method

Yiwei Chen, K. Guo, Yilin Pan

The formalism of Tensor Network (TN) provides a compact way to approximate many-body quantum states with 1D chain of tensors. The 1D chain of tensors is found to be efficient in capturing the local correlations between neighboring subsystems, and machine learning approaches have been proposed using artificial neural networks (NN) of similar structure. However, a long chain of tensors is difficult to train due to exploding and vanishing gradients. In this paper, we propose methods to decompose the long-chain TN into short chains, which could improve the convergence property of the training algorithm by allowing stable stochastic gradient descent (SGD). In addition, the short-chain methods are robust to network initializations. Numerical experiments show that the short-chain TN achieves almost the same classification accuracy on MNIST dataset as LeNet-5 with less trainable network parameters and connections.

Youth Academic Annual Conference of Chinese Association of Automation (YAC)•May 1, 2018•5 citations

An end-to-end trainable hybrid classical-quantum classifier

Samuel Yen-Chi Chen, Chih-Min Huang, Chia-Wei Hsing +1 more

Abstract We introduce a hybrid model combining a quantum-inspired tensor network and a variational quantum circuit to perform supervised learning tasks. This architecture allows for the classical and quantum parts of the model to be trained simultaneously, providing an end-to-end training framework. We show that compared to the principal component analysis, a tensor network based on the matrix product state with low bond dimensions performs better as a feature extractor for the input data of the variational quantum circuit in the binary and ternary classification of MNIST and Fashion-MNIST datasets. The architecture is highly adaptable and the classical-quantum boundary can be adjusted according to the availability of the quantum resource by exploiting the correspondence between tensor networks and quantum circuits.

arXiv:2011.14651•Machine Learning Science and Technology•Jun 30, 2021•143 citations

Hybrid Tensor Networks: The Integration of Quantum and Classical Machine Learning

Jiaqi Yao

Tensor networks can compactly represent high-dimensional data in the form of low-rank tensor decompositions, which helps capture complex structures and correlations within the data. However, their inherently multilinear nature limits their potential as universal approximators. To address this limitation, tensor networks are combined with classical neural network layers, introducing nonlinear elements to enhance the model’s expressive power. The Hybrid Tensor Networks (HTN) system encompasses two prevalent tensor network configurations, Matrix Product States and Tree Tensor Networks, which facilitates the adaptable design of models customized for particular machine learning applications. This framework has been validated on classification and image reconstruction tasks, demonstrating its potential to handle complex data structures and its applicability within the machine learning field. Employing this approach, the goal is to establish a foundation for future research on hybrid quantum-classical models, thereby opening up avenues for the realm of quantum machine learning.

International Conference on Computer, Vision and Intelligent Technology (ICCVIT)•Nov 24, 2024•1 citation

Quantum neural networks with deep residual learning.

Yanying Liang, Wei Peng, Zhu‐Jun Zheng +2 more

Inspired by the success of neural networks in the classical machine learning tasks, there has been tremendous effort to develop quantum neural networks (QNNs), especially for quantum data or tasks that are inherently quantum in nature. Currently, with the imminent advent of quantum computing processors to evade the computational and thermodynamic limitation of classical computations,} designing an efficient quantum neural network becomes a valuable task in quantum machine learning. In this paper, a novel quantum neural network with deep residual learning (ResQNN) is proposed. {Specifically, a multiple layer quantum perceptron with residual connection is provided. Our ResQNN is able to learn an unknown unitary and get remarkable performance. Besides, the model can be trained with an end-to-end fashion, as analogue of the backpropagation in the classical neural networks. To explore the effectiveness of our ResQNN , we perform extensive experiments on the quantum data under the setting of both clean and noisy training data. The experimental results show the robustness and superiority of our ResQNN, when compared to current remarkable work, which is from \textit{Nature communications, 2020}. Moreover, when training with higher proportion of noisy data, the superiority of our ResQNN model can be even significant, which implies the generalization ability and the remarkable tolerance for noisy data of the proposed method.

•Dec 14, 2020•0 citations

Tensor ring optimized quantum-enhanced tensor neural networks

Debanjan Konar, Dheeraj Peddireddy, Bijaya Ketan Panigrahi +1 more

Abstract Quantum machine learning researchers often rely on incorporating tensor networks (TN) into deep neural networks (DNN) and variational optimization. However, the standard optimization techniques used for training the contracted trainable weights of each model layer suffer from the correlations and entanglement structure between the model parameters in classical implementations. To address this issue, a multi-layer design of a tensor ring optimized variational quantum learning classifier (Quan-TR) comprising cascading entangling gates replacing the fully connected (dense) layers of a TN is proposed, and it is referred to as tensor ring optimized quantum-enhanced tensor neural networks (TR-QNet). TR-QNet parameters are optimized using the stochastic gradient descent algorithm on qubit measurements. The proposed TR-QNet is evaluated on three distinct datasets, namely Iris, MNIST, and CIFAR-10, to demonstrate the enhanced precision achieved for binary classification. In quantum simulations, the proposed TR-QNet achieves promising precision of $$94.5\%$$ 94.5 % , $$86.16\%$$ 86.16 % , and $$83.54\%$$ 83.54 % on the Iris, MNIST, and CIFAR-10 datasets. Benchmark studies have been conducted on state-of-the-art quantum and classical implementations of TN models to show the efficacy of the proposed TR-QNet. Moreover, the scalability of TR-QNet highlights its potential for exhibiting in deep learning applications on a large scale. The PyTorch implementation of TR-QNet is available on Github https://github.com/konar1987/TR-QNet/ .

arXiv:2310.01515•Quantum Machine Intelligence•Apr 22, 2025•4 citations

Residual matrix product state for machine learning

Ye-Ming Meng, Jing Zhang, Peng Zhang +2 more

Tensor network, which originates from quantum physics, is emerging as an efficient tool for classical and quantum machine learning. Nevertheless, there still exists a considerable accuracy gap between tensor network and the sophisticated neural network models for classical machine learning. In this work, we combine the ideas of matrix product state (MPS), the simplest tensor network structure, and residual neural network and propose the residual matrix product state (ResMPS). The ResMPS can be treated as a network where its layers map the "hidden" features to the outputs (e.g., classifications), and the variational parameters of the layers are the functions of the features of the samples (e.g., pixels of images). This is different from neural network, where the layers map feed-forwardly the features to the output. The ResMPS can equip with the non-linear activations and dropout layers, and outperforms the state-of-the-art tensor network models in terms of efficiency, stability, and expression power. Besides, ResMPS is interpretable from the perspective of polynomial expansion, where the factorization and exponential machines naturally emerge. Our work contributes to connecting and hybridizing neural and tensor networks, which is crucial to further enhance our understanding of the working mechanisms and improve the performance of both models.

arXiv:2012.11841•SciPost Physics•Jun 2, 2023•14 citations
States of quantum many-body systems are defined in a high-dimensional Hilbert space, where rich and complex interactions among subsystems can be modeled. In machine learning, complex multiple multilinear correlations may also exist within input features. In this article, we present a quantum-inspired multilinear model, named residual tensor train (ResTT), to capture the multiple multilinear correlations of features, from low to high orders, within a single model. ResTT is able to build a robust decision boundary in a high-dimensional space for solving fitting and classification tasks. In particular, we prove that the fully connected layer and the Volterra series can be taken as special cases of ResTT. Furthermore, we derive the rule for weight initialization that stabilizes the training of ResTT based on a mean-field analysis. We prove that such a rule is much more relaxed than that of tensor train (TT), which means that ResTT can easily address the vanishing and exploding gradient problem that exists in the existing TT models. Numerical experiments demonstrate that ResTT outperforms the state-of-the-art tensor network and benchmark deep learning models on MNIST and Fashion-MNIST datasets. Moreover, ResTT achieves better performance than other statistical methods on two practical examples with limited data, which are known to have complex feature interactions.
The formalism of Tensor Network (TN) provides a compact way to approximate many-body quantum states with 1D chain of tensors. The 1D chain of tensors is found to be efficient in capturing the local correlations between neighboring subsystems, and machine learning approaches have been proposed using artificial neural networks (NN) of similar structure. However, a long chain of tensors is difficult to train due to exploding and vanishing gradients. In this paper, we propose methods to decompose the long-chain TN into short chains, which could improve the convergence property of the training algorithm by allowing stable stochastic gradient descent (SGD). In addition, the short-chain methods are robust to network initializations. Numerical experiments show that the short-chain TN achieves almost the same classification accuracy on MNIST dataset as LeNet-5 with less trainable network parameters and connections.
Abstract We introduce a hybrid model combining a quantum-inspired tensor network and a variational quantum circuit to perform supervised learning tasks. This architecture allows for the classical and quantum parts of the model to be trained simultaneously, providing an end-to-end training framework. We show that compared to the principal component analysis, a tensor network based on the matrix product state with low bond dimensions performs better as a feature extractor for the input data of the variational quantum circuit in the binary and ternary classification of MNIST and Fashion-MNIST datasets. The architecture is highly adaptable and the classical-quantum boundary can be adjusted according to the availability of the quantum resource by exploiting the correspondence between tensor networks and quantum circuits.
Tensor networks can compactly represent high-dimensional data in the form of low-rank tensor decompositions, which helps capture complex structures and correlations within the data. However, their inherently multilinear nature limits their potential as universal approximators. To address this limitation, tensor networks are combined with classical neural network layers, introducing nonlinear elements to enhance the model’s expressive power. The Hybrid Tensor Networks (HTN) system encompasses two prevalent tensor network configurations, Matrix Product States and Tree Tensor Networks, which facilitates the adaptable design of models customized for particular machine learning applications. This framework has been validated on classification and image reconstruction tasks, demonstrating its potential to handle complex data structures and its applicability within the machine learning field. Employing this approach, the goal is to establish a foundation for future research on hybrid quantum-classical models, thereby opening up avenues for the realm of quantum machine learning.
Inspired by the success of neural networks in the classical machine learning tasks, there has been tremendous effort to develop quantum neural networks (QNNs), especially for quantum data or tasks that are inherently quantum in nature. Currently, with the imminent advent of quantum computing processors to evade the computational and thermodynamic limitation of classical computations,} designing an efficient quantum neural network becomes a valuable task in quantum machine learning. In this paper, a novel quantum neural network with deep residual learning (ResQNN) is proposed. {Specifically, a multiple layer quantum perceptron with residual connection is provided. Our ResQNN is able to learn an unknown unitary and get remarkable performance. Besides, the model can be trained with an end-to-end fashion, as analogue of the backpropagation in the classical neural networks. To explore the effectiveness of our ResQNN , we perform extensive experiments on the quantum data under the setting of both clean and noisy training data. The experimental results show the robustness and superiority of our ResQNN, when compared to current remarkable work, which is from \textit{Nature communications, 2020}. Moreover, when training with higher proportion of noisy data, the superiority of our ResQNN model can be even significant, which implies the generalization ability and the remarkable tolerance for noisy data of the proposed method.
Abstract Quantum machine learning researchers often rely on incorporating tensor networks (TN) into deep neural networks (DNN) and variational optimization. However, the standard optimization techniques used for training the contracted trainable weights of each model layer suffer from the correlations and entanglement structure between the model parameters in classical implementations. To address this issue, a multi-layer design of a tensor ring optimized variational quantum learning classifier (Quan-TR) comprising cascading entangling gates replacing the fully connected (dense) layers of a TN is proposed, and it is referred to as tensor ring optimized quantum-enhanced tensor neural networks (TR-QNet). TR-QNet parameters are optimized using the stochastic gradient descent algorithm on qubit measurements. The proposed TR-QNet is evaluated on three distinct datasets, namely Iris, MNIST, and CIFAR-10, to demonstrate the enhanced precision achieved for binary classification. In quantum simulations, the proposed TR-QNet achieves promising precision of 94.5%94.5\%94.5% 94.5 % , 86.16%86.16\%86.16% 86.16 % , and 83.54%83.54\%83.54% 83.54 % on the Iris, MNIST, and CIFAR-10 datasets. Benchmark studies have been conducted on state-of-the-art quantum and classical implementations of TN models to show the efficacy of the proposed TR-QNet. Moreover, the scalability of TR-QNet highlights its potential for exhibiting in deep learning applications on a large scale. The PyTorch implementation of TR-QNet is available on Github https://github.com/konar1987/TR-QNet/ .
Tensor network, which originates from quantum physics, is emerging as an efficient tool for classical and quantum machine learning. Nevertheless, there still exists a considerable accuracy gap between tensor network and the sophisticated neural network models for classical machine learning. In this work, we combine the ideas of matrix product state (MPS), the simplest tensor network structure, and residual neural network and propose the residual matrix product state (ResMPS). The ResMPS can be treated as a network where its layers map the "hidden" features to the outputs (e.g., classifications), and the variational parameters of the layers are the functions of the features of the samples (e.g., pixels of images). This is different from neural network, where the layers map feed-forwardly the features to the output. The ResMPS can equip with the non-linear activations and dropout layers, and outperforms the state-of-the-art tensor network models in terms of efficiency, stability, and expression power. Besides, ResMPS is interpretable from the perspective of polynomial expansion, where the factorization and exponential machines naturally emerge. Our work contributes to connecting and hybridizing neural and tensor networks, which is crucial to further enhance our understanding of the working mechanisms and improve the performance of both models.