Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2412.17497 · Algorithms

Advantages of Density in Tensor Network Geometries for Gradient-Based Training

Sergi Masot-Llima, Artur García-Sáez

Published Jan 31, 2025• 1 citation
arXivHTMLPDF

Abstract

Tensor networks are a very powerful data structure tool originating from simulations of quantum systems. In recent years, they have seen increased use in machine learning, mostly in trainings with gradient-based techniques, due to their flexibility and performance achieved by exploiting hardware acceleration. As ansatzes, tensor networks can be used with flexible geometries, and it is known that for highly regular ones, their dimensionality has a large impact on performance and representation power. For heterogeneous structures, however, these effects are not completely characterized. In this article, we train tensor networks with different geometries to encode a random quantum state, and see that densely connected structures achieve better infidelities than more sparse structures, with higher success rates and less time. Additionally, we give some general insight on how to improve the memory requirements of these sparse structures and the impact of such improvement on the trainings. Finally, as we use HPC resources for the calculations, we discuss the requirements for this approach and showcase performance improvements with GPU acceleration on a last-generation supercomputer.

DOI
10.3390/a18020070

Related papers

Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learning

Fergus Barratt, James Dborin, Wright, Lewis

Tensor networks have demonstrated significant value for machine learning in a myriad of different applications. However, optimizing tensor networks using standard gradient descent has proven to be difficult in practice. Tensor networks suffer from initialization problems resulting in exploding or vanishing gradients and require extensive hyperparameter tuning. Efforts to overcome these problems usually depend on specific network architectures, or ad hoc prescriptions. In this paper we address the problems of initialization and hyperparameter tuning, making it possible to train tensor networks using established machine learning techniques. We introduce a `copy node' method that successfully initializes arbitrary tensor networks, in addition to a gradient based regularization technique for bond dimensions. We present numerical results that show that the combination of techniques presented here produces quantum inspired tensor network models with far fewer parameters, while improving generalization performance.

arXiv:2203.03366•Mar 3, 2022•16 citations

Tensor Networks: a quantum-information perspective on numerical renormalization groups

Pietro Silvi

Exact many-body quantum problems are known to be computationally hard due to the exponential scaling of the numerical resources required. Since the advent of the Density Matrix Renormalization Group, it became clear that a successful strategy to work around this obstacle was to develop numerical methods based on the well-known theoretical renormalization group. In recent years, it was realized that quantum states engineered via numerical renormalization allow a variational representation in terms of a tensor network picture. The discovery provided a further boost to the effectiveness of these techniques, not only due to the increased flexibility and manipulability, but also because tensor network states embed a direct interface to the entanglement they carry, so that one can directly address many-body quantum correlations within these variational ansatz states. This lead to the application of several numerical tools, originally developed in the field of quantum-information, to approach condensed matter problems.

arXiv:1205.4198•May 18, 2012•4 citations

Classical versus quantum: Comparing tensor-network-based quantum circuits on Large Hadron Collider data

Jack Y. Araz, Michael Spannowsky

Tensor networks (TN) are approximations of high-dimensional tensors designed to represent locally entangled quantum many-body systems efficiently. This paper provides a comprehensive comparison between classical TNs and TN-inspired quantum circuits in the context of machine learning on highly complex, simulated Large Hadron Collider data. We show that classical TNs require exponentially large bond dimensions and higher Hilbert-space mapping to perform comparably to their quantum counterparts. While such an expansion in the dimensionality allows better performance, we observe that, with increased dimensionality, classical TNs lead to a highly flat loss landscape, rendering the usage of gradient-based optimization methods highly challenging. Furthermore, by employing quantitative metrics, such as the Fisher information and effective dimensions, we show that classical TNs require a more extensive training sample to represent the data as efficiently as TN-inspired quantum circuits. We also engage with the idea of hybrid classical-quantum TNs and show possible architectures to employ a larger phase space from the data. We offer our results using three main TN Ans\"atze: Tree tensor networks, matrix product states, and multiscale entanglement renormalization Ans\"atze.

arXiv:2202.10471•Physical Review A•Dec 19, 2022•44 citations

Application of quantum-inspired tensor networks to optimize federated learning systems

Amandeep Singh Bhatia, Mandeep Kaur Saggi, Sabre Kais

Abstract Federated learning (FL) has gained significant traction across diverse industries, which allows multiple clients or institutions to enhance model performance and outcomes while preserving data privacy collaboratively. In recent years, tensor networks (TNs) have become important in machine learning because they allow the compact representation of high-dimensional tensors by decomposing them into lower-dimensional components with polynomial complexity. The application of TNs in FL is a natural extension because of its flexible framework for representing and optimizing models. Inspired by quantum computing principles, we have integrated a quantum-inspired tensor network into the FL framework. This framework focuses on a one-dimensional matrix product state (MPS) tensor network (TN) in a federated setting (FedTN), with data distributed across homogeneous and heterogeneous partitions among clients. Our experiments demonstrate that tensor network-based federated learning can be made practical, as FedTN is robust to the unbalanced and non-IID data distributions typically encountered in such settings. Our research assessed the effectiveness and feasibility of comparing quantum-inspired TN and conventional methods, evaluating their performance, and exploring the benefits of incorporating quantum principles in FL settings. Furthermore, we have investigated its performance when training for many local epochs (large E) between the averaging steps.

Quantum Machine Intelligence•Jan 28, 2025•15 citations

Entangling Solid Solutions: Machine Learning of Tensor Networks for Materials Property Prediction

David E. Sommer, Scott T. Dunham

Progress in the application of machine learning techniques to the prediction of solid-state and molecular materials properties has been greatly facilitated by the development state-of-the-art feature representations and novel deep learning architectures. A large class of atomic structure representations based on expansions of smoothed atomic densities have been shown to correspond to specific choices of basis sets in an abstract many-body Hilbert space. Concurrently, tensor network structures, conventionally the purview of quantum many-body physics and quantum information, have been successfully applied in supervised and unsupervised learning tasks in computer vision and natural language processing. In this work, we argue that architectures based on tensor networks are well-suited to machine learning on Hilbert-space representations of atomic structures. This is demonstrated on supervised learning tasks involving widely available datasets of density functional theory calculations of metal and semiconductor alloys. In particular, we show that certain standard tensor network topologies exhibit strong generalizability even on small training datasets while being parametrically efficient. We further relate this generalizability to the presence of complex entanglement in the trained tensor networks. We also discuss connections to learning with generalized structural kernels and related strategies for compressing large input feature spaces.

arXiv:2203.09613•Mar 17, 2022•4 citations

On scalable quantum-inspired distributed machine learning

Jean-Michel Sellier, Hardik Dalal

Recently a new approach to train machine learning models, such as neural networks, has been introduced. This method is based on the simulation of a type of quantum systems, in particular systems of one-dimensional electrons coupled by a force, which offers important advantages. First, by exploiting simulations of the quantum tunnelling effect, a different, and faster, convergence in the training phase can be obtained. Secondly, it does not require the computation of any kind of gradient (in other words, it can be considered a derivative-free or gradient-free optimizer) thus allowing the training of complex models impossible to train otherwise, for instance with the gradient descent method, or variations of it. Finally, it allows the use of parallelization schemes which can scale linearly with the dimensions of problem at hand, i.e., the method is nearly "embarrassingly parallelizable". This opens the way towards scalable and performant distributed machine learning capabilities. The goal of this paper is to focus specifically on this very advantage, suggest various strategies for parallelization, and discuss their consequences.

•Feb 8, 2024•0 citations
Tensor networks are a very powerful data structure tool originating from simulations of quantum systems. In recent years, they have seen increased use in machine learning, mostly in trainings with gradient-based techniques, due to their flexibility and performance achieved by exploiting hardware acceleration. As ansatzes, tensor networks can be used with flexible geometries, and it is known that for highly regular ones, their dimensionality has a large impact on performance and representation power. For heterogeneous structures, however, these effects are not completely characterized. In this article, we train tensor networks with different geometries to encode a random quantum state, and see that densely connected structures achieve better infidelities than more sparse structures, with higher success rates and less time. Additionally, we give some general insight on how to improve the memory requirements of these sparse structures and the impact of such improvement on the trainings. Finally, as we use HPC resources for the calculations, we discuss the requirements for this approach and showcase performance improvements with GPU acceleration on a last-generation supercomputer.
Tensor networks have demonstrated significant value for machine learning in a myriad of different applications. However, optimizing tensor networks using standard gradient descent has proven to be difficult in practice. Tensor networks suffer from initialization problems resulting in exploding or vanishing gradients and require extensive hyperparameter tuning. Efforts to overcome these problems usually depend on specific network architectures, or ad hoc prescriptions. In this paper we address the problems of initialization and hyperparameter tuning, making it possible to train tensor networks using established machine learning techniques. We introduce a `copy node' method that successfully initializes arbitrary tensor networks, in addition to a gradient based regularization technique for bond dimensions. We present numerical results that show that the combination of techniques presented here produces quantum inspired tensor network models with far fewer parameters, while improving generalization performance.
Exact many-body quantum problems are known to be computationally hard due to the exponential scaling of the numerical resources required. Since the advent of the Density Matrix Renormalization Group, it became clear that a successful strategy to work around this obstacle was to develop numerical methods based on the well-known theoretical renormalization group. In recent years, it was realized that quantum states engineered via numerical renormalization allow a variational representation in terms of a tensor network picture. The discovery provided a further boost to the effectiveness of these techniques, not only due to the increased flexibility and manipulability, but also because tensor network states embed a direct interface to the entanglement they carry, so that one can directly address many-body quantum correlations within these variational ansatz states. This lead to the application of several numerical tools, originally developed in the field of quantum-information, to approach condensed matter problems.
Tensor networks (TN) are approximations of high-dimensional tensors designed to represent locally entangled quantum many-body systems efficiently. This paper provides a comprehensive comparison between classical TNs and TN-inspired quantum circuits in the context of machine learning on highly complex, simulated Large Hadron Collider data. We show that classical TNs require exponentially large bond dimensions and higher Hilbert-space mapping to perform comparably to their quantum counterparts. While such an expansion in the dimensionality allows better performance, we observe that, with increased dimensionality, classical TNs lead to a highly flat loss landscape, rendering the usage of gradient-based optimization methods highly challenging. Furthermore, by employing quantitative metrics, such as the Fisher information and effective dimensions, we show that classical TNs require a more extensive training sample to represent the data as efficiently as TN-inspired quantum circuits. We also engage with the idea of hybrid classical-quantum TNs and show possible architectures to employ a larger phase space from the data. We offer our results using three main TN Ans\"atze: Tree tensor networks, matrix product states, and multiscale entanglement renormalization Ans\"atze.
Abstract Federated learning (FL) has gained significant traction across diverse industries, which allows multiple clients or institutions to enhance model performance and outcomes while preserving data privacy collaboratively. In recent years, tensor networks (TNs) have become important in machine learning because they allow the compact representation of high-dimensional tensors by decomposing them into lower-dimensional components with polynomial complexity. The application of TNs in FL is a natural extension because of its flexible framework for representing and optimizing models. Inspired by quantum computing principles, we have integrated a quantum-inspired tensor network into the FL framework. This framework focuses on a one-dimensional matrix product state (MPS) tensor network (TN) in a federated setting (FedTN), with data distributed across homogeneous and heterogeneous partitions among clients. Our experiments demonstrate that tensor network-based federated learning can be made practical, as FedTN is robust to the unbalanced and non-IID data distributions typically encountered in such settings. Our research assessed the effectiveness and feasibility of comparing quantum-inspired TN and conventional methods, evaluating their performance, and exploring the benefits of incorporating quantum principles in FL settings. Furthermore, we have investigated its performance when training for many local epochs (large E) between the averaging steps.
Progress in the application of machine learning techniques to the prediction of solid-state and molecular materials properties has been greatly facilitated by the development state-of-the-art feature representations and novel deep learning architectures. A large class of atomic structure representations based on expansions of smoothed atomic densities have been shown to correspond to specific choices of basis sets in an abstract many-body Hilbert space. Concurrently, tensor network structures, conventionally the purview of quantum many-body physics and quantum information, have been successfully applied in supervised and unsupervised learning tasks in computer vision and natural language processing. In this work, we argue that architectures based on tensor networks are well-suited to machine learning on Hilbert-space representations of atomic structures. This is demonstrated on supervised learning tasks involving widely available datasets of density functional theory calculations of metal and semiconductor alloys. In particular, we show that certain standard tensor network topologies exhibit strong generalizability even on small training datasets while being parametrically efficient. We further relate this generalizability to the presence of complex entanglement in the trained tensor networks. We also discuss connections to learning with generalized structural kernels and related strategies for compressing large input feature spaces.
Recently a new approach to train machine learning models, such as neural networks, has been introduced. This method is based on the simulation of a type of quantum systems, in particular systems of one-dimensional electrons coupled by a force, which offers important advantages. First, by exploiting simulations of the quantum tunnelling effect, a different, and faster, convergence in the training phase can be obtained. Secondly, it does not require the computation of any kind of gradient (in other words, it can be considered a derivative-free or gradient-free optimizer) thus allowing the training of complex models impossible to train otherwise, for instance with the gradient descent method, or variations of it. Finally, it allows the use of parallelization schemes which can scale linearly with the dimensions of problem at hand, i.e., the method is nearly "embarrassingly parallelizable". This opens the way towards scalable and performant distributed machine learning capabilities. The goal of this paper is to focus specifically on this very advantage, suggest various strategies for parallelization, and discuss their consequences.