Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2611.03859

Factorized approximate inverse preconditioners in the tensor train format

Pau Torrente Badia, Daniel Pérez García, Artur Garcia-Saez, Stefano Carignano

math.NAquant-phSubmitted Nov 2, 2026• Announced Nov 4, 2026• 0 citations
arXivHTMLPDF

Abstract

We present a preconditioning pipeline that accelerates Tensor Train (TT) linear solvers for symmetric positive-definite problems. Extending the Factorized Sparse Approximate Inverse framework to tensor networks, we construct a preconditioner as the product of a TT operator and its transpose. This formulation is positive semidefinite for any bond dimension and truncation, and definite whenever the factor is non-singular, satisfying the mathematical requirements of the Conjugate Gradient method. We detail the construction of this preconditioner and its integration into Density Matrix Renormalization Group (DMRG)-like linear solvers. Performance evaluations show consistent iteration reductions for anisotropic diffusion and Poisson problems across various configurations, leading to net speedups of up to 25% in ill-conditioned settings of time stepping schemes that can reuse the approximate inverses. These benchmarks were performed with the open-source Julia library DifferenTT.jl, which we also introduce in this work to provide out-of-the-box access to these preconditioned pipelines.

AI Summary

Tensor-train (TT) approximate inverses can lose positive definiteness under truncation, making them unsuitable for preconditioned conjugate gradient (PCG). This paper adapts factorized sparse approximate inversion to construct a TT preconditioner $G^T G$, which remains positive semidefinite regardless of factor truncation and is positive definite when $G$ is nonsingular. The construction uses a bond-dimension-two triangularity mask to formulate a projected linear system for an approximate inverse Cholesky factor without computing the Cholesky decomposition. Diagonal rescaling uses recursive sketched interpolation. For DMRG-like MALS solvers, projecting the factor and applying its transpose locally avoids forming or truncating the full product. A proved bound controls diagonal-rescaling errors: $|\tilde G_{ii}/\tilde G^\star_{ii}-1|\leq\nu_i$, where $\nu_i$ is the relative row energy error; a nonpositive diagonal requires $\nu_i\geq1$. Benchmarks cover rotated anisotropic diffusion in 10 and 20 dimensions and repeated pressure-Poisson solves in a QTT fluid simulation. For anisotropic diffusion, iteration reductions relative to a separable preconditioner reach 50% and 30%, respectively, but application overhead and construction costs negate runtime gains. On a $1024^2$ fluid grid, factor bond dimensions 4–7 reduce iterations by factors of 4.0–5.6 and give reported net speedups of approximately 20–25% over 1,000 steps, including construction; the $256^2$ case reaches 13%. Practical benefits depend on small local dimensions and preconditioner reuse. Diagonal positivity is not guaranteed by the solver’s residual tolerance, projected factors can be singular, and performance on less structured operators, nonuniform grids, or strongly varying coefficients remains untested. Code and benchmark data are provided.

Little sign of AI-generated textAI use disclosedIncrementalMissing comparison
the authors used Claude (Anthropic) and Gemini (Google) LLMs to assist in the formalization of Lemma~\ref{lem:diag_positivity} and the rotated anisotropy benchmark of Sec.~\ref{subsec:anisotropy_experiment}

Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.

Comments
50 pages, 11 figures

Related papers

Solution of Linear Systems and Matrix Inversion in the TT-Format

Ivan Valer'evich Oseledets, Sergey Dolgov

Tensors arise naturally in high-dimensional problems in chemistry, financial mathematics, and many other areas. The numerical treatment of such problems is difficult due to the curse of dimensionality: the number of unknowns and the computational complexity grow exponentially with the dimension of the problem. To break the curse of dimensionality, low-parametric representations, or formats, have to be used. In this paper we make use of the TT-format (tensor-train format) which is one of the most effective stable representations of high-dimensional tensors. Basic linear algebra operations in the TT-format are now well developed. Our goal is to provide a “black-box” type of solver for linear systems where both the matrix and the right-hand side are in the TT-format. An efficient DMRG (density matrix renormalization group) method is proposed, and several tricks are employed to make it work. The numerical experiments confirm the effectiveness of our approach.

SIAM Journal on Scientific Computing•Jan 1, 2012•186 citations

Accelerating two-dimensional tensor network optimization by preconditioning

Xing-Yu Zhang, Qi Yang, Philippe Corboz +2 more

We revisit gradient-based optimization for infinite projected entangled pair states (iPEPS), a tensor network ansatz for simulating many-body quantum systems. This approach is hindered by two major challenges: the high computational cost of evaluating energies and gradients, and an ill-conditioned optimization landscape that slows convergence. To reduce the number of optimization steps, we introduce an efficient preconditioner derived from the leading term of the metric tensor. We benchmark our method against standard optimization techniques on the Heisenberg and Kitaev models, demonstrating substantial improvements in overall computational efficiency. Our approach is broadly applicable across various contraction schemes, unit cell sizes, and Hamiltonians, highlighting the potential of preconditioned optimization to advance tensor network algorithms for strongly correlated systems.

arXiv:2511.09546•Physical review. B./Physical review. B•Feb 13, 2026•7 citations

Efficient Finite Initialization with Partial Norms for Tensorized Neural Networks and Tensor Networks Algorithms

Alejandro Mata Ali, Iñigo Perez Delgado, Marina Ristol Roura +1 more

We present two algorithms to initialize layers of tensorized neural networks and general tensor network algorithms using partial computations of their Frobenius norms and positive lineal entrywise sums, depending on the type of tensor network involved. The core of this method is the use of the norm of subnetworks of the tensor network in an iterative way, so that we normalize by the finite values of the norms that led to the divergence or zero norm. In addition, the method benefits from the reuse of intermediate calculations. We have also applied it to the Matrix Product State/Tensor Train (MPS/TT) and Matrix Product Operator/Tensor Train Matrix (MPO/TT-M) layers and have seen its scaling versus the number of nodes, bond dimension, and physical dimension. All code is publicly available.

arXiv:2309.06577•Sep 11, 2023•4 citations

Time Integration of Tensor Trains

Christian Lubich, Ivan V. Oseledets, Bart Vandereycken

A robust and efficient time integrator for dynamical tensor approximation in the tensor train or matrix product state format is presented. The method is based on splitting the projector onto the tangent space of the tensor manifold. The algorithm can be used for updating time-dependent tensors in the given data-sparse tensor train/matrix product state format and for computing an approximate solution to high-dimensional tensor differential equations within this data-sparse format. The formulation, implementation, and theoretical properties of the proposed integrator are studied, and numerical experiments with problems from quantum molecular dynamics and with iterative processes in the tensor train format are included.

arXiv:1407.2042•SIAM Journal on Numerical Analysis•Jan 1, 2015•215 citations

A Simple GPU-Accelerated Solver for the Schrödinger Operator with Applications to Ground States and Hamiltonian Simulation

Xinyu Liu, Xiangxiong Zhang

We extend the tensor-product direct solver from the Laplacian to the Schrödinger operator $-Δ+ V$. When the potential $V_1$ is separable, the operator $-Δ+ V_1$ is inverted or exponentiated at cost $O(N^{1+1/d})$ in $d$ dimensions via per-axis eigendecomposition. On a single NVIDIA A100 GPU, this costs less than one second for $10^9$ degrees of freedom in 3D. For non-separable potentials $V = V_1 + V_2$, the same solver provides a preconditioner $(-Δ+ V_1)^{-1}$ for the preconditioned conjugate gradient (PCG) method and a propagator for operator-splitting time integrators. For bounded $V_2$, we prove that the preconditioned operator has a bounded condition number and a clustered spectrum with at most finitely many outlier eigenvalues, independently of the mesh size, and also independently of the domain size when $V_1$ is a confining potential. This explains the mesh- and domain-independent PCG iteration counts observed in practice. We apply this method to ground state computation via inverse iteration for linear problems and via the $a_u$ gradient flow for Gross--Pitaevskii energy in 3D, and also Hamiltonian simulation via the approximated qHOP and Magnus-2 splitting methods from 3D to 9D on a single NVIDIA GH200 GPU.

math.NACross-listedarXiv:2605.20491•May 21, 2026

TT-net: Quantum Inspired Tensor Network Denoising in Conditional GANs

Michal A. Sterzel, Marko J. Rančić

Developed as a workhorse for classical simulations of quantum algorithms and quantum many-body systems, Tensor Network methods have entered the scientific mainstream in quantum physics. Among various types of tensor networks, Tensor Trains (commonly know as Matrix Product States in the quantum computing community) have already found applications in machine learning. These methods often rely on a powerful linear algebra tool called the Singular Value Decomposition (SVD). Several conditional GAN architectures for image denoising incorporate SVD as a single-cut decomposition step applied to generator feature maps. In this work we introduce TT-Net, which replaces the per-channel SVD denoising block with a two-cut tensor-train decomposition capable of accessing cross-channel information directly, a capability absent from contemporary alternatives. In a controlled comparison differing only in this decomposition mechanism, TT-Net outperforms SVD-Net on PSNR and SSIM across all three noise types tested (Gaussian, motion blur, and salt-and-pepper), supporting the hypothesis that cross-channel access improves denoising quality. Training-dynamics analysis further shows that TT-Net's adversarial loss term consistently saturates to a stagnant state across all three noise types, more so than SVD-Net's, while reconstruction quality continues to improve regardless, raising an open question about the adversarial component's contribution that this work identifies but does not resolve. Furthermore, for Gaussian noise our method outperforms both the EigenGAN and the state of the art Pix2pix method which does not assume any linear algebra decompositions and does not retain any linear algebra information. Our manuscript shows how quantum inspired tools can be used as practical real world feature filters for deep learning applications.

cs.AICross-listedarXiv:2608.19789•Aug 21, 2026
We present a preconditioning pipeline that accelerates Tensor Train (TT) linear solvers for symmetric positive-definite problems. Extending the Factorized Sparse Approximate Inverse framework to tensor networks, we construct a preconditioner as the product of a TT operator and its transpose. This formulation is positive semidefinite for any bond dimension and truncation, and definite whenever the factor is non-singular, satisfying the mathematical requirements of the Conjugate Gradient method. We detail the construction of this preconditioner and its integration into Density Matrix Renormalization Group (DMRG)-like linear solvers. Performance evaluations show consistent iteration reductions for anisotropic diffusion and Poisson problems across various configurations, leading to net speedups of up to 25% in ill-conditioned settings of time stepping schemes that can reuse the approximate inverses. These benchmarks were performed with the open-source Julia library DifferenTT.jl, which we also introduce in this work to provide out-of-the-box access to these preconditioned pipelines.
Tensor-train (TT) approximate inverses can lose positive definiteness under truncation, making them unsuitable for preconditioned conjugate gradient (PCG). This paper adapts factorized sparse approximate inversion to construct a TT preconditioner GTGG^T GGTG, which remains positive semidefinite regardless of factor truncation and is positive definite when GGG is nonsingular.
The construction uses a bond-dimension-two triangularity mask to formulate a projected linear system for an approximate inverse Cholesky factor without computing the Cholesky decomposition. Diagonal rescaling uses recursive sketched interpolation. For DMRG-like MALS solvers, projecting the factor and applying its transpose locally avoids forming or truncating the full product. A proved bound controls diagonal-rescaling errors: ∣G~ii/G~ii⋆−1∣≤νi|\tilde G_{ii}/\tilde G^\star_{ii}-1|\leq\nu_i∣G~ii​/G~ii⋆​−1∣≤νi​, where νi\nu_iνi​ is the relative row energy error; a nonpositive diagonal requires νi≥1\nu_i\geq1νi​≥1.
Benchmarks cover rotated anisotropic diffusion in 10 and 20 dimensions and repeated pressure-Poisson solves in a QTT fluid simulation. For anisotropic diffusion, iteration reductions relative to a separable preconditioner reach 50% and 30%, respectively, but application overhead and construction costs negate runtime gains. On a 102421024^210242 fluid grid, factor bond dimensions 4–7 reduce iterations by factors of 4.0–5.6 and give reported net speedups of approximately 20–25% over 1,000 steps, including construction; the 2562256^22562 case reaches 13%.
Practical benefits depend on small local dimensions and preconditioner reuse. Diagonal positivity is not guaranteed by the solver’s residual tolerance, projected factors can be singular, and performance on less structured operators, nonuniform grids, or strongly varying coefficients remains untested. Code and benchmark data are provided.
Tensors arise naturally in high-dimensional problems in chemistry, financial mathematics, and many other areas. The numerical treatment of such problems is difficult due to the curse of dimensionality: the number of unknowns and the computational complexity grow exponentially with the dimension of the problem. To break the curse of dimensionality, low-parametric representations, or formats, have to be used. In this paper we make use of the TT-format (tensor-train format) which is one of the most effective stable representations of high-dimensional tensors. Basic linear algebra operations in the TT-format are now well developed. Our goal is to provide a “black-box” type of solver for linear systems where both the matrix and the right-hand side are in the TT-format. An efficient DMRG (density matrix renormalization group) method is proposed, and several tricks are employed to make it work. The numerical experiments confirm the effectiveness of our approach.
We revisit gradient-based optimization for infinite projected entangled pair states (iPEPS), a tensor network ansatz for simulating many-body quantum systems. This approach is hindered by two major challenges: the high computational cost of evaluating energies and gradients, and an ill-conditioned optimization landscape that slows convergence. To reduce the number of optimization steps, we introduce an efficient preconditioner derived from the leading term of the metric tensor. We benchmark our method against standard optimization techniques on the Heisenberg and Kitaev models, demonstrating substantial improvements in overall computational efficiency. Our approach is broadly applicable across various contraction schemes, unit cell sizes, and Hamiltonians, highlighting the potential of preconditioned optimization to advance tensor network algorithms for strongly correlated systems.
We present two algorithms to initialize layers of tensorized neural networks and general tensor network algorithms using partial computations of their Frobenius norms and positive lineal entrywise sums, depending on the type of tensor network involved. The core of this method is the use of the norm of subnetworks of the tensor network in an iterative way, so that we normalize by the finite values of the norms that led to the divergence or zero norm. In addition, the method benefits from the reuse of intermediate calculations. We have also applied it to the Matrix Product State/Tensor Train (MPS/TT) and Matrix Product Operator/Tensor Train Matrix (MPO/TT-M) layers and have seen its scaling versus the number of nodes, bond dimension, and physical dimension. All code is publicly available.
A robust and efficient time integrator for dynamical tensor approximation in the tensor train or matrix product state format is presented. The method is based on splitting the projector onto the tangent space of the tensor manifold. The algorithm can be used for updating time-dependent tensors in the given data-sparse tensor train/matrix product state format and for computing an approximate solution to high-dimensional tensor differential equations within this data-sparse format. The formulation, implementation, and theoretical properties of the proposed integrator are studied, and numerical experiments with problems from quantum molecular dynamics and with iterative processes in the tensor train format are included.
We extend the tensor-product direct solver from the Laplacian to the Schrödinger operator −Δ+V-Δ+ V−Δ+V. When the potential V1V_1V1​ is separable, the operator −Δ+V1-Δ+ V_1−Δ+V1​ is inverted or exponentiated at cost O(N1+1/d)O(N^{1+1/d})O(N1+1/d) in ddd dimensions via per-axis eigendecomposition. On a single NVIDIA A100 GPU, this costs less than one second for 10910^9109 degrees of freedom in 3D. For non-separable potentials V=V1+V2V = V_1 + V_2V=V1​+V2​, the same solver provides a preconditioner (−Δ+V1)−1(-Δ+ V_1)^{-1}(−Δ+V1​)−1 for the preconditioned conjugate gradient (PCG) method and a propagator for operator-splitting time integrators. For bounded V2V_2V2​, we prove that the preconditioned operator has a bounded condition number and a clustered spectrum with at most finitely many outlier eigenvalues, independently of the mesh size, and also independently of the domain size when V1V_1V1​ is a confining potential. This explains the mesh- and domain-independent PCG iteration counts observed in practice. We apply this method to ground state computation via inverse iteration for linear problems and via the aua_uau​ gradient flow for Gross--Pitaevskii energy in 3D, and also Hamiltonian simulation via the approximated qHOP and Magnus-2 splitting methods from 3D to 9D on a single NVIDIA GH200 GPU.
Developed as a workhorse for classical simulations of quantum algorithms and quantum many-body systems, Tensor Network methods have entered the scientific mainstream in quantum physics. Among various types of tensor networks, Tensor Trains (commonly know as Matrix Product States in the quantum computing community) have already found applications in machine learning. These methods often rely on a powerful linear algebra tool called the Singular Value Decomposition (SVD). Several conditional GAN architectures for image denoising incorporate SVD as a single-cut decomposition step applied to generator feature maps. In this work we introduce TT-Net, which replaces the per-channel SVD denoising block with a two-cut tensor-train decomposition capable of accessing cross-channel information directly, a capability absent from contemporary alternatives. In a controlled comparison differing only in this decomposition mechanism, TT-Net outperforms SVD-Net on PSNR and SSIM across all three noise types tested (Gaussian, motion blur, and salt-and-pepper), supporting the hypothesis that cross-channel access improves denoising quality. Training-dynamics analysis further shows that TT-Net's adversarial loss term consistently saturates to a stagnant state across all three noise types, more so than SVD-Net's, while reconstruction quality continues to improve regardless, raising an open question about the adversarial component's contribution that this work identifies but does not resolve. Furthermore, for Gaussian noise our method outperforms both the EigenGAN and the state of the art Pix2pix method which does not assume any linear algebra decompositions and does not retain any linear algebra information. Our manuscript shows how quantum inspired tools can be used as practical real world feature filters for deep learning applications.