Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2212.11811

Renormalization in the neural network-quantum field theory correspondence

Harold Erbin, Vincent Lahoche, Dine Ousmane Samary

Published Dec 22, 2022• 15 citations
arXivHTMLPDF

Abstract

A statistical ensemble of neural networks can be described in terms of a quantum field theory (NN-QFT correspondence). The infinite-width limit is mapped to a free field theory, while finite N corrections are mapped to interactions. After reviewing the correspondence, we will describe how to implement renormalization in this context and discuss preliminary numerical results for translation-invariant kernels. A major outcome is that changing the standard deviation of the neural network weight distribution corresponds to a renormalization flow in the space of networks.

DOI
10.48550/arxiv.2212.11811

Related papers

Neural Network Field Theory at Finite Width

Christian Ferko, Aaron Mutchler

Under mild assumptions, any quantum mechanical (QM) model or quantum field theory (QFT) admits a representation in terms of an ensemble of neural networks with countably many random parameters. We investigate the features of NN-QM and NN-FT models with finitely many parameters, such as a feedforward network of width $N < \infty$. We find that, generically, such models must violate one of the properties of conventional Euclidean QFTs, such as reflection positivity or cluster decomposition. We present several complementary ways of understanding which features can and cannot be preserved at finite $N$, both in QM and in QFT.

arXiv:2608.21588•Aug 21, 2026•1 citation

What Neural Network Field Theory Can and Cannot Realise on a Computer

Thomas R. Harvey

One aim of neural network field theory is to put a quantum or effective field theory on a computer, with the network ensemble itself as the theory. We ask how far that aim can be pushed for a function class regular enough to be computed with. Our main result is a no-go theorem with assumptions that hold for standard network architectures. We use it to separate four versions of neural network field theory, according to whether the defining object is the finite width ensemble or its infinite width limit, and whether the target we want to compute is a quantum or an effective field theory. Neither finite width interpretation is straightforwardly consistent. For finite width ensembles with finite variance at each point, the QFT interpretation fails reflection positivity, while the EFT interpretation establishes no scale separation by which the positivity violation can be placed outside its domain of validity. Of the two limit versions, one can be simulated in full and the other only in part, as only its smeared correlators are computable with a controlled error. As such, at the level of a controlled numerical computation, the QFT and EFT versions cannot be distinguished. One dimension escapes the obstruction, yet reflection positivity is shown to still fail there at every finite width for the cosine network. Two escapes from the theorem remain, giving up either finite variance at a point or exact rotation invariance, and we discuss both of these possibilities.

arXiv:2608.21523•Aug 21, 2026•2 citations

Criticality in Neural Network Function Space through Wilsonian Fixed Points and Finite-Width Corrections

Eric Howard, Iftekher S. Chowdhury, Hardique Dasore +1 more

Neural networks can be studied not only as parameterized computational models but also as probability distributions over functions. In this paper we develop a Wilsonian interpretation of criticality in the neural network-quantum field theory correspondence, treating the infinite width Gaussian-process limit as a free field fixed point and finite width corrections as perturbations that create non-Gaussian interactions. In this way, we do not see the departure from infinite width as a small approximation error but as the process by which interaction, complexity, expressivity and phase-like behaviour enter neural network function space. Width, depth, activation nonlinearity, initialization variance, and training dynamics are considered as control parameters that change the effective action of the network ensemble. The critical regime is when the higher-order connected correlation functions become non-negligible, and when the finite width operators become relevant or marginal scaling factors, and when the function distribution becomes sensitive to scale-dependent structure. There we can view overparameterization as suppressing a relationship of interacting terms and we find that the critical structure of finite neural networks is given by finite-width effects. This framework is a theoretical basis for studying trainability, generalization, and architectural universality of neural network functions through Wilsonian fixed points, perturbations, and critical surfaces.

cond-mat.dis-nnCross-listedarXiv:2608.20403•Aug 5, 2026

The edge of chaos: quantum field theory and deep neural networks

Kevin T. Grosvenor, Ro Jefferson

We explicitly construct the quantum field theory corresponding to a general class of deep neural networks encompassing both recurrent and feedforward architectures. We first consider the mean-field theory (MFT) obtained as the leading saddlepoint in the action, and derive the condition for criticality via the largest Lyapunov exponent. We then compute the loop corrections to the correlation function in a perturbative expansion in the ratio of depth T to width N, and find a precise analogy with the well-studied O(N) vector model, in which the variance of the weight initializations plays the role of the 't Hooft coupling. In particular, we compute both the O(1) corrections quantifying fluctuations from typicality in the ensemble of networks, and the subleading O(T/N) corrections due to finite-width effects. These provide corrections to the correlation length that controls the depth to which information can propagate through the network, and thereby sets the scale at which such networks are trainable by gradient descent. Our analysis provides a first-principles approach to the rapidly emerging NN-QFT correspondence, and opens several interesting avenues to the study of criticality in deep neural networks.

arXiv:2109.13247•SciPost Physics•Mar 3, 2022•53 citations

Neural networks and quantum field theory

James Halverson, Anindita Maiti, Keegan Stoner

Abstract We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from ’t Hooft’s technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.

arXiv:2008.08601•Machine Learning Science and Technology•Mar 8, 2021•131 citations

Perturbative dynamic renormalization of scalar field theories in statistical physics

Papanikolaou, Nikos, Speck, Thomas

Renormalization is a powerful technique in statistical physics to extract the large-scale behavior of interacting many-body models. These notes aim to give an introduction to perturbative methods that operate on the level of the stochastic evolution equation for a scalar field (e.g., density), including systems that are driven away from equilibrium and thus lack a free energy. While there is a large number of reviews and lecture notes, many are somewhat scarce on technical details and written in the language of quantum field theory, which can be more confusing than helpful. Here we attempt a minimal and concise yet pedagogical introduction to dynamic renormalization in the language of statistical physics with a strong focus on how to actually perform calculations. We provide a symbolic algebra implementation of the discussed techniques including Jupyter notebooks of two illustrations: the KPZ equation and a neural network model.

arXiv:2303.02222•Mar 3, 2023•1 citation
A statistical ensemble of neural networks can be described in terms of a quantum field theory (NN-QFT correspondence). The infinite-width limit is mapped to a free field theory, while finite N corrections are mapped to interactions. After reviewing the correspondence, we will describe how to implement renormalization in this context and discuss preliminary numerical results for translation-invariant kernels. A major outcome is that changing the standard deviation of the neural network weight distribution corresponds to a renormalization flow in the space of networks.
Under mild assumptions, any quantum mechanical (QM) model or quantum field theory (QFT) admits a representation in terms of an ensemble of neural networks with countably many random parameters. We investigate the features of NN-QM and NN-FT models with finitely many parameters, such as a feedforward network of width N<∞N < \inftyN<∞. We find that, generically, such models must violate one of the properties of conventional Euclidean QFTs, such as reflection positivity or cluster decomposition. We present several complementary ways of understanding which features can and cannot be preserved at finite NNN, both in QM and in QFT.
One aim of neural network field theory is to put a quantum or effective field theory on a computer, with the network ensemble itself as the theory. We ask how far that aim can be pushed for a function class regular enough to be computed with. Our main result is a no-go theorem with assumptions that hold for standard network architectures. We use it to separate four versions of neural network field theory, according to whether the defining object is the finite width ensemble or its infinite width limit, and whether the target we want to compute is a quantum or an effective field theory. Neither finite width interpretation is straightforwardly consistent. For finite width ensembles with finite variance at each point, the QFT interpretation fails reflection positivity, while the EFT interpretation establishes no scale separation by which the positivity violation can be placed outside its domain of validity. Of the two limit versions, one can be simulated in full and the other only in part, as only its smeared correlators are computable with a controlled error. As such, at the level of a controlled numerical computation, the QFT and EFT versions cannot be distinguished. One dimension escapes the obstruction, yet reflection positivity is shown to still fail there at every finite width for the cosine network. Two escapes from the theorem remain, giving up either finite variance at a point or exact rotation invariance, and we discuss both of these possibilities.
Neural networks can be studied not only as parameterized computational models but also as probability distributions over functions. In this paper we develop a Wilsonian interpretation of criticality in the neural network-quantum field theory correspondence, treating the infinite width Gaussian-process limit as a free field fixed point and finite width corrections as perturbations that create non-Gaussian interactions. In this way, we do not see the departure from infinite width as a small approximation error but as the process by which interaction, complexity, expressivity and phase-like behaviour enter neural network function space. Width, depth, activation nonlinearity, initialization variance, and training dynamics are considered as control parameters that change the effective action of the network ensemble. The critical regime is when the higher-order connected correlation functions become non-negligible, and when the finite width operators become relevant or marginal scaling factors, and when the function distribution becomes sensitive to scale-dependent structure. There we can view overparameterization as suppressing a relationship of interacting terms and we find that the critical structure of finite neural networks is given by finite-width effects. This framework is a theoretical basis for studying trainability, generalization, and architectural universality of neural network functions through Wilsonian fixed points, perturbations, and critical surfaces.
We explicitly construct the quantum field theory corresponding to a general class of deep neural networks encompassing both recurrent and feedforward architectures. We first consider the mean-field theory (MFT) obtained as the leading saddlepoint in the action, and derive the condition for criticality via the largest Lyapunov exponent. We then compute the loop corrections to the correlation function in a perturbative expansion in the ratio of depth T to width N, and find a precise analogy with the well-studied O(N) vector model, in which the variance of the weight initializations plays the role of the 't Hooft coupling. In particular, we compute both the O(1) corrections quantifying fluctuations from typicality in the ensemble of networks, and the subleading O(T/N) corrections due to finite-width effects. These provide corrections to the correlation length that controls the depth to which information can propagate through the network, and thereby sets the scale at which such networks are trainable by gradient descent. Our analysis provides a first-principles approach to the rapidly emerging NN-QFT correspondence, and opens several interesting avenues to the study of criticality in deep neural networks.
Abstract We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from ’t Hooft’s technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.
Renormalization is a powerful technique in statistical physics to extract the large-scale behavior of interacting many-body models. These notes aim to give an introduction to perturbative methods that operate on the level of the stochastic evolution equation for a scalar field (e.g., density), including systems that are driven away from equilibrium and thus lack a free energy. While there is a large number of reviews and lecture notes, many are somewhat scarce on technical details and written in the language of quantum field theory, which can be more confusing than helpful. Here we attempt a minimal and concise yet pedagogical introduction to dynamic renormalization in the language of statistical physics with a strong focus on how to actually perform calculations. We provide a symbolic algebra implementation of the discussed techniques including Jupyter notebooks of two illustrations: the KPZ equation and a neural network model.