We present a variational renormalization group (RG) approach based on a reversible generative model with hierarchical architecture. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural network directly maps independent Gaussian noises to physical configurations following the inverse RG flow. The model has an exact and tractable likelihood, which allows unbiased training and direct access to the renormalized energy function of the latent variables. To train the model, we employ probability density distillation for the bare energy function of the physical problem, in which the training loss provides a variational upper bound of the physical free energy. We demonstrate practical usage of the approach by identifying mutually independent collective variables of the Ising model and performing accelerated hybrid Monte Carlo sampling in the latent space. Lastly, we comment on the connection of the present approach to the wavelet formulation of RG and the modern pursuit of information preserving RG.
We present a variational renormalization group (RG) approach based on a reversible generative model with hierarchical architecture. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural network directly maps independent Gaussian noises to physical configurations following the inverse RG flow. The model has an exact and tractable likelihood, which allows unbiased training and direct access to the renormalized energy function of the latent variables. To train the model, we employ probability density distillation for the bare energy function of the physical problem, in which the training loss provides a variational upper bound of the physical free energy. We demonstrate practical usage of the approach by identifying mutually independent collective variables of the Ising model and performing accelerated hybrid Monte Carlo sampling in the latent space. Lastly, we comment on the connection of the present approach to the wavelet formulation of RG and the modern pursuit of information preserving RG.
We propose a general framework for solving statistical mechanics of systems with finite size. The approach extends the celebrated variational mean-field approaches using autoregressive neural networks, which support direct sampling and exact calculation of normalized probability of configurations. It computes variational free energy, estimates physical quantities such as entropy, magnetizations and correlations, and generates uncorrelated samples all at once. Training of the network employs the policy gradient approach in reinforcement learning, which unbiasedly estimates the gradient of variational parameters. We apply our approach to several classic systems, including 2D Ising models, the Hopfield model, the Sherrington-Kirkpatrick model, and the inverse Ising model, for demonstrating its advantages over existing variational mean-field methods. Our approach sheds light on solving statistical physics problems using modern deep generative neural networks.
Even though deep learning has proved to be very powerful as the core method of machine learning, theoretical understanding behind its success is still unclear. It is been pointed out in recent years that the behaviour of deep neural networks is reminiscent of a fundamental framework in statistical physics: the renormalization group (RG). Motivated by this analogy, we develop an analytical method of directly obtaining the trained weights W resulting from a training set of 1D-Ising samples with coupling J and temperature T. The found relation W(J) drives a flow that takes 1D-Ising configurations at non-zero temperature to the critical point at T = 0. This behaviour is opposite to what a typical RG-flow dictates.
The key idea behind the renormalization group (RG) transformation is that properties of physical systems with very different microscopic makeups can be characterized by a few universal parameters.However, finding a systematic way to construct RG transformation for particular systems remains difficult due to the many possible choices of the weight factors in the RG procedure.Here we show, by identifying the conditional distribution in the restricted Boltzmann machine and the weight factor distribution in the RG procedure, that a valid real-space RG transformation can be learned without prior knowledge of the physical system.This neural Monte Carlo RG algorithm allows for direct computation of the RG flow and critical exponents.Our results establish a solid connection between the RG transformation in physics and the deep architecture in machine learning, paving the way for further interdisciplinary research.
The renormalization group (RG) is an essential technique in statistical physics and quantum field theory, which considers scale-invariant properties of physical theories and how these theories' parameters change with scaling. Deep learning is a powerful computational technique that uses multi-layered neural networks to solve a myriad of complicated problems. Previous research suggests the possibility that unsupervised deep learning may be a form of RG flow, by being a layer-by-layer coarse graining of the original data. We examined this connection on a more rigorous basis for the simple example of Kadanoff block renormalization of the 2D nearest-neighbor Ising model, with our deep learning accomplished via Restricted Boltzmann Machines (RBMs). We developed extensive renormalization techniques for the 1D and 2D Ising model to provide a baseline for comparison. For the 1D Ising model, we successfully used Adam optimization on a correlation length loss function to learn the group flow, yielding results consistent with the analytical model for infinite N. For the 2D Ising model, we successfully generated Ising model samples using the Wolff algorithm, and performed the group flow using a quasi-deterministic method, validating these results by calculating the critical exponent ν. We then examined RBM learning of the Ising model layer by layer, finding a blocking structure in the learning that is qualitatively similar to RG. Lastly, we directly compared the weights of each layer from the learning to Ising spin renormalization, but found quantitative inconsistencies for the simple case of nearest-neighbor Ising models.
Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value T_{c}=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards T_{c} and how the RBM learns to extract features of spin configurations.
Neural renormalization-group (RG) methods and machine-learned holographic maps train a network on lattice samples or on a lattice action and read physics off the result: coarse couplings, critical exponents, or an emergent bulk geometry. We ask which of these outputs the training objective and the readout themselves determine. For a bijection trained by a divergence to a factorized target with a free coarse coupling, every admissible coupling attains zero loss through a smooth optimal-transport map that respects every symmetry acting identically on field and latents, so neither more training nor an equivariant architecture selects the coupling. Restricting the class can: in the Gaussian chain a zero-loss linear map that is local below the coarse correlation length has the decimation coupling, the threshold is sharp, and on the square lattice no finite-range linear map reaches zero loss. A Gaussian kernel on a network graph yields a distance bounded above and below by the graph distance whatever the data, and exact wavelet latents of a power-law Gaussian give an isotropic hyperbolic geometry except along chains through the wavelet's jumps, which a free-fermion computation ties to the anisotropy reported for the exact holographic mapping. We then re-examine three published pipelines. A hyperbolicity test of neural-RG holography is passed by kernels built without boundary data. In RGFlow at the reported sizes, general linear maps reach zero loss at a continuum of couplings, whereas in a small, unconverged φ⁴ computation RGFlow's own RealNVP class selects a coupling through its Jacobian structure, close to the decimation value in the Gaussian sector. In inverse RG of the three-dimensional spin glass, with a ten-parameter stand-in upscaler on small lattices, the one clean upscaler that closes under blocking reproduced the forward Monte Carlo RG fixed point of its training pair, non-closing ones drifted, and the lower training loss went to a non-closing upscaler in both training pairs. The sharp results are Gaussian. We close with diagnostics, each with a recipe.