Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2409.20377 · Physical review. B./Physical review. B

Learning thermodynamics and topological order of the two-dimensional XY model with generative real-valued restricted Boltzmann machines

Kai Zhang

Published Jan 23, 2025• 5 citations
arXivHTMLPDF

Abstract

Detecting the topological Kosterlitz-Thouless (KT) transition in the prototypical two-dimensional (2D) XY model using unsupervised machine learning methods has long been a challenging problem due to the lack of suitable order parameters. To address this issue, we begin with a conventional real-valued restricted Boltzmann machine (RBM) (RBM-xy), which uses exponential conditional probabilities to generate visible units. We then develop a novel real-valued RBM (RBM-CosSin) featuring nonlinear $cos/sin$ activation, whose visible units follow the von Mises distribution. Our findings reveal that RBM-CosSin effectively learns the underlying Boltzmann distribution of 2D XY systems and generates authentic XY configurations that accurately capture both thermodynamics and topological order (vortex). Furthermore, we demonstrate that it is possible to extract phase transition information, including the KT transition, from the weight matrices without relying on prior physics knowledge.

DOI
10.1103/physrevb.111.024112

Related papers

Thermodynamics of the Ising Model Encoded in Restricted Boltzmann Machines

Jing Gu, Kai Zhang

The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point Tc. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the 2d and 3d Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of the visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.

arXiv:2210.06203•Entropy•Nov 22, 2022•8 citations

Capturing the Topological Phase Transition and Thermodynamics of the 2D XY Model via Manifold-Aware Score-Based Generative Modeling

Pratyush Jha

The application of generative modeling to many-body physics offers a promising pathway for analyzing high-dimensional state spaces of spin systems. However, unlike computer vision tasks where visual fidelity suffices, physical systems require the rigorous reproduction of higher-order statistical moments and thermodynamic quantities. While Score-Based Generative Models (SGMs) have emerged as a powerful tool, their standard formulation on Euclidean embedding space is ill-suited for continuous spin systems, where variables inherently reside on a manifold. In this work, we demonstrate that training on the Euclidean space compromises the model's ability to learn the target distribution as it prioritizes to learn the manifold constraints. We address this limitation by proposing the use of Manifold-Aware Score-Based Generative Modeling framework applied to the 64x64 2D XY model (a 4096-dimensional torus). We show that our method estimates the theoretical Boltzmann score with superior precision compared to standard diffusion models. Consequently, we successfully capture the Berezinskii-Kosterlitz Thouless (BKT) phase transition and accurately reproduce second-moment quantities, such as heat capacity without explicit feature engineering. Furthermore, we demonstrate zero-shot generalization to unseen lattice sizes, accurately recovering the physics of variable system scales without retraining. Since this approach bypasses domain-specific feature engineering, it remains intrinsically generalizable to other continuous spin systems.

arXiv:2602.07548•Feb 7, 2026•0 citations

Machine-learning extraction of size-dependent temperature scales in the 2D XY model

Qingao Fan, Xu Li, Tingting Xue

Abstract Machine learning (ML) has become a useful tool for studying phase transitions in statistical systems. For the two-dimensional classical XY model, however, the topological character of the Berezinskii–Kosterlitz–Thouless (BKT) transition and pronounced finite-size effects make it nontrivial to extract reproducible size-dependent pseudo-critical temperatures from configuration data. Existing studies often stop at phase classification, leaving open how standard neural-network outputs can be turned into quantitatively testable observables. Here we develop a ML-assisted framework for the 2D XY model that uses standard network outputs to extract a size-dependent sequence of operational pseudo-critical temperatures T ∗ ( L ) . Specifically, we generate Monte Carlo configurations using embedded cluster updates, train a standard ResNet18 only on samples from the Quasi-ordered Phase and the Disordered Phase, and determine T ∗ ( L ) from bootstrap-averaged probability curves using the 50% crossing criterion. We then analyze the finite-size drift of this temperature sequence using BKT-motivated scaling and compare it with susceptibility-peak temperatures. The selected T ∗ ( L ) values display an overall downward finite-size drift compatible with BKT-motivated behavior and lie within the conventional susceptibility fluctuation regime. We further characterize and assess the extracted scale through threshold-sensitivity tests, window-variation analysis, probability calibration, comparison with a simple-physics baseline, vortex-density measurement, input-encoding comparison, and saliency mapping. These diagnostics reveal that while the network response is dominated by macroscopic physical observables, the extracted T ∗ ( L ) is conditional on representation, augmentation, and model-selection protocols. These results demonstrate that standard classifier outputs can be converted into an operational finite-size temperature scale on a well-characterized benchmark model, while also revealing the dependence of that scale on the adopted inference protocol.

arXiv:2604.00386•Physica Scripta•Sep 30, 2026•0 citations

Machine learning of phase transitions in the percolation and XY models

Wanzhou Zhang, Jiayu Liu, Tzu-Chieh Wei

In this paper, we apply machine learning methods to study phase transitions in certain statistical mechanical models on the two-dimensional lattices, whose transitions involve nonlocal or topological properties, including site and bond percolations, the XY model, and the generalized XY model. We find that using just one hidden layer in a fully connected neural network, the percolation transition can be learned and the data collapse by using the average output layer gives correct estimate of the critical exponent ν. We also study the Berezinskii-Kosterlitz-Thouless transition, which involves binding and unbinding of topological defects, vortices and antivortices, in the classical XY model. The generalized XY model contains richer phases, such as the nematic phase, the paramagnetic and the quasi-long-range ferromagnetic phases, and we also apply machine learning method to it. We obtain a consistent phase diagram from the network trained with only data along the temperature axis at two particular parameter Δ values, where Δ is the relative weight of pure XY coupling. Aside from using the spin configurations (either angles or spin components) as the input information in a convolutional neural network, we devise a feature engineering approach using the histograms of the spin orientations in order to train the network to learn the three phases in the generalized XY model and demonstrate that it indeed works. The trained network by using system size L×L can be used to the phase diagram for other sizes (L^{'}×L^{'}, where L^{'}≠L) without any further training.

arXiv:1804.02709•Physical review. E•Mar 29, 2019•126 citations

Continuous-Mixture Autoregressive Networks Learning the Kosterlitz-Thouless Transition

Lingxiao Wang, Yin Jiang, Lianyi He +1 more

We develop deep autoregressive networks with multi channels to compute many-body systems with continuous spin degrees of freedom directly. As a concrete example, we demonstrate the two-dimensional XY model with the continuous-mixture networks and rediscover the Kosterlitz–Thouless (KT) phase transition on a periodic square lattice. Vortices characterizing the quasi-long range order are accurately detected by the generative model. By learning the microscopic probability distributions from the macroscopic thermal distribution, the networks are trained as an efficient physical sampler which can approximate the free energy and estimate thermodynamic observables unbiasedly with importance sampling. As a more precise evaluation, we compute the helicity modulus to determine the KT transition temperature. Although the training process becomes more time-consuming with larger lattice sizes, the training time remains unchanged around the KT transition temperature. The continuous-mixture autoregressive networks we developed thus can be potentially used to study other many-body systems with continuous degrees of freedom.

arXiv:2005.04857•Chinese Physics Letters•Nov 21, 2022•32 citations

Machine-learning detection of the Berezinskii-Kosterlitz-Thouless transition and the second-order phase transition in XXZ models

Yusuke Miyajima, Masahito Mochizuki

We propose two machine-learning methods based on neural networks, which we respectively call the phase-classification method and the temperature-identification method, for detecting different types of phase transitions in the XXZ models without prior knowledge of their critical temperatures. The XXZ models have exchange couplings which are anisotropic in the spin space where the strength is represented by a parameter $\mathrm{\ensuremath{\Delta}}(>0)$. The models exhibit second-order phase transition when $\mathrm{\ensuremath{\Delta}}>1$, but the Berezinskii-Kosterlitz-Thouless (BKT) phase transition when $\mathrm{\ensuremath{\Delta}}<1$. In the phase-classification method, the neural network is trained using spin or vortex configurations of well-known classical spin models other than the XXZ models, e.g., the Ising models and the XY models, to classify those of the XXZ models to corresponding phases. We demonstrate that the trained neural network successfully detects the phase transitions for both $\mathrm{\ensuremath{\Delta}}>1$ and $\mathrm{\ensuremath{\Delta}}<1$, and the evaluated critical temperatures coincide well with those evaluated by conventional numerical calculations. In the temperature-identification method, on the other hand, the neural network is trained so as to identify temperatures at which the input spin or vortex configurations are generated by the Monte Carlo thermalization. The critical temperatures are evaluated by analyzing the optimized weight matrix, which coincides with the result of numerical calculation for the second-order phase transition in the Ising-like XXZ model with $\mathrm{\ensuremath{\Delta}}=1.05$ but cannot be determined uniquely for the BKT transition in the XY-like XXZ model with $\mathrm{\ensuremath{\Delta}}=0.95$.

arXiv:2304.08871•Physical review. B./Physical review. B•Apr 17, 2023•15 citations
Detecting the topological Kosterlitz-Thouless (KT) transition in the prototypical two-dimensional (2D) XY model using unsupervised machine learning methods has long been a challenging problem due to the lack of suitable order parameters. To address this issue, we begin with a conventional real-valued restricted Boltzmann machine (RBM) (RBM-xy), which uses exponential conditional probabilities to generate visible units. We then develop a novel real-valued RBM (RBM-CosSin) featuring nonlinear cos/sincos/sincos/sin activation, whose visible units follow the von Mises distribution. Our findings reveal that RBM-CosSin effectively learns the underlying Boltzmann distribution of 2D XY systems and generates authentic XY configurations that accurately capture both thermodynamics and topological order (vortex). Furthermore, we demonstrate that it is possible to extract phase transition information, including the KT transition, from the weight matrices without relying on prior physics knowledge.
The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point Tc. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the 2d and 3d Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of the visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.
The application of generative modeling to many-body physics offers a promising pathway for analyzing high-dimensional state spaces of spin systems. However, unlike computer vision tasks where visual fidelity suffices, physical systems require the rigorous reproduction of higher-order statistical moments and thermodynamic quantities. While Score-Based Generative Models (SGMs) have emerged as a powerful tool, their standard formulation on Euclidean embedding space is ill-suited for continuous spin systems, where variables inherently reside on a manifold. In this work, we demonstrate that training on the Euclidean space compromises the model's ability to learn the target distribution as it prioritizes to learn the manifold constraints. We address this limitation by proposing the use of Manifold-Aware Score-Based Generative Modeling framework applied to the 64x64 2D XY model (a 4096-dimensional torus). We show that our method estimates the theoretical Boltzmann score with superior precision compared to standard diffusion models. Consequently, we successfully capture the Berezinskii-Kosterlitz Thouless (BKT) phase transition and accurately reproduce second-moment quantities, such as heat capacity without explicit feature engineering. Furthermore, we demonstrate zero-shot generalization to unseen lattice sizes, accurately recovering the physics of variable system scales without retraining. Since this approach bypasses domain-specific feature engineering, it remains intrinsically generalizable to other continuous spin systems.
Abstract Machine learning (ML) has become a useful tool for studying phase transitions in statistical systems. For the two-dimensional classical XY model, however, the topological character of the Berezinskii–Kosterlitz–Thouless (BKT) transition and pronounced finite-size effects make it nontrivial to extract reproducible size-dependent pseudo-critical temperatures from configuration data. Existing studies often stop at phase classification, leaving open how standard neural-network outputs can be turned into quantitatively testable observables. Here we develop a ML-assisted framework for the 2D XY model that uses standard network outputs to extract a size-dependent sequence of operational pseudo-critical temperatures T ∗ ( L ) . Specifically, we generate Monte Carlo configurations using embedded cluster updates, train a standard ResNet18 only on samples from the Quasi-ordered Phase and the Disordered Phase, and determine T ∗ ( L ) from bootstrap-averaged probability curves using the 50% crossing criterion. We then analyze the finite-size drift of this temperature sequence using BKT-motivated scaling and compare it with susceptibility-peak temperatures. The selected T ∗ ( L ) values display an overall downward finite-size drift compatible with BKT-motivated behavior and lie within the conventional susceptibility fluctuation regime. We further characterize and assess the extracted scale through threshold-sensitivity tests, window-variation analysis, probability calibration, comparison with a simple-physics baseline, vortex-density measurement, input-encoding comparison, and saliency mapping. These diagnostics reveal that while the network response is dominated by macroscopic physical observables, the extracted T ∗ ( L ) is conditional on representation, augmentation, and model-selection protocols. These results demonstrate that standard classifier outputs can be converted into an operational finite-size temperature scale on a well-characterized benchmark model, while also revealing the dependence of that scale on the adopted inference protocol.
In this paper, we apply machine learning methods to study phase transitions in certain statistical mechanical models on the two-dimensional lattices, whose transitions involve nonlocal or topological properties, including site and bond percolations, the XY model, and the generalized XY model. We find that using just one hidden layer in a fully connected neural network, the percolation transition can be learned and the data collapse by using the average output layer gives correct estimate of the critical exponent ν. We also study the Berezinskii-Kosterlitz-Thouless transition, which involves binding and unbinding of topological defects, vortices and antivortices, in the classical XY model. The generalized XY model contains richer phases, such as the nematic phase, the paramagnetic and the quasi-long-range ferromagnetic phases, and we also apply machine learning method to it. We obtain a consistent phase diagram from the network trained with only data along the temperature axis at two particular parameter Δ values, where Δ is the relative weight of pure XY coupling. Aside from using the spin configurations (either angles or spin components) as the input information in a convolutional neural network, we devise a feature engineering approach using the histograms of the spin orientations in order to train the network to learn the three phases in the generalized XY model and demonstrate that it indeed works. The trained network by using system size L×L can be used to the phase diagram for other sizes (L^{'}×L^{'}, where L^{'}≠L) without any further training.
We develop deep autoregressive networks with multi channels to compute many-body systems with continuous spin degrees of freedom directly. As a concrete example, we demonstrate the two-dimensional XY model with the continuous-mixture networks and rediscover the Kosterlitz–Thouless (KT) phase transition on a periodic square lattice. Vortices characterizing the quasi-long range order are accurately detected by the generative model. By learning the microscopic probability distributions from the macroscopic thermal distribution, the networks are trained as an efficient physical sampler which can approximate the free energy and estimate thermodynamic observables unbiasedly with importance sampling. As a more precise evaluation, we compute the helicity modulus to determine the KT transition temperature. Although the training process becomes more time-consuming with larger lattice sizes, the training time remains unchanged around the KT transition temperature. The continuous-mixture autoregressive networks we developed thus can be potentially used to study other many-body systems with continuous degrees of freedom.
We propose two machine-learning methods based on neural networks, which we respectively call the phase-classification method and the temperature-identification method, for detecting different types of phase transitions in the XXZ models without prior knowledge of their critical temperatures. The XXZ models have exchange couplings which are anisotropic in the spin space where the strength is represented by a parameter $\mathrm{\ensuremath{\Delta}}(>0)$. The models exhibit second-order phase transition when $\mathrm{\ensuremath{\Delta}}>1$, but the Berezinskii-Kosterlitz-Thouless (BKT) phase transition when $\mathrm{\ensuremath{\Delta}}<1$. In the phase-classification method, the neural network is trained using spin or vortex configurations of well-known classical spin models other than the XXZ models, e.g., the Ising models and the XY models, to classify those of the XXZ models to corresponding phases. We demonstrate that the trained neural network successfully detects the phase transitions for both $\mathrm{\ensuremath{\Delta}}>1$ and $\mathrm{\ensuremath{\Delta}}<1$, and the evaluated critical temperatures coincide well with those evaluated by conventional numerical calculations. In the temperature-identification method, on the other hand, the neural network is trained so as to identify temperatures at which the input spin or vortex configurations are generated by the Monte Carlo thermalization. The critical temperatures are evaluated by analyzing the optimized weight matrix, which coincides with the result of numerical calculation for the second-order phase transition in the Ising-like XXZ model with $\mathrm{\ensuremath{\Delta}}=1.05$ but cannot be determined uniquely for the BKT transition in the XY-like XXZ model with $\mathrm{\ensuremath{\Delta}}=0.95$.