Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2609.20912

Do Quantum Models Scale Like LLMs?

David S. Berman, Ying-Jer Kao, Roger G. Melko, Alexander G. Stapleton

cs.LGquant-phSubmitted Sep 17, 2026• Announced Sep 21, 2026• 1 citation
arXivHTMLPDF

Abstract

In this work, we study the neural scaling laws of RydbergGPT, an autoregressive transformer model trained on qubit projective measurement data gathered from interacting Rydberg atom arrays. The quantum system is known to exhibit a finite-size remnant of a critical point as the laser detuning parameter is varied. We find that near the critical point the transformer loss as a function of training dataset size is well described by a power-law with a loss floor correction. However, away from criticality the quality of the power-law description is substantially reduced. We then compare the statistical structure of both Rydberg measurements and natural-language corpora using an entropy-normalised, finite sample corrected mutual information "two-point" function. We find that near-critical statistics of the two point functions are closest to those observed in natural-language, whilst other qubit configurations far from the critical point have two-point functions that decay more rapidly. This supports the hypothesis that multi-scale dependence contributes to stable neural scaling, and that scaling behaviour should be viewed as a property of the model-data pair.

AI Summary

The paper asks whether transformers trained on quantum measurement distributions exhibit language-model-like data-scaling laws, and whether their stability relates to multiscale statistical dependence. It studies RydbergGPT, an autoregressive transformer trained on synthetic occupation-basis samples from quantum Monte Carlo simulations of an open $6\times6$ Rydberg array. Model size is fixed, with $R_b/a=1.15$, $\beta\Omega=8$, and six detunings spanning the finite-size critical region near $\delta/\Omega\simeq1.1$. Training-data fractions from 10% to 100% are evaluated over ten seeds and fitted to $\mathcal L(D)=\mathcal L_\infty+BD^{-\gamma}$. At detunings 1.05 and 1.17, aggregate validation fits give $R^2=0.989$ and $0.983$, exponents $\gamma=0.656$ and $0.791$, and loss floors 0.4839 and 0.4683, respectively. Fits become less reproducible and have lower goodness of fit at remote detunings. This tests fixed-capacity data scaling, rather than compute-optimal joint parameter/data scaling. To investigate a possible statistical explanation, the paper compares entropy-normalised, permutation-null-corrected pairwise mutual-information profiles from $10^5$ Rydberg configurations with character-level profiles from AG News and Yahoo Answers. Near-critical distributions show strong local dependence followed by a weaker extended tail, qualitatively resembling the language profiles; remote-detuning profiles approach the null level more rapidly. The proposed connection between multiscale dependence and stable learning curves remains a hypothesis. Stated limitations include the small, finite-temperature array, open boundaries, dependence on the mutual-information reference construction, and symmetry sensitivity. Neither universal exponents nor a shared mechanism with language are established.

Some signs of AI-generated textIncrementalSmall numerics onlyMissing comparison

Error found: The reported power-law fits use ten dataset fractions spanning one decade at one model size and one lattice size, without alternative-curve comparisons or extrapolation tests, so they establish an in-range description rather than a uniquely supported scaling law. The analysis selects the minimum logged validation loss for each dataset fraction and seed without reporting an independent evaluation of the selected checkpoint, leaving potential selection optimism unquantified.

Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.

Comments
10 pages, 6 figures

Related papers

Large language model scaling laws for neural quantum states in quantum chemistry

Oliver Knitter, Dan Zhao, Stefan Leichenauer +1 more

Abstract Scaling laws have been used to describe how large language model (LLM) performance scales with model size, training data size, or amount of computational resources. Motivated by the fact that the field of neural quantum states (NQS) has increasingly adopted LLM-based components, we seek to understand NQS scaling laws, thereby shedding light on the scalability and optimal performance-resource trade-offs of NQS ansätze. In particular, we identify scaling laws that predict the performance, as measured by absolute error and V -score, for transformer-based NQS as a function of problem size in second-quantized quantum chemistry applications. By performing analogous compute-constrained optimization of the obtained parametric curves using a simplified compute estimate model, we find that the relationship between model size and training time is highly dependent on loss metric and ansatz, and does not follow the approximately linear compute-optimal tradeoff found by the Chinchilla laws.

arXiv:2509.12679•Machine Learning Science and Technology•Mar 17, 2026•2 citations

Scaling Laws for Neural-Network Quantum States

Riccardo Rende, Alessandro Sinibaldi, Luciano Loris Viteritti +3 more

Scaling laws, the power-law relations between loss, architecture size, and compute observed in modern neural networks, offer a quantitative way to characterize the complexity of a learning problem, with the exponent governing the decay of the loss reflecting how rapidly additional resources translate into improved accuracy, and thus how hard the target is to learn. Whether an analogous framework can characterize the complexity of physical problems remains open. We address this question for Neural-Network Quantum States, a leading variational approach for strongly correlated quantum many-body systems. Using transformer wave functions to approximate ground states of the $J_1$-$J_2$ Heisenberg model on triangular and square lattices with up to $20\times 20$ sites, we find that the $V$-score, a measure of accuracy of a variational state, decays as a power law in training compute. Under an appropriate rescaling of compute, results for different system sizes collapse onto a single curve, analogous to scaling collapse in critical phenomena. The resulting power law is, to a good approximation, independent of the number of sites, showing that the transformer Ansatz is size-consistent for the systems considered. The exponent decreases systematically with frustration, identifying it as a quantitative measure of representational difficulty of the ground state and establishing scaling laws as a general framework for benchmarking variational ansätze.

cond-mat.dis-nnCross-listedarXiv:2606.02794•Jun 3, 2026•9 citations

RydbergGPT

David Fitzek, Yi Hong Teoh, Hin Pok Fung +5 more

Abstract We introduce a generative pretrained transformer (GPT) designed to learn the measurement outcomes of a neutral atom array quantum computer. Based on a vanilla transformer, our encoder–decoder architecture takes as input the interacting Hamiltonian, and outputs an autoregressive sequence of qubit measurement probabilities. Its performance is studied in the vicinity of a quantum phase transition in Rydberg atoms in a square lattice array. We explore the model’s generalization capabilities by demonstrating that it can accurately predict ground-state measurement outcomes for Hamiltonian parameter values that were not included in the training data. We evaluate three model variants, each trained for a fixed duration on a single NVIDIA A100 GPU, by examining their predictions of key physical observables. These results establish performance benchmarks for scaling to larger RydbergGPT models. These can act as benchmarks for the scaling of larger RydbergGPT models in the future. Finally, we release RydbergGPT as open-source software to facilitate the development of foundation models for diverse quantum computing platforms and datasets.

arXiv:2405.21052•Machine Learning Science and Technology•Nov 7, 2025•32 citations

Empirical Scaling Laws of Quantum Vision Transformers

Yuqing Li, Fengyi Gao, Xin Jin +3 more

Quantum machine learning (QML) is a promising application of quantum computing, yet when and how practical learning models can achieve quantum advantage remains poorly understood. Even a polynomial improvement in the resources needed to attain a given predictive performance would be significant. In this work, we investigate empirical scaling laws of quantum vision transformers (QViTs), which connect QML to the transformer architecture underlying modern large language models. We ask whether QViTs exhibit scaling behavior analogous to classical models and how this behavior changes when qubit number is included alongside dataset size and model parameters. The validation losses in our numerical image-classification experiments are well described by a joint shifted power-law model in qubit number, transformer depth, and training dataset size. Expressing this fit in terms of the classical encoder parameter budget identifies a regime in which increasing the number of qubits can compensate for fewer classical encoder parameters at comparable predicted performance. This finding is consistent with earlier observations of parameter efficiency in quantum learning models and suggests a direction for exploring possible resource savings in QML.

quant-pharXiv:2610.13970•Sep 15, 2026•0 citations

Extremely slow scaling of minimal Hamming distance in quantum sampling data

P. S. Golubev, I. A. Iakovlev, V. V. Mazurenko

Quantum data can be obtained from a diverse range of sources, including direct measurements from noisy quantum processors, cold-atom simulators, and classical approximations such as variational neural-network states. However, our ability to characterize these systems is fundamentally limited, as the available measurement data are often sparse compared to the exponentially large Hilbert space of the system. To address this, we propose using the expected minimal Hamming distance calculated for a set of unique bitstrings as a robust metric revealing a distinct power-law behaviour. Crucially, we have established a direct connection between the empirical power-law parameters and the intrinsic dimension of the quantum measurement outcomes. Through various examples of real experiments and simulations, we show that the power-law parameters reliably characterize the underlying geometric structure of measurement outcomes and identify quantum phase transitions from limited quantum information, without the need for accumulating extensive statistics or explicitly calculating physical observables. This enables the analysis of completely different quantum experiments within a single framework. Furthermore, we demonstrate the versatility of our approach on classical high-dimensional data, showing that the power-law parameters can successfully discriminate different classes of handwritten digits from the MNIST dataset.

quant-pharXiv:2606.04558•Jun 4, 2026•0 citations

Information-Theoretic Scaling Laws of Neural Quantum States

Yiming Lu, Sriram Bharadwaj, Dikshant Rathore +1 more

We establish an information-theoretic scaling law for generic autoregressive neural quantum states, determined by the middle-cut mutual information of the wavefunction amplitude. By formalizing the virtual bond as an effective information channel across a sequence bipartition, we rigorously prove that exact autoregressive representation of a quantum state requires the virtual-bond dimension to scale with the amplitude mutual information. For stabilizer-state families, we show that this law yields an explicit, analytical rank formula. Applying this framework across quantum-state tomography, ground-state and finite-temperature learning, our numerical experiments expose precise exponent matching, architecture-dependent scaling differences between recurrent and Transformer neural quantum state, and the critical role of autoregressive basis ordering. These results establish a rigorous physical link between the intrinsic structure of a quantum many-body state and the corresponding neural-network capacity required for its faithful representation.

quant-pharXiv:2603.23468•Mar 25, 2026•7 citations
In this work, we study the neural scaling laws of RydbergGPT, an autoregressive transformer model trained on qubit projective measurement data gathered from interacting Rydberg atom arrays. The quantum system is known to exhibit a finite-size remnant of a critical point as the laser detuning parameter is varied. We find that near the critical point the transformer loss as a function of training dataset size is well described by a power-law with a loss floor correction. However, away from criticality the quality of the power-law description is substantially reduced. We then compare the statistical structure of both Rydberg measurements and natural-language corpora using an entropy-normalised, finite sample corrected mutual information "two-point" function. We find that near-critical statistics of the two point functions are closest to those observed in natural-language, whilst other qubit configurations far from the critical point have two-point functions that decay more rapidly. This supports the hypothesis that multi-scale dependence contributes to stable neural scaling, and that scaling behaviour should be viewed as a property of the model-data pair.
The paper asks whether transformers trained on quantum measurement distributions exhibit language-model-like data-scaling laws, and whether their stability relates to multiscale statistical dependence. It studies RydbergGPT, an autoregressive transformer trained on synthetic occupation-basis samples from quantum Monte Carlo simulations of an open 6×66\times66×6 Rydberg array. Model size is fixed, with Rb/a=1.15R_b/a=1.15Rb​/a=1.15, βΩ=8\beta\Omega=8βΩ=8, and six detunings spanning the finite-size critical region near δ/Ω≃1.1\delta/\Omega\simeq1.1δ/Ω≃1.1.
Training-data fractions from 10% to 100% are evaluated over ten seeds and fitted to L(D)=L∞+BD−γ\mathcal L(D)=\mathcal L_\infty+BD^{-\gamma}L(D)=L∞​+BD−γ. At detunings 1.05 and 1.17, aggregate validation fits give R2=0.989R^2=0.989R2=0.989 and 0.9830.9830.983, exponents γ=0.656\gamma=0.656γ=0.656 and 0.7910.7910.791, and loss floors 0.4839 and 0.4683, respectively. Fits become less reproducible and have lower goodness of fit at remote detunings. This tests fixed-capacity data scaling, rather than compute-optimal joint parameter/data scaling.
To investigate a possible statistical explanation, the paper compares entropy-normalised, permutation-null-corrected pairwise mutual-information profiles from 10510^5105 Rydberg configurations with character-level profiles from AG News and Yahoo Answers. Near-critical distributions show strong local dependence followed by a weaker extended tail, qualitatively resembling the language profiles; remote-detuning profiles approach the null level more rapidly.
The proposed connection between multiscale dependence and stable learning curves remains a hypothesis. Stated limitations include the small, finite-temperature array, open boundaries, dependence on the mutual-information reference construction, and symmetry sensitivity. Neither universal exponents nor a shared mechanism with language are established.
Abstract Scaling laws have been used to describe how large language model (LLM) performance scales with model size, training data size, or amount of computational resources. Motivated by the fact that the field of neural quantum states (NQS) has increasingly adopted LLM-based components, we seek to understand NQS scaling laws, thereby shedding light on the scalability and optimal performance-resource trade-offs of NQS ansätze. In particular, we identify scaling laws that predict the performance, as measured by absolute error and V -score, for transformer-based NQS as a function of problem size in second-quantized quantum chemistry applications. By performing analogous compute-constrained optimization of the obtained parametric curves using a simplified compute estimate model, we find that the relationship between model size and training time is highly dependent on loss metric and ansatz, and does not follow the approximately linear compute-optimal tradeoff found by the Chinchilla laws.
Scaling laws, the power-law relations between loss, architecture size, and compute observed in modern neural networks, offer a quantitative way to characterize the complexity of a learning problem, with the exponent governing the decay of the loss reflecting how rapidly additional resources translate into improved accuracy, and thus how hard the target is to learn. Whether an analogous framework can characterize the complexity of physical problems remains open. We address this question for Neural-Network Quantum States, a leading variational approach for strongly correlated quantum many-body systems. Using transformer wave functions to approximate ground states of the J1J_1J1​-J2J_2J2​ Heisenberg model on triangular and square lattices with up to 20×2020\times 2020×20 sites, we find that the VVV-score, a measure of accuracy of a variational state, decays as a power law in training compute. Under an appropriate rescaling of compute, results for different system sizes collapse onto a single curve, analogous to scaling collapse in critical phenomena. The resulting power law is, to a good approximation, independent of the number of sites, showing that the transformer Ansatz is size-consistent for the systems considered. The exponent decreases systematically with frustration, identifying it as a quantitative measure of representational difficulty of the ground state and establishing scaling laws as a general framework for benchmarking variational ansätze.
Abstract We introduce a generative pretrained transformer (GPT) designed to learn the measurement outcomes of a neutral atom array quantum computer. Based on a vanilla transformer, our encoder–decoder architecture takes as input the interacting Hamiltonian, and outputs an autoregressive sequence of qubit measurement probabilities. Its performance is studied in the vicinity of a quantum phase transition in Rydberg atoms in a square lattice array. We explore the model’s generalization capabilities by demonstrating that it can accurately predict ground-state measurement outcomes for Hamiltonian parameter values that were not included in the training data. We evaluate three model variants, each trained for a fixed duration on a single NVIDIA A100 GPU, by examining their predictions of key physical observables. These results establish performance benchmarks for scaling to larger RydbergGPT models. These can act as benchmarks for the scaling of larger RydbergGPT models in the future. Finally, we release RydbergGPT as open-source software to facilitate the development of foundation models for diverse quantum computing platforms and datasets.
Quantum machine learning (QML) is a promising application of quantum computing, yet when and how practical learning models can achieve quantum advantage remains poorly understood. Even a polynomial improvement in the resources needed to attain a given predictive performance would be significant. In this work, we investigate empirical scaling laws of quantum vision transformers (QViTs), which connect QML to the transformer architecture underlying modern large language models. We ask whether QViTs exhibit scaling behavior analogous to classical models and how this behavior changes when qubit number is included alongside dataset size and model parameters. The validation losses in our numerical image-classification experiments are well described by a joint shifted power-law model in qubit number, transformer depth, and training dataset size. Expressing this fit in terms of the classical encoder parameter budget identifies a regime in which increasing the number of qubits can compensate for fewer classical encoder parameters at comparable predicted performance. This finding is consistent with earlier observations of parameter efficiency in quantum learning models and suggests a direction for exploring possible resource savings in QML.
Quantum data can be obtained from a diverse range of sources, including direct measurements from noisy quantum processors, cold-atom simulators, and classical approximations such as variational neural-network states. However, our ability to characterize these systems is fundamentally limited, as the available measurement data are often sparse compared to the exponentially large Hilbert space of the system. To address this, we propose using the expected minimal Hamming distance calculated for a set of unique bitstrings as a robust metric revealing a distinct power-law behaviour. Crucially, we have established a direct connection between the empirical power-law parameters and the intrinsic dimension of the quantum measurement outcomes. Through various examples of real experiments and simulations, we show that the power-law parameters reliably characterize the underlying geometric structure of measurement outcomes and identify quantum phase transitions from limited quantum information, without the need for accumulating extensive statistics or explicitly calculating physical observables. This enables the analysis of completely different quantum experiments within a single framework. Furthermore, we demonstrate the versatility of our approach on classical high-dimensional data, showing that the power-law parameters can successfully discriminate different classes of handwritten digits from the MNIST dataset.
We establish an information-theoretic scaling law for generic autoregressive neural quantum states, determined by the middle-cut mutual information of the wavefunction amplitude. By formalizing the virtual bond as an effective information channel across a sequence bipartition, we rigorously prove that exact autoregressive representation of a quantum state requires the virtual-bond dimension to scale with the amplitude mutual information. For stabilizer-state families, we show that this law yields an explicit, analytical rank formula. Applying this framework across quantum-state tomography, ground-state and finite-temperature learning, our numerical experiments expose precise exponent matching, architecture-dependent scaling differences between recurrent and Transformer neural quantum state, and the critical role of autoregressive basis ordering. These results establish a rigorous physical link between the intrinsic structure of a quantum many-body state and the corresponding neural-network capacity required for its faithful representation.