arXiv:2609.23334
Stochastic Reconfiguration as Statistical Filtering for Overparameterized Neural Quantum States
Tak Hur
Abstract
Stochastic reconfiguration (SR) is the standard optimizer for neural quantum states (NQS), but modern NQS often have far more parameters than Monte Carlo samples. We show that in this regime the diagonal shift is more than a numerical stabilizer. It acts as a statistical filter for finite-sample generalization. At a fixed wave function, SR is ridge regression from tangent features to the centered local energy. Its residual is the expressivity gap, the part of imaginary-time evolution outside the current tangent space. This gap is orthogonal to the tangent space in population, but finite batches make it act as noise that SR can overfit. The shift therefore balances shrinkage of useful update directions against variance from fitting sampled residuals. Exact diagnostics on a $4\times4$ Heisenberg graph separate two effects of overparameterization. Larger tangent spaces help when they reduce the expressivity gap, but they can hurt when they overfit a fixed gap. In a $100$-site transverse-field Ising family trained with a foundation NQS, validation risk is U-shaped in the shift while variance decreases, matching the noisy-ridge model. This view leads to multi-shift SR (MS-SR), which averages independent ridge solves at data-adaptive shifts to form a richer, lower-variance spectral filter. Checkpoint-local experiments show that MS-SR lowers validation risk and update variance relative to the fixed-shift SR baseline. We further compare MS-SR and SR in paired online training continuations, with independent endpoint energy evaluations and a separate update-cost benchmark.
AI Summary
The paper asks how the diagonal shift in stochastic reconfiguration (SR) affects finite-sample generalization when neural quantum states have more parameters than Monte Carlo samples. At a fixed checkpoint, it formulates SR as ridge regression from tangent features to centered local energy. The residual outside the population tangent space—the expressivity gap—is orthogonal to the features in population but can be overfit on finite batches. Exact held-out-risk and bias–variance identities, together with an approximate spectral calculation, connect regularization to update error in the quantum-geometric-tensor seminorm and hence local infidelity. Exact amplitude-channel diagnostics on a $4\times4$ Heisenberg graph separate gap reduction from residual overfitting. Shared-checkpoint experiments on a 100-site conditional TFIM state show U-shaped validation residuals and decreasing update variance as the shift increases. Multi-shift SR (MS-SR) combines independently sampled ridge updates at spectrum-adaptive shifts, with held-out stacking weights. With four candidates, it reduces validation residual/update variance versus fixed-training-shift SR by $39\%/43\%$ on TFIM and $14\%/66\%$ on an $8\times8$ frustrated Heisenberg model. Validation-tuned single-shift bagging nevertheless has lower mean residuals in both systems. The measured update cost is $4.04\times$ standard SR. Five paired online continuations give mean energy-per-site difference $-1.78\times10^{-5}$, with a 95% interval spanning zero. These continuations use a different MS-SR variant, extra sampling/solves, and one shared source state. Other limitations include the approximate spectral-risk model, amplitude-only small-system diagnostics, and no established equal-cost training advantage or access to states standard optimizers miss.
Error found: The main text reports 100 independent batches for the matched-state small-system diagnostic, but its detailed appendix reports 10, leaving the sampling basis of the reported standard errors inconsistent. The TFIM validation diagnostic centers residuals using only 20 configurations per field, making its expectation 19/20 of the stated population risk under independent sampling, with a covariance-dependent correction under correlated sampling.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
- Comments
- 25 pages, 7 figures