arXiv:2610.06581
Shaping double descent in quantum extreme learning through model design
Sabri Meyer, Ege Yilmaz, Aurelien Lucchi, Francesco Tacchino, Francesco Scala
Abstract
Increasing capacity can induce a machine learning model to generalize worse until it perfectly interpolates the training data, and better once it moves beyond interpolation. This inversion, known as \emph{double descent}, is well understood for independent features. For correlated features, its shape is governed by the spectrum of the feature covariance, but existing analyses typically yield only implicit equations. We first close this gap for classical linear regression, analytically characterizing the entire risk curve through two quantities, namely the condition number of the feature population covariance and a spectral measure of correlation strength. Then, we apply this theory on quantum models, focusing on quantum extreme learning machines (QELMs). We show that three tunable design knobs, \textit{i.e.,} encoding strength, input reuploads, and measurement shots, each regularize the QELM output covariance, and derive conditions under which the interpolation threshold remains stable. Finite measurement shot noise can be leveraged to guarantee this stability. Numerical experiments confirm each prediction, yielding concrete design principles for QELMs that generalize well beyond interpolation.
AI Summary
The paper asks how circuit architecture and measurement noise shape double descent in quantum extreme learning machines (QELMs), which fit a minimum-norm classical linear readout to fixed quantum features. It connects linear-regression risk bounds to the population second-moment matrix $\Sigma=\mathbb{E}[\phi(x)\phi(x)^\top]$, then uses the features’ finite Fourier expansion to analyze encoding strength, input reuploads, and measurement shots. For underparameterized, realizable regression, it recovers $\mathcal{R}(\gamma)=\sigma^2/(1-\gamma)$, with $\gamma=d/n$, under the sufficient correlation condition $\lambda_{\max}(\operatorname{Cov}(z\otimes z))=o(d)$, where $z=\Sigma^{-1/2}\phi(x)$. For large $\gamma$, sub-Gaussian whitened features satisfying $\|z\|=\sqrt d$ almost surely give $\mathcal{R}(\gamma)=\sigma^2+\mathcal{O}(\|w_*\|^2\lambda_{\max}(\Sigma)+\sigma^2\kappa(\Sigma)/(\sqrt\gamma-C_K)^2)$. These results use generalized Marchenko–Pastur theory and singular-value concentration. Strong input scaling approaches a Fourier-coefficient covariance, with conditioning corrections $\mathcal{O}(s^{-K})$ under density regularity assumptions. Additional reuploads improve a conditioning bound when new frequencies populate the weakest covariance eigenspace. Independent measurement batches add diagonal shot-noise covariance; under stated observable assumptions, $m_0=o(\sqrt d)$ shots per feature suffice for the correlation condition preserving the canonical threshold. Statevector experiments up to 16 qubits and real-data tests examine these predictions. The stated limitations include restrictive geometric assumptions for the overparameterized bound, incomplete justification for non-Gaussian inputs and moderate dimensions, and no general guarantee for the reupload injection condition. The analysis does not explain secondary risk peaks or guarantee that overparameterization outperforms the underparameterized regime.
Central claim may not hold: The advertised characterization of the entire double-descent curve is substantially stronger than the paper's conditional, large-overparameterization upper bound. The claimed universal small-encoding condition-number divergence fails for valid quantum features (sin(s x_1), sin(s x_2)) whose second-moment matrix remains proportional to the identity. The reupload bound requires architecture-dependent injection assumptions and does not generally compare the deeper circuit's conditioning with that of the original circuit. For labels generated from exact quantum features, finite-shot features produce regressor-correlated residuals, invalidating direct application of the paper's independent-label-noise risk theorem. The assumptions used to obtain Marchenko–Pastur convergence do not ensure convergence of inverse spectral traces or expected risk, and permit counterexamples to the stated under-parametrized risk formula.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
- Comments
- Main: 10 pages, 2 figures; Supplementary: 89 pages, 5 figures