The widespread success of deep neural networks has revealed a surprise in classical machine learning: very complex models often generalize well while simultaneously overfitting training data. This phenomenon of benign overfitting has been studied for a variety of classical models with the goal of better understanding the mechanisms behind deep learning. Characterizing the phenomenon in the context of quantum machine learning might similarly improve our understanding of the relationship between overfitting, overparameterization, and generalization. In this work, we provide a characterization of benign overfitting in quantum models. To do this, we derive the behavior of a classical interpolating Fourier features models for regression on noisy signals, and show how a class of quantum models exhibits analogous features, thereby linking the structure of quantum circuits (such as data-encoding and state preparation operations) to overparameterization and overfitting in quantum models. We intuitively explain these features according to the ability of the quantum model to interpolate noisy data with locally "spiky" behavior and provide a concrete demonstration example of benign overfitting.
The widespread success of deep neural networks has revealed a surprise in classical machine learning: very complex models often generalize well while simultaneously overfitting training data. This phenomenon of benign overfitting has been studied for a variety of classical models with the goal of better understanding the mechanisms behind deep learning. Characterizing the phenomenon in the context of quantum machine learning might similarly improve our understanding of the relationship between overfitting, overparameterization, and generalization. In this work, we provide a characterization of benign overfitting in quantum models. To do this, we derive the behavior of a classical interpolating Fourier features models for regression on noisy signals, and show how a class of quantum models exhibits analogous features, thereby linking the structure of quantum circuits (such as data-encoding and state preparation operations) to overparameterization and overfitting in quantum models. We intuitively explain these features according to the ability of the quantum model to interpolate noisy data with locally "spiky" behavior and provide a concrete demonstration example of benign overfitting.
The double descent phenomenon challenges traditional statistical learning theory by revealing scenarios where larger models do not necessarily lead to reduced performance on unseen data. While this counterintuitive behavior has been observed in a variety of classical machine learning models, particularly modern neural network architectures, it remains elusive within the context of quantum machine learning. In this work, we analytically demonstrate that linear regression models in quantum feature spaces can exhibit double descent behavior by drawing on insights from classical linear regression and random matrix theory. Additionally, our numerical experiments on quantum kernel methods across different real-world datasets and system sizes further confirm the existence of a test error peak, a characteristic feature of double descent. Our findings provide evidence that quantum models can operate in the modern, overparameterized regime without experiencing overfitting, potentially opening pathways to improved learning performance beyond traditional statistical learning theory.
Kernel methods compare inputs through feature maps. Quantum kernels follow the same principle: input data are encoded into quantum states, which define quantum feature representations in Hilbert spaces. Kernel values are then obtained by estimating inner products between these states using suitable quantum circuit measurements. As a result, quantum kernels may be intractable to compute classically while remaining efficiently computable on quantum hardware, potentially leading to a quantum advantage. However, designing effective quantum kernels remains a major challenge. Many quantum kernels, such as the fidelity kernel, suffer from exponential concentration. This results in near-identity kernel matrices that fail to capture meaningful data correlations and lead to overfitting and poor generalization. In this paper, we propose a novel strategy for constructing quantum kernels that achieve good generalization performance, drawing inspiration from benign overfitting in classical machine learning. We introduce the concept of Local-Global quantum kernels, which combine two components: a local quantum kernel based on measurements of small subsystems, and a global quantum kernel derived from full-system measurements. To support the effectiveness of the proposed construction, we show theoretically and empirically that Local-Global quantum kernels exhibit benign overfitting.
Abstract In classical machine learning (ML), “overfitting” is the phenomenon occurring when a given model learns the training data excessively well, and it thus performs poorly on unseen data. A commonly employed technique in ML is the so called “dropout,” which prevents computational units from becoming too specialized, hence reducing the risk of overfitting. With the advent of quantum neural networks (QNNs) as learning models, overfitting might soon become an issue, owing to the increasing depth of quantum circuits as well as multiple embedding of classical features, which are employed to give the computational nonlinearity. Here, a generalized approach is presented to apply the dropout technique in QNN models, defining and analyzing different quantum dropout strategies to avoid overfitting and achieve a high level of generalization. This study allows to envision the power of quantum dropout in enabling generalization, providing useful guidelines on determining the maximal dropout probability for a given model, based on overparametrization theory. It also highlights how quantum dropout does not impact the features of the QNN models, such as expressibility and entanglement. All these conclusions are supported by extensive numerical simulations and may pave the way to efficiently employing deep quantum machine learning (QML) models based on state‐of‐the‐art QNNs.
Quantum machine learning models have shown successful generalization performance even when trained with few data. In this work, through systematic randomization experiments, we show that traditional approaches to understanding generalization fail to explain the behavior of such quantum models. Our experiments reveal that state-of-the-art quantum neural networks accurately fit random states and random labeling of training data. This ability to memorize random data defies current notions of small generalization error, problematizing approaches that build on complexity measures such as the VC dimension, the Rademacher complexity, and all their uniform relatives. We complement our empirical results with a theoretical construction showing that quantum neural networks can fit arbitrary labels to quantum states, hinting at their memorization ability. Our results do not preclude the possibility of good generalization with few training data but rather rule out any possible guarantees based only on the properties of the model family. These findings expose a fundamental challenge in the conventional understanding of generalization in quantum machine learning and highlight the need for a paradigm shift in the study of quantum models for machine learning tasks.
Quantum convolutional neural network (QCNN), an early application for quantum computers in the noisy intermediate-scale quantum era, has been consistently proven successful as a machine learning (ML) algorithm for several tasks with significant accuracy. Derived from its classical counterpart, QCNN is prone to overfitting. Overfitting is a typical shortcoming of ML models that are trained too closely to the availed training dataset and perform relatively poorly on unseen datasets for a similar problem. In this work we study post-training approaches for mitigating overfitting in QCNNs. We find that a straightforward adaptation of a classical post-training method, known as neuron dropout, to the quantum setting leads to a significant and undesirable consequence: a substantial decrease in success probability of the QCNN. We argue that this effect exposes the crucial role of entanglement in QCNNs and the vulnerability of QCNNs to entanglement loss. Hence, we propose a parameter adaptation method as an alternative method. Our method is computationally efficient and is found to successfully handle overfitting in the test cases.
This phenomenon is due to the development of complex high-dimensional datasets within machine learning that may not be effectively processed, identified or classified by classical computers. Multi-dimensional data are increasingly common in a wide range of scientific disciplines including genomics, finance modeling, computer vision, climate prediction et al, and standard approaches to analysis have begun to falter in the face of the curse of dimensionality, computational efficiency, and reduced interpretability. Quantum computing, in this context, is a disruptive paradigm that can bring about a substantial overhaul to machines ability to learn. In this work, we investigate a family of new quantumenhanced protocols specially designed for enhancing pattern recognition in multi-dimensional machine learning problems. We demonstrate a hybrid quantum-classical scheme, combining quantum state representation, quantum feature encoding and quantum kernel estimation to make complex data transformations in a time that scales only with the squareroot of the data size. Important techniques are the use of variational quantum circuits, quantum principal component analysis (qPCA), and quantum support vector machines (QSVMs) to uncover the structure of non-linear and highdimensional features spaces that would have been hidden otherwise. We benchmark these quantum models against classical deep learning models on three datasets: healthcare diagnostics, image classification, and sensor-based time series prediction. Experiments demonstrate that quantum-enhanced models not only accelerate the training process and save the memory space but also achieve better generalization on identifying shape features and abstract patterns. By entanglement and quantum parallelism, we can encode and process information of a multi-dimensional input at the same time and provide a compact and informative representation. In addition to that, quantum circuits show a good potential to deal with noisy and overfitting when dealing with sparsed and imbalanced data. This work demonstrates the quantumcomputational power for pattern recognition, and paves the way for scalable, efficient, and interpretable quantum machine learning. Through connecting quantum algorithms to practical machine learning problems, the work paves the way for applications of quantum computing in challenging highdimensional data analysis problems and in a new territory of research interests at the intersection of quantum information and artificial intelligence.