Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2603.21586

Distilling the knowledge with quantum neural networks

Yuxuan Yan, Sitian Qian, Qi Zhao, Xingjian Zhang

quant-phSubmitted Mar 23, 2026• Announced Mar 24, 2026• 2 citations
arXivHTMLPDF

Abstract

Quantum Neural Networks (QNNs) are a promising class of quantum machine learning models with potential quantum advantages when implemented on scalable, error-corrected quantum computers. However, as system sizes increase, deploying QNNs becomes challenging. Similar to their classical counterparts, a key obstacle to their practical applications is that large-scale QNNs may not be easily deployed on smaller systems that have limited resources. Here, we tackle this challenge by compressing QNNs via knowledge distillation. We demonstrate how well-trained QNNs on large systems can be distilled into smaller architectures with similar configurations. We numerically show that knowledge distillation helps reduce the training cost of QNNs in terms of the number of qubits and circuit depth. Additionally, we find that a self-knowledge-distillation approach can accelerate training convergence. We believe our results offer new strategies for the efficient compression and practical deployment of QNNs.

Related papers

Bridging Classical and Quantum Machine Learning: Knowledge Transfer From Classical to Quantum Neural Networks Using Knowledge Distillation

Mohammad Junayed Hasan, M. R. C. Mahdy

Quantum neural networks (QNNs), harnessing superposition and entanglement, have shown potential to surpass classical methods in complex learning tasks but remain limited by hardware constraints and noisy conditions. In this work, we present a novel framework for transferring knowledge from classical convolutional neural networks (CNNs) to QNNs via knowledge distillation, thereby reducing the need for resource intensive quantum training and error mitigation. We conduct extensive experiments using two parameterized quantum circuits (PQCs) with 4 and 8 qubits on MNIST, Fashion MNIST, and CIFAR10 datasets. The approach demonstrates consistent accuracy improvements attributed to distilled knowledge from larger classical networks. Through ablation studies, we systematically compare the effect of state of the art dimensionality reduction techniques fully connected layers, center cropping, principal component analysis, and pooling to compress high-dimensional image data prior to quantum encoding. Our findings reveal that fully connected layers retain the most salient features for QNN inference, thereby surpassing other down sampling approaches. Additionally, we examine state of the art data encoding methods (amplitude, angle, and qubit encoding) and identify amplitude encoding as the optimal strategy, yielding superior accuracy across all tested datasets and qubit configurations. Through computational analyses, we show that our distilled 4-qubit and 8-qubit QNNs achieve competitive performance while utilizing significantly fewer parameters than their classical counterparts. Our results establish a promising paradigm for bridging classical deep learning and emerging quantum computing, paving the way for more powerful, resource conscious models in quantum machine intelligence.

arXiv:2311.13810•Nov 23, 2023•13 citations

Knowledge Distillation in Quantum Neural Network Using Approximate Synthesis

Mahabubul Alam, Satwik Kundu, Swaroop Ghosh

Recent assertions of a potential advantage of Quantum Neural Network (QNN) for specific Machine Learning (ML) tasks have sparked the curiosity of a sizable number of application researchers. The parameterized quantum circuit (PQC), a major building block of a QNN, consists of several layers of single-qubit rotations and multi-qubit entanglement operations. The optimum number of PQC layers for a particular ML task is generally unknown. A larger network often provides better performance in noiseless simulations. However, it may perform poorly on hardware compared to a shallower network. Because the amount of noise varies amongst quantum devices, the optimal depth of PQC can vary significantly. Additionally, the gates chosen for the PQC may be suitable for one type of hardware but not for another due to compilation overhead. This makes it difficult to generalize a QNN design to wide range of hardware and noise levels. An alternate approach is to build and train multiple QNN models targeted for each hardware which can be expensive. To circumvent these issues, we introduce the concept of knowledge distillation in QNN using approximate synthesis. The proposed approach will create a new QNN network with (i) a reduced number of layers or (ii) a different gate set without having to train it from scratch. Training the new network for a few epochs can compensate for the loss caused by approximation error. Through empirical analysis, we demonstrate ≈71.4% reduction in circuit layers, and still achieve ≈16.2% better accuracy under noise.

arXiv:2207.01801•Asia and South Pacific Design Automation Conference (ASP-DAC)•Jan 16, 2023•10 citations

Classical to Quantum Knowledge Distillation: a Study on the Impact of Hybridization

Simone Piperno, Giacomo Vittori, David Windridge +2 more

Knowledge distillation is a widely explored technique in classical machine learning, in which a smaller or more efficient model is trained to mimic the behavior of a larger, more complex model. In this study, we extend the concept of knowledge distillation from classical architectures to quantum architectures, with the goal of improving the training of quantum models while potentially reducing the number of parameters compared to their classical counterparts. Given the inherent challenges in training quantum neural networks, leveraging knowledge from well-established classical models could provide valuable insights and advantages, particularly in terms of model efficiency and performance. In this work we explore the potential benefits of this approach, evaluating a hybrid quantum model against a non-hybrid quantum baseline. While the proposed study is still in the preliminary stage, it aims to set the scene for further investigation into the most appropriate architecture for classical-to-quantum knowledge distillation in order to enhance the development and optimization of quantum neural networks more generally.

International Joint Conference on Neural Networks (IJCNN)•Jun 30, 2025•2 citations

Quantum-enhanced hybrid-model compression using knowledge distillation

Luigi , Massimo Esposito, Francesco Gargiulo

Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high-dimensional solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while preserving the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via feature-based knowledge distillation, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller hybrid student model. The proposed hybrid quantum-classical student architecture incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, employing a linear $$\varvec{R_y}$$ angle-encoding feature map followed by an EfficientSU2 variational ansatz, with features bounded through a $$\varvec{\tanh (\cdot )\times \frac{\pi }{2}}$$ scaling to ensure stable gradient propagation. Gradient computation is performed via Reverse-Mode (adjoint) differentiation through the StatevectorEstimator primitive, which provides exact, shot-noise-free gradients at the cost of a single forward–backward pass per optimization step. The hybrid student is compared against a parameter-matched purely classical student under strict 1:1 parameter parity across a six-task binary benchmark spanning MNIST and Fashion-MNIST, as well as a four-class multi-class benchmark, totaling eleven classification experiments. A hardware validation experiment is additionally conducted on the IQM Garnet 20-qubit quantum processor using the Parameter-Shift Rule. The results indicate that the hybrid student outperforms its classical counterpart in five of six binary tasks (83%) and in the two hardest multi-class configurations, with advantages most pronounced in high-confusion regimes where compact classical classifiers approach the limits of their expressive capacity. On real quantum hardware, accuracy decreases from 99.04% in noiseless simulation to 80.50%, reflecting the impact of gate noise, readout errors, and transpilation overhead. While the hybrid model demonstrates improved parameter efficiency and expressive capacity, it incurs higher computational overhead and exhibits greater epoch-to-epoch training variability relative to classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and identify directions for future work including hardware-efficient gradient estimation and advanced error mitigation protocols. Overall, our findings suggest that variational quantum circuits can serve as compact yet expressive components within neural compression schemes, with their representational advantage most effectively realized in binary and high-ambiguity classification scenarios.

Quantum Machine Intelligence•Sep 4, 2026•0 citations

Quantum Knowledge Distillation: State-of-the-art, Applications and Open Problems

Aldana Al-Sulaiti, Kuan‐Cheng Chen, Muhammad Bilal Akram Dastagir +2 more

Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.

arXiv:2611.08461•Nov 5, 2026•0 citations

Quantum Machine Learning Model Compression Techniques

Jincheng Zhang

Quantum machine learning (QML) holds immense promise for tackling complex computational problems. However, realizing this potential is significantly hampered by the inherent challenges associated with the size and computational demands of quantum algorithms. Full-scale QML models, often based on complex quantum circuits, require substantial resources for both storage and execution, presenting a major barrier to practical application. This research investigates the critical need for model compression techniques tailored specifically for QML. We propose a novel approach combining quantization, pruning, and knowledge distillation to effectively reduce the size and computational complexity of QML models without significantly compromising their performance. The core aim is to enable the deployment of QML algorithms on resource-constrained quantum devices. This paper outlines the theoretical framework, detailing the mathematical formulations and techniques employed, and discusses potential future research directions.

•Sep 3, 2026•0 citations
Quantum Neural Networks (QNNs) are a promising class of quantum machine learning models with potential quantum advantages when implemented on scalable, error-corrected quantum computers. However, as system sizes increase, deploying QNNs becomes challenging. Similar to their classical counterparts, a key obstacle to their practical applications is that large-scale QNNs may not be easily deployed on smaller systems that have limited resources. Here, we tackle this challenge by compressing QNNs via knowledge distillation. We demonstrate how well-trained QNNs on large systems can be distilled into smaller architectures with similar configurations. We numerically show that knowledge distillation helps reduce the training cost of QNNs in terms of the number of qubits and circuit depth. Additionally, we find that a self-knowledge-distillation approach can accelerate training convergence. We believe our results offer new strategies for the efficient compression and practical deployment of QNNs.
Quantum neural networks (QNNs), harnessing superposition and entanglement, have shown potential to surpass classical methods in complex learning tasks but remain limited by hardware constraints and noisy conditions. In this work, we present a novel framework for transferring knowledge from classical convolutional neural networks (CNNs) to QNNs via knowledge distillation, thereby reducing the need for resource intensive quantum training and error mitigation. We conduct extensive experiments using two parameterized quantum circuits (PQCs) with 4 and 8 qubits on MNIST, Fashion MNIST, and CIFAR10 datasets. The approach demonstrates consistent accuracy improvements attributed to distilled knowledge from larger classical networks. Through ablation studies, we systematically compare the effect of state of the art dimensionality reduction techniques fully connected layers, center cropping, principal component analysis, and pooling to compress high-dimensional image data prior to quantum encoding. Our findings reveal that fully connected layers retain the most salient features for QNN inference, thereby surpassing other down sampling approaches. Additionally, we examine state of the art data encoding methods (amplitude, angle, and qubit encoding) and identify amplitude encoding as the optimal strategy, yielding superior accuracy across all tested datasets and qubit configurations. Through computational analyses, we show that our distilled 4-qubit and 8-qubit QNNs achieve competitive performance while utilizing significantly fewer parameters than their classical counterparts. Our results establish a promising paradigm for bridging classical deep learning and emerging quantum computing, paving the way for more powerful, resource conscious models in quantum machine intelligence.
Recent assertions of a potential advantage of Quantum Neural Network (QNN) for specific Machine Learning (ML) tasks have sparked the curiosity of a sizable number of application researchers. The parameterized quantum circuit (PQC), a major building block of a QNN, consists of several layers of single-qubit rotations and multi-qubit entanglement operations. The optimum number of PQC layers for a particular ML task is generally unknown. A larger network often provides better performance in noiseless simulations. However, it may perform poorly on hardware compared to a shallower network. Because the amount of noise varies amongst quantum devices, the optimal depth of PQC can vary significantly. Additionally, the gates chosen for the PQC may be suitable for one type of hardware but not for another due to compilation overhead. This makes it difficult to generalize a QNN design to wide range of hardware and noise levels. An alternate approach is to build and train multiple QNN models targeted for each hardware which can be expensive. To circumvent these issues, we introduce the concept of knowledge distillation in QNN using approximate synthesis. The proposed approach will create a new QNN network with (i) a reduced number of layers or (ii) a different gate set without having to train it from scratch. Training the new network for a few epochs can compensate for the loss caused by approximation error. Through empirical analysis, we demonstrate ≈71.4% reduction in circuit layers, and still achieve ≈16.2% better accuracy under noise.
Knowledge distillation is a widely explored technique in classical machine learning, in which a smaller or more efficient model is trained to mimic the behavior of a larger, more complex model. In this study, we extend the concept of knowledge distillation from classical architectures to quantum architectures, with the goal of improving the training of quantum models while potentially reducing the number of parameters compared to their classical counterparts. Given the inherent challenges in training quantum neural networks, leveraging knowledge from well-established classical models could provide valuable insights and advantages, particularly in terms of model efficiency and performance. In this work we explore the potential benefits of this approach, evaluating a hybrid quantum model against a non-hybrid quantum baseline. While the proposed study is still in the preliminary stage, it aims to set the scene for further investigation into the most appropriate architecture for classical-to-quantum knowledge distillation in order to enhance the development and optimization of quantum neural networks more generally.
Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high-dimensional solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while preserving the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via feature-based knowledge distillation, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller hybrid student model. The proposed hybrid quantum-classical student architecture incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, employing a linear $$\varvec{R_y}$$ angle-encoding feature map followed by an EfficientSU2 variational ansatz, with features bounded through a $$\varvec{\tanh (\cdot )\times \frac{\pi }{2}}$$ scaling to ensure stable gradient propagation. Gradient computation is performed via Reverse-Mode (adjoint) differentiation through the StatevectorEstimator primitive, which provides exact, shot-noise-free gradients at the cost of a single forward–backward pass per optimization step. The hybrid student is compared against a parameter-matched purely classical student under strict 1:1 parameter parity across a six-task binary benchmark spanning MNIST and Fashion-MNIST, as well as a four-class multi-class benchmark, totaling eleven classification experiments. A hardware validation experiment is additionally conducted on the IQM Garnet 20-qubit quantum processor using the Parameter-Shift Rule. The results indicate that the hybrid student outperforms its classical counterpart in five of six binary tasks (83%) and in the two hardest multi-class configurations, with advantages most pronounced in high-confusion regimes where compact classical classifiers approach the limits of their expressive capacity. On real quantum hardware, accuracy decreases from 99.04% in noiseless simulation to 80.50%, reflecting the impact of gate noise, readout errors, and transpilation overhead. While the hybrid model demonstrates improved parameter efficiency and expressive capacity, it incurs higher computational overhead and exhibits greater epoch-to-epoch training variability relative to classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and identify directions for future work including hardware-efficient gradient estimation and advanced error mitigation protocols. Overall, our findings suggest that variational quantum circuits can serve as compact yet expressive components within neural compression schemes, with their representational advantage most effectively realized in binary and high-ambiguity classification scenarios.
Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.
Quantum machine learning (QML) holds immense promise for tackling complex computational problems. However, realizing this potential is significantly hampered by the inherent challenges associated with the size and computational demands of quantum algorithms. Full-scale QML models, often based on complex quantum circuits, require substantial resources for both storage and execution, presenting a major barrier to practical application. This research investigates the critical need for model compression techniques tailored specifically for QML. We propose a novel approach combining quantization, pruning, and knowledge distillation to effectively reduce the size and computational complexity of QML models without significantly compromising their performance. The core aim is to enable the deployment of QML algorithms on resource-constrained quantum devices. This paper outlines the theoretical framework, detailing the mathematical formulations and techniques employed, and discusses potential future research directions.