Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Proceedings of the AAAI Symposium Series

Structured Supervision from Quantum Models: Distilling Robustness into Classical Networks

Syed Rameez Naqvi, Zhengming Ding, Fang Qi, Md Mostafizer RAHMAN, Lu Peng

Published May 18, 2026• 0 citations
Publisher

Abstract

Quantum machine learning offers a practical opportunity to leverage near-term quantum devices without requiring quantum inference at deployment. We propose a quantum-classical hybrid knowledge distillation framework in which a variational quantum circuit, equipped with a Quantum Fourier Transform-inspired positional encoding, acts as a soft-label teacher for a compact classical student. Rather than serving as a standalone classifier, the quantum model is used offline to generate structured probability distributions that encode global spectral information and uncertainty. These quantum-derived soft labels are distilled into a lightweight classical network using a hybrid loss that combines hard and soft supervision. Experiments on MNIST demonstrate that students trained with quantum soft labels exhibit consistent and statistically meaningful robustness improvements to Gaussian noise and in-plane rotations compared to classical distillation baselines, while maintaining strong clean accuracy and calibration. These results highlight a distinct role for NISQ-era quantum models as supervisory signal generators, enabling quantum-informed learning within fully classical deployment pipelines.

DOI
10.1609/aaaiss.v8i1.42582

Related papers

Quantum Knowledge Distillation: State-of-the-art, Applications and Open Problems

Aldana Al-Sulaiti, Kuan‐Cheng Chen, Muhammad Bilal Akram Dastagir +2 more

Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.

arXiv:2611.08461•Nov 5, 2026•0 citations

Hybrid Quantum Classical Machine Learning with Knowledge Distillation

Mingze Li, Lei Fan, Aaron Cummings +3 more

The rapid advancement of machine learning (ML) and the growing need for computational power have led to the exploration of quantum computing, which offers significant potential for faster complex calculations. However, Quantum Machine Learning (QML) faces challenges due to the limited number of qubits and noise of quantum circuits, particularly with Noisy Intermediate-Scale Quantum (NISQ) devices. These challenges severely limit the current capacity to train accurate and stable Quantum Machine Learning Models. In this paper, we propose a novel framework for QML that employs the knowledge distillation method to harness the power of well-trained classical machine learning (CML) models and enhance the training performance of QML models. In this framework, we utilize the well-trained CML as a teacher model to assist the training of the student QML model using the knowledge distillation method. By distilling knowledge from the robust CML model, our framework can potentially address the problem of the barren plateau which hinders effective model training. Knowledge distillation is well suited for this framework through the transfer of knowledge without parameter sharing. Through empirical tests, our framework has demonstrated not only an increase in the accuracy of QML models but also a notable improvement in training stability.

International Conference on Communications•Jun 9, 2024•8 citations

Quantum Transfer Learning with Distilled Quantum Teachers for Low-Resource Classification Tasks

S. Kavitha

Low-resource classification remains a persistent bottleneck for quantum machine learning due to label scarcity, class imbalance, and finite-shot measurement variance under NISQ constraints. This study proposes a hybrid framework that couples Quantum Transfer Learning with Distilled Quantum Teachers, where a high-capacity quantum teacher provides temperature-smoothed soft targets to train a compact student quantum adapter on top of a pretrained classical encoder. Experiments on three low-resource tasks (LR-Text-4, LR-Image-3, LR-Sensor-5) demonstrate consistent gains over non-distilled training at fixed student inference cost (2048 shots). The distilled student improves accuracy from 0.794 to 0.821 on LR-Text-4, from 0.781 to 0.812 on LR-Image-3, and from 0.768 to 0.809 on LR-Sensor-5, while macro F1 increases from 0.781 to 0.811, from 0.772 to 0.804, and from 0.742 to 0.789, respectively. Calibration improves substantially, with expected calibration error decreasing from 0.071 to 0.049 (LR-Text-4), from 0.074 to 0.052 (LR-Image-3), and from 0.083 to 0.056 (LR-Sensor-5), indicating more reliable confidence under scarcity. A temperature sweep reveals an optimal stability region around T ≈ 3.0 to 3.5, where aggregate accuracy peaks near 0.831 and mean calibration reaches its minimum near 0.043. Transfer ablations show that moderate encoder unfreezing (ratio 0.25) yields the best trade-off, achieving mean accuracy 0.842 and mean ECE 0.039, while full fine-tuning reduces accuracy to 0.818 and degrades calibration to 0.058. Resource analysis using a cost proxy (shots × depth) confirms that distilled students dominate the efficient regime, achieving 0.828 accuracy at cost 20,480 versus the teacher’s 0.867 accuracy at cost 180,224. Compared with classical baselines, the distilled student remains most advantageous in the high-overlap sensor setting, outperforming shallow random forests (0.809 vs 0.778 accuracy) while maintaining better calibrated uncertainty. These results position distilled quantum teachers as an effective offline supervision mechanism for compressing high-capacity quantum decision structure into deployable low-cost students for low-resource classification.

Journal of Quantum Artificial Intelligence•Aug 31, 2026•0 citations

Quantum-enhanced hybrid-model compression using knowledge distillation

Luigi , Massimo Esposito, Francesco Gargiulo

Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high-dimensional solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while preserving the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via feature-based knowledge distillation, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller hybrid student model. The proposed hybrid quantum-classical student architecture incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, employing a linear $$\varvec{R_y}$$ angle-encoding feature map followed by an EfficientSU2 variational ansatz, with features bounded through a $$\varvec{\tanh (\cdot )\times \frac{\pi }{2}}$$ scaling to ensure stable gradient propagation. Gradient computation is performed via Reverse-Mode (adjoint) differentiation through the StatevectorEstimator primitive, which provides exact, shot-noise-free gradients at the cost of a single forward–backward pass per optimization step. The hybrid student is compared against a parameter-matched purely classical student under strict 1:1 parameter parity across a six-task binary benchmark spanning MNIST and Fashion-MNIST, as well as a four-class multi-class benchmark, totaling eleven classification experiments. A hardware validation experiment is additionally conducted on the IQM Garnet 20-qubit quantum processor using the Parameter-Shift Rule. The results indicate that the hybrid student outperforms its classical counterpart in five of six binary tasks (83%) and in the two hardest multi-class configurations, with advantages most pronounced in high-confusion regimes where compact classical classifiers approach the limits of their expressive capacity. On real quantum hardware, accuracy decreases from 99.04% in noiseless simulation to 80.50%, reflecting the impact of gate noise, readout errors, and transpilation overhead. While the hybrid model demonstrates improved parameter efficiency and expressive capacity, it incurs higher computational overhead and exhibits greater epoch-to-epoch training variability relative to classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and identify directions for future work including hardware-efficient gradient estimation and advanced error mitigation protocols. Overall, our findings suggest that variational quantum circuits can serve as compact yet expressive components within neural compression schemes, with their representational advantage most effectively realized in binary and high-ambiguity classification scenarios.

Quantum Machine Intelligence•Sep 4, 2026•0 citations

Quantum Probabilistic Label Refining: Enhancing Label Quality for Robust Image Classification

Qi Fang, Peng Lu, Zhengming Ding

Learning with softmax cross-entropy on one-hot labels often leads to overconfident predictions and poor robustness under noise or perturbations. Label smoothing mitigates this by redistributing some confidence uniformly, but treats all samples equally, ignoring intra-class variability. We propose a hybrid quantum–classical framework that leverages quantum non-determinism to refine data labels into probabilistic ones, offering more nuanced, human-like uncertainty representations than label smoothing or Bayesian approaches. A variational quantum circuit (VQC) encodes inputs into multi-qubit quantum states, using entanglement and superposition to capture subtle feature correlations. Measurement via the Born rule extracts probabilistic soft labels that reflect input-specific uncertainty. These labels are then used to train a classical convolutional neural network (CNN) with soft-target cross-entropy loss. On MNIST and Fashion-MNIST, our method improves robustness—achieving up to 50% higher accuracy under noise—while maintaining competitive clean-data accuracy. It also enhances model calibration and interpretability, as CNN outputs better reflect quantum-derived uncertainty. This work introduces Quantum Probabilistic Label Refining, which bridges quantum measurement and classical deep learning to enable robust training via refined, correlation-aware labels, without architectural changes or adversarial techniques.

arXiv:2510.00528•ACM Great Lakes Symposium on VLSI (GLSVLSI)•Jun 18, 2026•0 citations

Bridging Classical and Quantum Models via Attention-Guided Feature Distillation

Tareque Bashar Ovi, Nomaiya Bashree, Faiaz Hasanuzzaman Rhythm +3 more

While Quantum Neural Networks (QNNs) offer the potential for resource-efficient image classification, practical implementations are hindered by qubit instability and finite availability, causing them to often underperform classical counterparts. This performance disparity constitutes a critical research gap. We propose a novel hybrid classical-quantum knowledge distillation approach to bridge this gap. Our method utilizes a pre-trained classical CNN (AlexNet) as a teacher to guide the training of a quantum student model by minimizing the Kullback-Leibler (KL) divergence (DKL) between their output distributions. The quantum student model employs an architecture designed to facilitate intricate feature interactions using iterated CNOT gates. This technique substantially improves quantum model accuracy, demonstrating gains of 0.34% (6-class, 3 qubits) and 0.28% (8-class, 3 qubits) on the MNIST dataset, and 0.74% (8-class, 4 qubits). Our contribution lies in operationalizing a knowledge distillation paradigm for applied Quantum Machine Learning (QML), enabling the effective transfer of robust knowledge from established classical architectures to resource-constrained quantum models, thereby enhancing faster convergence and accelerating practical QML deployment.

International Conference on Next-Generation Computing, IoT and Machine Learning (NCIM)•Jun 27, 2025•0 citations
Quantum machine learning offers a practical opportunity to leverage near-term quantum devices without requiring quantum inference at deployment. We propose a quantum-classical hybrid knowledge distillation framework in which a variational quantum circuit, equipped with a Quantum Fourier Transform-inspired positional encoding, acts as a soft-label teacher for a compact classical student. Rather than serving as a standalone classifier, the quantum model is used offline to generate structured probability distributions that encode global spectral information and uncertainty. These quantum-derived soft labels are distilled into a lightweight classical network using a hybrid loss that combines hard and soft supervision. Experiments on MNIST demonstrate that students trained with quantum soft labels exhibit consistent and statistically meaningful robustness improvements to Gaussian noise and in-plane rotations compared to classical distillation baselines, while maintaining strong clean accuracy and calibration. These results highlight a distinct role for NISQ-era quantum models as supervisory signal generators, enabling quantum-informed learning within fully classical deployment pipelines.
Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.
The rapid advancement of machine learning (ML) and the growing need for computational power have led to the exploration of quantum computing, which offers significant potential for faster complex calculations. However, Quantum Machine Learning (QML) faces challenges due to the limited number of qubits and noise of quantum circuits, particularly with Noisy Intermediate-Scale Quantum (NISQ) devices. These challenges severely limit the current capacity to train accurate and stable Quantum Machine Learning Models. In this paper, we propose a novel framework for QML that employs the knowledge distillation method to harness the power of well-trained classical machine learning (CML) models and enhance the training performance of QML models. In this framework, we utilize the well-trained CML as a teacher model to assist the training of the student QML model using the knowledge distillation method. By distilling knowledge from the robust CML model, our framework can potentially address the problem of the barren plateau which hinders effective model training. Knowledge distillation is well suited for this framework through the transfer of knowledge without parameter sharing. Through empirical tests, our framework has demonstrated not only an increase in the accuracy of QML models but also a notable improvement in training stability.
Low-resource classification remains a persistent bottleneck for quantum machine learning due to label scarcity, class imbalance, and finite-shot measurement variance under NISQ constraints. This study proposes a hybrid framework that couples Quantum Transfer Learning with Distilled Quantum Teachers, where a high-capacity quantum teacher provides temperature-smoothed soft targets to train a compact student quantum adapter on top of a pretrained classical encoder. Experiments on three low-resource tasks (LR-Text-4, LR-Image-3, LR-Sensor-5) demonstrate consistent gains over non-distilled training at fixed student inference cost (2048 shots). The distilled student improves accuracy from 0.794 to 0.821 on LR-Text-4, from 0.781 to 0.812 on LR-Image-3, and from 0.768 to 0.809 on LR-Sensor-5, while macro F1 increases from 0.781 to 0.811, from 0.772 to 0.804, and from 0.742 to 0.789, respectively. Calibration improves substantially, with expected calibration error decreasing from 0.071 to 0.049 (LR-Text-4), from 0.074 to 0.052 (LR-Image-3), and from 0.083 to 0.056 (LR-Sensor-5), indicating more reliable confidence under scarcity. A temperature sweep reveals an optimal stability region around T ≈ 3.0 to 3.5, where aggregate accuracy peaks near 0.831 and mean calibration reaches its minimum near 0.043. Transfer ablations show that moderate encoder unfreezing (ratio 0.25) yields the best trade-off, achieving mean accuracy 0.842 and mean ECE 0.039, while full fine-tuning reduces accuracy to 0.818 and degrades calibration to 0.058. Resource analysis using a cost proxy (shots × depth) confirms that distilled students dominate the efficient regime, achieving 0.828 accuracy at cost 20,480 versus the teacher’s 0.867 accuracy at cost 180,224. Compared with classical baselines, the distilled student remains most advantageous in the high-overlap sensor setting, outperforming shallow random forests (0.809 vs 0.778 accuracy) while maintaining better calibrated uncertainty. These results position distilled quantum teachers as an effective offline supervision mechanism for compressing high-capacity quantum decision structure into deployable low-cost students for low-resource classification.
Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high-dimensional solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while preserving the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via feature-based knowledge distillation, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller hybrid student model. The proposed hybrid quantum-classical student architecture incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, employing a linear $$\varvec{R_y}$$ angle-encoding feature map followed by an EfficientSU2 variational ansatz, with features bounded through a $$\varvec{\tanh (\cdot )\times \frac{\pi }{2}}$$ scaling to ensure stable gradient propagation. Gradient computation is performed via Reverse-Mode (adjoint) differentiation through the StatevectorEstimator primitive, which provides exact, shot-noise-free gradients at the cost of a single forward–backward pass per optimization step. The hybrid student is compared against a parameter-matched purely classical student under strict 1:1 parameter parity across a six-task binary benchmark spanning MNIST and Fashion-MNIST, as well as a four-class multi-class benchmark, totaling eleven classification experiments. A hardware validation experiment is additionally conducted on the IQM Garnet 20-qubit quantum processor using the Parameter-Shift Rule. The results indicate that the hybrid student outperforms its classical counterpart in five of six binary tasks (83%) and in the two hardest multi-class configurations, with advantages most pronounced in high-confusion regimes where compact classical classifiers approach the limits of their expressive capacity. On real quantum hardware, accuracy decreases from 99.04% in noiseless simulation to 80.50%, reflecting the impact of gate noise, readout errors, and transpilation overhead. While the hybrid model demonstrates improved parameter efficiency and expressive capacity, it incurs higher computational overhead and exhibits greater epoch-to-epoch training variability relative to classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and identify directions for future work including hardware-efficient gradient estimation and advanced error mitigation protocols. Overall, our findings suggest that variational quantum circuits can serve as compact yet expressive components within neural compression schemes, with their representational advantage most effectively realized in binary and high-ambiguity classification scenarios.
Learning with softmax cross-entropy on one-hot labels often leads to overconfident predictions and poor robustness under noise or perturbations. Label smoothing mitigates this by redistributing some confidence uniformly, but treats all samples equally, ignoring intra-class variability. We propose a hybrid quantum–classical framework that leverages quantum non-determinism to refine data labels into probabilistic ones, offering more nuanced, human-like uncertainty representations than label smoothing or Bayesian approaches. A variational quantum circuit (VQC) encodes inputs into multi-qubit quantum states, using entanglement and superposition to capture subtle feature correlations. Measurement via the Born rule extracts probabilistic soft labels that reflect input-specific uncertainty. These labels are then used to train a classical convolutional neural network (CNN) with soft-target cross-entropy loss. On MNIST and Fashion-MNIST, our method improves robustness—achieving up to 50% higher accuracy under noise—while maintaining competitive clean-data accuracy. It also enhances model calibration and interpretability, as CNN outputs better reflect quantum-derived uncertainty. This work introduces Quantum Probabilistic Label Refining, which bridges quantum measurement and classical deep learning to enable robust training via refined, correlation-aware labels, without architectural changes or adversarial techniques.
While Quantum Neural Networks (QNNs) offer the potential for resource-efficient image classification, practical implementations are hindered by qubit instability and finite availability, causing them to often underperform classical counterparts. This performance disparity constitutes a critical research gap. We propose a novel hybrid classical-quantum knowledge distillation approach to bridge this gap. Our method utilizes a pre-trained classical CNN (AlexNet) as a teacher to guide the training of a quantum student model by minimizing the Kullback-Leibler (KL) divergence (DKL) between their output distributions. The quantum student model employs an architecture designed to facilitate intricate feature interactions using iterated CNOT gates. This technique substantially improves quantum model accuracy, demonstrating gains of 0.34% (6-class, 3 qubits) and 0.28% (8-class, 3 qubits) on the MNIST dataset, and 0.74% (8-class, 4 qubits). Our contribution lies in operationalizing a knowledge distillation paradigm for applied Quantum Machine Learning (QML), enabling the effective transfer of robust knowledge from established classical architectures to resource-constrained quantum models, thereby enhancing faster convergence and accelerating practical QML deployment.