Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Journal of Artificial Intelligence and Machine Learning

A Hybrid Quantum Re-Uploading Framework for Multimodal Emotion Recognition

Sai Siva Naga Chandra Pathuru, S. Sri Harsha

Published Jan 1, 2026• 0 citations
Publisher

Abstract

Multimodal emotion recognition can benefit from combining complementary information from facial images, speech, and text. However, integrating heterogeneous modalities obtained from independently collected datasets introduces challenges in data alignment and label consistency. This study presents a hybrid quantum–classical framework that combines modality-specific classical encoders with a quantum re-uploading variational classifier for six-class emotion recognition. Facial images, speech samples, and textual samples were obtained from separate publicly available datasets and aligned at the emotion-label level to construct a unified multimodal experimental setting. Image, speech, and text encoders generated modality-specific representations that were fused through a classical neural network and projected into a six-dimensional quantum input space. A six-qubit quantum circuit with three re-uploading layers repeatedly encoded the fused features before six-class classification. On a test set of 6,347 samples, the classical Image–Speech–Text fusion model achieved 73.94% accuracy and a 70.64% macro F1-score, whereas the proposed quantum re-uploading model achieved 72.57% accuracy and a 68.76% macro F1-score. The quantum model used 8,706 trainable parameters compared with 15,440,006 for the classical multimodal model, representing a 99.94% reduction in trainable parameters. The results indicate that quantum re-uploading can provide competitive performance for multimodal emotion recognition while highlighting the limitations of label-level alignment across independently collected datasets.

DOI
10.51483/ijaiml.6.11s.2026.1130-1147

Related papers

A Hybrid Quantum Re-Uploading Framework for Multimodal Emotion Recognition

Sai Siva Naga Chandra Pathuru, S. Sri Harsha

Multimodal emotion recognition can benefit from combining complementary information from facial images, speech, and text.However, integrating heterogeneous modalities obtained from independently collected datasets introduces challenges in data alignment and label consistency.This study presents a hybrid quantum-classical framework that combines modality-specific classical encoders with a quantum re-uploading variational classifier for six-class emotion recognition.Facial images, speech samples, and textual samples were obtained from separate publicly available datasets and aligned at the emotion-label level to construct a unified multimodal experimental setting.Image, speech, and text encoders generated modality-specific representations that were fused through a classical neural network and projected into a six-dimensional quantum input space.A six-qubit quantum circuit with three re-uploading layers repeatedly encoded the fused features before six-class classification.On a test set of 6,347 samples, the classical Image-Speech-Text fusion model achieved 73.94% accuracy and a 70.64% macro F1-score, whereas the proposed quantum reuploading model achieved 72.57% accuracy and a 68.76% macro F1-score.The quantum model used 8,706 trainable parameters compared with 15,440,006 for the classical multimodal model, representing a 99.94% reduction in trainable parameters.The results indicate that quantum re-uploading can provide competitive performance for multimodal emotion recognition while highlighting the limitations of label-level alignment across independently collected datasets.

International Journal of Drug Delivery Technology•Sep 16, 2026•0 citations

HQTN-SER: Speech Emotion Recognition with Hybrid Quantum Tensor Networks

Mahad Mohtashim, Nouhaila Innan, Muhammad Shafique

Speech emotion recognition (SER) remains fragile in real-world conditions because emotional cues are subtle, speaker-dependent, and easily confounded by recording variability, while high-performing deep models typically rely on large and carefully curated training sets. Quantum machine learning offers an alternative way to introduce nonlinear correlation modeling with compact modules, yet existing quantum SER studies remain limited, and the impact of circuit structure is not well understood. This paper presents HQTN-SER, a hybrid quantum-classical framework that investigates how quantum tensor network connectivity can support SER under small-qubit settings. HQTN-SER introduces (i) an MPS-inspired quantum tensor network module that enforces structured interactions to model correlations in speech representations with a small number of trainable parameters, and (ii) a fusion strategy that combines quantum measurement features with a learned classical latent embedding for end-to-end emotion classification. We evaluate HQTN-SER on three public benchmarks (RAVDESS, SAVEE, and MDER) under a unified preprocessing and training protocol. The proposed model achieves test accuracies of 80.09\% on RAVDESS, 78.12\% on SAVEE, and 73.51\% on MDER, with corresponding macro-F1 scores of 0.8012, 0.7826, and 0.7351, respectively. The model also exhibits stable convergence while using only three or four qubits, suggesting that MPS-inspired circuit structure can be an effective and hardware-aware design choice for quantum-assisted SER. These results provide a compact experimental baseline for studying structured quantum modules in affective computing.

quant-pharXiv:2605.14523•May 15, 2026•0 citations

Hybrid Framework for Quantum - Enhanced Emotion Recognition

G.Vishnupriya G.Vishnupriya, P. Devadharshini, Dr.R.Kabilan Dr.R.Kabilan

Emotion recognition seeks to identify a person’s state by examining signals such as facial expressions, speech, physiological signals and conversational content. Yet accurate emotion recognition presents obstacles because these signals can be complex, noisy, nonlinear and heavily affected by surrounding context. This paper investigates quantum-based representations combined with machine learning techniques to enhance emotion classification. The research examines emotion sources, including facial expressions, speech, EEG signals and conversational data. It explores how quantum-inspired techniques can merge with established deep learning methods to produce representations. The design presented fuses a neural network with a four qubit ZZFeatureMap component enabling the system to learn data from both conventional and quantum derived feature spaces. Experiments examined EEG segments for each participant and the system achieved average recognition rates of seventy nine point six percent for valence and eighty one point five percent for arousal across ten repeated trials. These findings show that quantum derived features can add information when combined with conventional deep learning methods. Nevertheless because experiments used simulated circuits the results do not prove a performance advantage over conventional techniques on actual quantum computers. The study also highlights difficulties such as quantum noise, computational scalability, experimental assessment, data privacy and real world deployment.

JOURNAL OF ADVANCE AND FUTURE RESEARCH•Oct 1, 2026•0 citations

Hybrid Quantum Convolutional Neural Networks in TensorFlow Quantum

Bishrelt Khurelsukh

This paper presents a quantum convolutional neural network (QCNN) that classifies the happy and sad emotions from the fabled facial emotion recognition dataset (FER), FER-2013. The model uses the data re-uploading concept on the qubit(s) of the parameterized quantum circuits (PQCs) in the quantum convolutional layer of the hybrid QCNN model, which achieved a 70.25 % validation accuracy on the test set. Later, a model averaging ensemble approach was implemented which yielded results motivating further investigation.

Irish Signals and Systems Conference•Jun 9, 2022•5 citations

Hybrid Quantum–Classical Convolutional Networks for Robust Multimodal Pattern Recognition

Qingmei Li

Robust multimodal pattern recognition remains challenging under real-world conditions characterized by sensor degradation, distribution shift, and missing modalities. This study introduces a Hybrid Quantum–Classical Convolutional Network (Hybrid QC–CNN) that combines modality-specific convolutional encoders with a compact Parameterized Quantum Circuit (PQC) feature transformation and mask-aware fusion to stabilize inference under partial and corrupted evidence. Under matched-capacity baselines, the proposed model achieves 0.913 clean accuracy and 0.907 macro-F1 while delivering superior robustness across corruption severities. Averaged over severity levels 1–5, Hybrid QC–CNN attains 0.812 accuracy compared to 0.782 (multimodal transformer) and 0.771 (CNN late fusion), and improves worst-severity (level 5) accuracy to 0.701, exceeding the transformer (0.602) by +9.9 percentage points and late fusion (0.583) by +11.8 points. Robustness summarized by AURC reaches 0.823, surpassing the transformer (0.793) and late fusion (0.781). Reliability under degradation is also improved: ECE is 0.032 on clean data and 0.061 at severity 3, compared to 0.089 (transformer) and 0.094 (late fusion). Missing-modality stress tests further show stable degradation: with only one modality available, Hybrid QC–CNN achieves 0.793 (image-only), 0.742 (audio-only), and 0.764 (text-only), outperforming both transformer and late-fusion baselines across all single-modality conditions. Ablation removing the quantum layer reduces average corruption accuracy from 0.812 to 0.792 and worst-severity accuracy from 0.701 to 0.641, indicating that the PQC contributes meaningful robustness beyond mask-aware fusion and regularization. Collectively, results support hybrid quantum–classical feature transformation as a targeted mechanism for multimodal robustness and calibrated confidence.

Journal of Quantum Artificial Intelligence•May 1, 2026•0 citations

Representation learning with parameterised quantum circuits for advancing speech emotion recognition

Thejan Rajapakshe, Rajib Kumar Rana, Farina Riaz +2 more

Quantum machine learning (QML) offers a promising avenue for advancing representation learning in complex signal domains. In this study, we investigate the use of parameterised quantum circuits (PQCs) for speech emotion recognition (SER)-a challenging task due to the subtle temporal variations and overlapping affective states in vocal signals. We propose a hybrid quantum-classical architecture that integrates PQCs into a conventional convolutional neural network (CNN), leveraging quantum properties such as superposition and entanglement to enrich emotional feature representations. Experimental evaluations on three benchmark datasets IEMOCAP, RECOLA, and MSP-IMPROV-demonstrate that our hybrid model achieves improved classification performance relative to a purely classical CNN baseline, with over 50% reduction in trainable parameters. Furthermore, Adjusted Rand Index (ARI) analysis demonstrates that the quantum model yields feature representations with improved alignment to true emotion classes compared with the classical model, reinforcing the observed performance gains. This work provides early evidence of the potential for QML to enhance emotion recognition and lays the foundation for future quantum-enabled affective computing systems.

arXiv:2501.12050•Scientific Reports•Nov 22, 2025•7 citations
Multimodal emotion recognition can benefit from combining complementary information from facial images, speech, and text. However, integrating heterogeneous modalities obtained from independently collected datasets introduces challenges in data alignment and label consistency. This study presents a hybrid quantum–classical framework that combines modality-specific classical encoders with a quantum re-uploading variational classifier for six-class emotion recognition. Facial images, speech samples, and textual samples were obtained from separate publicly available datasets and aligned at the emotion-label level to construct a unified multimodal experimental setting. Image, speech, and text encoders generated modality-specific representations that were fused through a classical neural network and projected into a six-dimensional quantum input space. A six-qubit quantum circuit with three re-uploading layers repeatedly encoded the fused features before six-class classification. On a test set of 6,347 samples, the classical Image–Speech–Text fusion model achieved 73.94% accuracy and a 70.64% macro F1-score, whereas the proposed quantum re-uploading model achieved 72.57% accuracy and a 68.76% macro F1-score. The quantum model used 8,706 trainable parameters compared with 15,440,006 for the classical multimodal model, representing a 99.94% reduction in trainable parameters. The results indicate that quantum re-uploading can provide competitive performance for multimodal emotion recognition while highlighting the limitations of label-level alignment across independently collected datasets.
Multimodal emotion recognition can benefit from combining complementary information from facial images, speech, and text.However, integrating heterogeneous modalities obtained from independently collected datasets introduces challenges in data alignment and label consistency.This study presents a hybrid quantum-classical framework that combines modality-specific classical encoders with a quantum re-uploading variational classifier for six-class emotion recognition.Facial images, speech samples, and textual samples were obtained from separate publicly available datasets and aligned at the emotion-label level to construct a unified multimodal experimental setting.Image, speech, and text encoders generated modality-specific representations that were fused through a classical neural network and projected into a six-dimensional quantum input space.A six-qubit quantum circuit with three re-uploading layers repeatedly encoded the fused features before six-class classification.On a test set of 6,347 samples, the classical Image-Speech-Text fusion model achieved 73.94% accuracy and a 70.64% macro F1-score, whereas the proposed quantum reuploading model achieved 72.57% accuracy and a 68.76% macro F1-score.The quantum model used 8,706 trainable parameters compared with 15,440,006 for the classical multimodal model, representing a 99.94% reduction in trainable parameters.The results indicate that quantum re-uploading can provide competitive performance for multimodal emotion recognition while highlighting the limitations of label-level alignment across independently collected datasets.
Speech emotion recognition (SER) remains fragile in real-world conditions because emotional cues are subtle, speaker-dependent, and easily confounded by recording variability, while high-performing deep models typically rely on large and carefully curated training sets. Quantum machine learning offers an alternative way to introduce nonlinear correlation modeling with compact modules, yet existing quantum SER studies remain limited, and the impact of circuit structure is not well understood. This paper presents HQTN-SER, a hybrid quantum-classical framework that investigates how quantum tensor network connectivity can support SER under small-qubit settings. HQTN-SER introduces (i) an MPS-inspired quantum tensor network module that enforces structured interactions to model correlations in speech representations with a small number of trainable parameters, and (ii) a fusion strategy that combines quantum measurement features with a learned classical latent embedding for end-to-end emotion classification. We evaluate HQTN-SER on three public benchmarks (RAVDESS, SAVEE, and MDER) under a unified preprocessing and training protocol. The proposed model achieves test accuracies of 80.09\% on RAVDESS, 78.12\% on SAVEE, and 73.51\% on MDER, with corresponding macro-F1 scores of 0.8012, 0.7826, and 0.7351, respectively. The model also exhibits stable convergence while using only three or four qubits, suggesting that MPS-inspired circuit structure can be an effective and hardware-aware design choice for quantum-assisted SER. These results provide a compact experimental baseline for studying structured quantum modules in affective computing.
Emotion recognition seeks to identify a person’s state by examining signals such as facial expressions, speech, physiological signals and conversational content. Yet accurate emotion recognition presents obstacles because these signals can be complex, noisy, nonlinear and heavily affected by surrounding context. This paper investigates quantum-based representations combined with machine learning techniques to enhance emotion classification. The research examines emotion sources, including facial expressions, speech, EEG signals and conversational data. It explores how quantum-inspired techniques can merge with established deep learning methods to produce representations. The design presented fuses a neural network with a four qubit ZZFeatureMap component enabling the system to learn data from both conventional and quantum derived feature spaces. Experiments examined EEG segments for each participant and the system achieved average recognition rates of seventy nine point six percent for valence and eighty one point five percent for arousal across ten repeated trials. These findings show that quantum derived features can add information when combined with conventional deep learning methods. Nevertheless because experiments used simulated circuits the results do not prove a performance advantage over conventional techniques on actual quantum computers. The study also highlights difficulties such as quantum noise, computational scalability, experimental assessment, data privacy and real world deployment.
This paper presents a quantum convolutional neural network (QCNN) that classifies the happy and sad emotions from the fabled facial emotion recognition dataset (FER), FER-2013. The model uses the data re-uploading concept on the qubit(s) of the parameterized quantum circuits (PQCs) in the quantum convolutional layer of the hybrid QCNN model, which achieved a 70.25 % validation accuracy on the test set. Later, a model averaging ensemble approach was implemented which yielded results motivating further investigation.
Robust multimodal pattern recognition remains challenging under real-world conditions characterized by sensor degradation, distribution shift, and missing modalities. This study introduces a Hybrid Quantum–Classical Convolutional Network (Hybrid QC–CNN) that combines modality-specific convolutional encoders with a compact Parameterized Quantum Circuit (PQC) feature transformation and mask-aware fusion to stabilize inference under partial and corrupted evidence. Under matched-capacity baselines, the proposed model achieves 0.913 clean accuracy and 0.907 macro-F1 while delivering superior robustness across corruption severities. Averaged over severity levels 1–5, Hybrid QC–CNN attains 0.812 accuracy compared to 0.782 (multimodal transformer) and 0.771 (CNN late fusion), and improves worst-severity (level 5) accuracy to 0.701, exceeding the transformer (0.602) by +9.9 percentage points and late fusion (0.583) by +11.8 points. Robustness summarized by AURC reaches 0.823, surpassing the transformer (0.793) and late fusion (0.781). Reliability under degradation is also improved: ECE is 0.032 on clean data and 0.061 at severity 3, compared to 0.089 (transformer) and 0.094 (late fusion). Missing-modality stress tests further show stable degradation: with only one modality available, Hybrid QC–CNN achieves 0.793 (image-only), 0.742 (audio-only), and 0.764 (text-only), outperforming both transformer and late-fusion baselines across all single-modality conditions. Ablation removing the quantum layer reduces average corruption accuracy from 0.812 to 0.792 and worst-severity accuracy from 0.701 to 0.641, indicating that the PQC contributes meaningful robustness beyond mask-aware fusion and regularization. Collectively, results support hybrid quantum–classical feature transformation as a targeted mechanism for multimodal robustness and calibrated confidence.
Quantum machine learning (QML) offers a promising avenue for advancing representation learning in complex signal domains. In this study, we investigate the use of parameterised quantum circuits (PQCs) for speech emotion recognition (SER)-a challenging task due to the subtle temporal variations and overlapping affective states in vocal signals. We propose a hybrid quantum-classical architecture that integrates PQCs into a conventional convolutional neural network (CNN), leveraging quantum properties such as superposition and entanglement to enrich emotional feature representations. Experimental evaluations on three benchmark datasets IEMOCAP, RECOLA, and MSP-IMPROV-demonstrate that our hybrid model achieves improved classification performance relative to a purely classical CNN baseline, with over 50% reduction in trainable parameters. Furthermore, Adjusted Rand Index (ARI) analysis demonstrates that the quantum model yields feature representations with improved alignment to true emotion classes compared with the classical model, reinforcing the observed performance gains. This work provides early evidence of the potential for QML to enhance emotion recognition and lays the foundation for future quantum-enabled affective computing systems.