Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Quantum Destructive Self-Attention for NISQ-EraTransformers

Yousra Bouakba, Hacene Belhadef

Published Feb 25, 2026• 0 citations
Publisher

Abstract

Abstract In the noisy intermediate-scale quantum (NISQ) era, developing attention mechanisms suitable for quantum hardware is essential for advancing quantum-enhanced Transformers applications. This work introduces a novel quantum Transformer model that integrates a low-depth, ancilla-free attention mechanism based on the Destructive swap-test variant, enabling efficient computation of attention scores without auxiliary qubits. Designed specifically for NISQ constraints, the model emphasizes noise resilience and reduced circuit depth, making it practical for current quantum devices. We evaluate the model using transparent and well-documented Natural Language Processing datasets designed for Quantum Computing (QC). Our results demonstrate competitive performance on language tasks, supported by both quantitative metrics that offer interpretable insights into the model's behavior. This work provides a viable pathway toward practical, interpretable quantum language models in the near term.

DOI
10.21203/rs.3.rs-8946631/v1

Related papers

Training Quantum Self-Attention Model in Near-Term Quantum Computer

Jieguang He, Yin Kan, Cheng Xue

Quantum computing is rapidly emerging as a revolutionary computing paradigm with significant development potential. Variational quantum algorithms, especially those employing variational quantum circuits, have demonstrated notable advantages in domains including finance, chemistry, and machine learning. However, the practical deployment of these algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices faces challenges due to quantum noise, state collapse, and limited qubits numbers. The real-world performance of these algorithms remains largely unexplored. This paper addresses this gap by proposing and implementing a novel method for training quantum self-attention model for text classification directly on quantum computing chips. This initiative marks the first instance of employing such models on actual quantum hardware. Our approach optimizes quantum circuit designs to mitigate the impact of quantum noise and introduces parallel training strategies to enhance efficiency. Our experiments, conducted on the “Wukong” superconducting 72-qubit quantum computer, demonstrate that our models can be effectively trained on real quantum hardware. The results from the actual quantum chips slightly outperformed those from simulators, confirming the practical feasibility of applying quantum self-attention model to natural language processing tasks. These results validate the effectiveness of our approach and provide valuable insights for the future deployment of quantum machine learning models.

International Conference on Wireless Communications and Signal Processing (WCSP)•Oct 24, 2024•4 citations

Hybrid Encoding-based Quantum Self Attention for Financial Time Series Analysis

S Lokes, Suresh Reddy, Samuel Yen Chi Chen

Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.

IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW)•Apr 6, 2025•1 citation

FitQT: a fully integrated and trainable quantum transformer for scalable natural language processing

Lisa Pham, Anh The Le

Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.

Engineering Research Express•Mar 19, 2026•0 citations

Self-Adaptive Quantum Circuits for NISQ Devices

PROF.JABEEN TAJ MK

Noisy Intermediate-Scale Quantum (NISQ) devices are promising platforms for practical quantum computing, but their performance is significantly affected by limited qubit availability, gate errors, decoherence, circuit depth, and hardware-dependent noise. Conventional quantum circuits are generally designed and optimized before execution and may not respond effectively to changing hardware conditions during computation. This research proposes a Self-Adaptive Quantum Circuit framework that dynamically modifies quantum circuits according to the current computational resources and noise characteristics of NISQ devices. The proposed framework monitors parameters such as qubit fidelity, gate error rates, circuit depth, execution time, and resource availability, and automatically selects suitable circuit configurations to improve computational performance. Adaptive mechanisms can be employed to reduce unnecessary gates, optimize qubit utilization, and mitigate the impact of hardware noise while maintaining the required computational accuracy. The framework will be evaluated using simulation platforms and available NISQ quantum hardware, comparing adaptive circuits with conventional static circuits using metrics such as accuracy, fidelity, circuit depth, gate count, execution time, and error rate. The expected outcome is a resource-efficient and noise-aware quantum circuit optimization approach capable of improving the reliability and performance of quantum applications on near-term quantum hardware. The proposed research aims to contribute toward more practical, flexible, and hardware-aware quantum computing systems for NISQ-era applications. Keywords: NISQ Computing, Self-Adaptive Quantum Circuits, Quantum Circuit Optimization, Quantum Error Mitigation, Quantum Noise, Quantum Computing.

JOURNAL OF ADVANCE AND FUTURE RESEARCH•Aug 1, 2026•0 citations

Vectorized Attention with Learnable Encoding for Quantum Transformer

Ziqing Guo, Ziwen Pan, Alex Khan +1 more

Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep-parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked-attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM’s state-of-the-art, high-fidelity Kingston QPU. Our noise intermediate-scale quantum (NISQ)-friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.

arXiv:2508.18464•Proceedings of the AAAI Symposium Series•Nov 23, 2025•2 citations

Measurements With A Quantum Vision Transformer: A Naive Approach

Dominic Pasquali, Michele Grossi, Sofia Vallecorsa

In mainstream machine learning, transformers are gaining widespread usage. As Vision Transformers rise in popularity in computer vision, they now aim to tackle a wide variety of machine learning applications. In particular, transformers for High Energy Physics (HEP) experiments continue to be investigated for tasks including jet tagging, particle reconstruction, and pile-up mitigation. An improved Quantum Vision Transformer (QViT) with a quantum-enhanced self-attention mechanism is introduced and discussed. A shallow circuit is proposed for each component of self-attention to leverage current Noisy Intermediate Scale Quantum (NISQ) devices. Variations of the hybrid architecture/model are explored and analyzed. The results demonstrate a successful proof of concept for the QViT, and establish a competitive performance benchmark for the proposed design and implementation. The findings also provide strong motivation to experiment with different architectures, hyperparameters, and datasets, setting the stage for implementation in HEP environments where transformers are increasingly used in state of the art machine learning solutions.

EPJ Web of Conferences•Jan 1, 2024•5 citations
Abstract In the noisy intermediate-scale quantum (NISQ) era, developing attention mechanisms suitable for quantum hardware is essential for advancing quantum-enhanced Transformers applications. This work introduces a novel quantum Transformer model that integrates a low-depth, ancilla-free attention mechanism based on the Destructive swap-test variant, enabling efficient computation of attention scores without auxiliary qubits. Designed specifically for NISQ constraints, the model emphasizes noise resilience and reduced circuit depth, making it practical for current quantum devices. We evaluate the model using transparent and well-documented Natural Language Processing datasets designed for Quantum Computing (QC). Our results demonstrate competitive performance on language tasks, supported by both quantitative metrics that offer interpretable insights into the model's behavior. This work provides a viable pathway toward practical, interpretable quantum language models in the near term.
Quantum computing is rapidly emerging as a revolutionary computing paradigm with significant development potential. Variational quantum algorithms, especially those employing variational quantum circuits, have demonstrated notable advantages in domains including finance, chemistry, and machine learning. However, the practical deployment of these algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices faces challenges due to quantum noise, state collapse, and limited qubits numbers. The real-world performance of these algorithms remains largely unexplored. This paper addresses this gap by proposing and implementing a novel method for training quantum self-attention model for text classification directly on quantum computing chips. This initiative marks the first instance of employing such models on actual quantum hardware. Our approach optimizes quantum circuit designs to mitigate the impact of quantum noise and introduces parallel training strategies to enhance efficiency. Our experiments, conducted on the “Wukong” superconducting 72-qubit quantum computer, demonstrate that our models can be effectively trained on real quantum hardware. The results from the actual quantum chips slightly outperformed those from simulators, confirming the practical feasibility of applying quantum self-attention model to natural language processing tasks. These results validate the effectiveness of our approach and provide valuable insights for the future deployment of quantum machine learning models.
Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
Noisy Intermediate-Scale Quantum (NISQ) devices are promising platforms for practical quantum computing, but their performance is significantly affected by limited qubit availability, gate errors, decoherence, circuit depth, and hardware-dependent noise. Conventional quantum circuits are generally designed and optimized before execution and may not respond effectively to changing hardware conditions during computation. This research proposes a Self-Adaptive Quantum Circuit framework that dynamically modifies quantum circuits according to the current computational resources and noise characteristics of NISQ devices. The proposed framework monitors parameters such as qubit fidelity, gate error rates, circuit depth, execution time, and resource availability, and automatically selects suitable circuit configurations to improve computational performance. Adaptive mechanisms can be employed to reduce unnecessary gates, optimize qubit utilization, and mitigate the impact of hardware noise while maintaining the required computational accuracy. The framework will be evaluated using simulation platforms and available NISQ quantum hardware, comparing adaptive circuits with conventional static circuits using metrics such as accuracy, fidelity, circuit depth, gate count, execution time, and error rate. The expected outcome is a resource-efficient and noise-aware quantum circuit optimization approach capable of improving the reliability and performance of quantum applications on near-term quantum hardware. The proposed research aims to contribute toward more practical, flexible, and hardware-aware quantum computing systems for NISQ-era applications. Keywords: NISQ Computing, Self-Adaptive Quantum Circuits, Quantum Circuit Optimization, Quantum Error Mitigation, Quantum Noise, Quantum Computing.
Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep-parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked-attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM’s state-of-the-art, high-fidelity Kingston QPU. Our noise intermediate-scale quantum (NISQ)-friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.
In mainstream machine learning, transformers are gaining widespread usage. As Vision Transformers rise in popularity in computer vision, they now aim to tackle a wide variety of machine learning applications. In particular, transformers for High Energy Physics (HEP) experiments continue to be investigated for tasks including jet tagging, particle reconstruction, and pile-up mitigation. An improved Quantum Vision Transformer (QViT) with a quantum-enhanced self-attention mechanism is introduced and discussed. A shallow circuit is proposed for each component of self-attention to leverage current Noisy Intermediate Scale Quantum (NISQ) devices. Variations of the hybrid architecture/model are explored and analyzed. The results demonstrate a successful proof of concept for the QViT, and establish a competitive performance benchmark for the proposed design and implementation. The findings also provide strong motivation to experiment with different architectures, hyperparameters, and datasets, setting the stage for implementation in HEP environments where transformers are increasingly used in state of the art machine learning solutions.