Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2609.18565

Variational Quantum Transformer Architecture for Synthetic Language Generation

Julian Hager, Michael Kölle, Gerhard Stenzel, Tobias Rohe, Jonas Stein, Claudia Linnhoff-Popien

quant-phcs.CLcs.LGSubmitted Sep 16, 2026• Announced Sep 17, 2026• 0 citations
arXivHTMLPDF

Abstract

We propose a compact NISQ-compatible quantum transformer architecture for synthetic QNLP sequence modelling. The model preserves the autoregressive next-token interface of a classical transformer, but replaces attention and feed-forward sublayers with variational quantum encoder blocks, connector circuits, decoder blocks and a direct two-qubit measurement readout. Token contexts are angle-encoded into small quantum registers, processed by parallel variational heads and encoder integration circuits and conditioned through decoder ancillae to produce a distribution over a four-token vocabulary. We evaluate several architecture variants on deterministic and lexicographic grammar-generation tasks against a compact classical transformer baseline. The quantum models are trainable end-to-end and learn nontrivial grammar structure, including perfect deterministic generation in individual runs and high lexicographic validity in the strongest variant. The classical baseline remains more accurate and stable and the quantum models are sensitive to initialization. The contribution is therefore not a claim of quantum advantage, but a concrete architecture and evaluation of transformer-inspired QNLP sequence modelling under near-term quantum constraints.

AI Summary

Can a compact variational quantum circuit implement transformer-inspired autoregressive sequence modelling? This paper constructs an encoder–connector–decoder architecture that maps four-token contexts to next-token probabilities over $\{A,B,C,\texttt{space}\}$. Tokens are angle-encoded, parallel variational heads and integration circuits compress context into two-qubit states, and decoder circuits condition a direct two-qubit measurement readout. The architecture uses at most six active qubits and 124–364 trainable parameters; it does not implement scaled dot-product attention. Evaluation uses exact simulation on two synthetic grammars: the periodic cycle `AAA BBB CCC` and nondecreasing three-letter words. Models train on 25-word texts and generate 100 tokens greedily, with results averaged over three seeds. On the deterministic task, the highest mean quantum grammar score is $0.755\pm0.213$, with token accuracy $0.557\pm0.384$; individual runs achieve perfect continuation. On the lexicographic task, the strongest quantum variant reaches grammar score $0.828\pm0.299$ and token accuracy $0.380\pm0.156$. A 700-parameter classical transformer achieves perfect grammar scores on both tasks and deterministic token accuracy of 1. Quantum and classical training schedules differ substantially: 20 versus 800 epochs. The stated limitations include initialization-sensitive optimization, synthetic tasks that do not test semantics or natural-language generalization, and no shot-based or noisy-hardware evaluation. Lexicographic validity can reflect repetitive outputs rather than distribution learning, while the two-qubit readout restricts the vocabulary to four tokens. The results establish small-scale feasibility, not quantum advantage.

Little sign of AI-generated textAI use disclosedIncrementalSmall numerics onlyUnclear novelty
The authors used OpenAI ChatGPT for language editing.

Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.

Comments
Accepted for publication in the QNLPAI 2026 proceedings (Springer Lecture Notes in Computer Science, LNCS). 10 pages, including references and appendix, 2 figures

Related papers

FitQT: a fully integrated and trainable quantum transformer for scalable natural language processing

Lisa Pham, Anh The Le

Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.

Engineering Research Express•Mar 19, 2026•0 citations

Beyond Classical Positional Encodings: A Learnable QFT-Inspired Framework for Transformer Language Models

Sara Tehsin, Tallha Akram, Syed Rameez Naqvi +2 more

Transformers have become the dominant architecture for sequence modeling in natural language processing; however, their effectiveness critically depends on how positional information is encoded. Conventional positional encodings, while effective, may have limited structural flexibility for capturing complex global sequence relationships. Recent quantum-inspired approaches have sought to address this limitation, yet many either oversimplify quantum principles or introduce substantial computational or hardware overhead. We introduce a novel Quantum Fourier Transform (QFT)-inspired positional encoding scheme for transformers, motivated by the structured frequency representation of the QFT. Unlike prior approaches that either emulate quantum operations superficially or require complex circuit constructions, the proposed method provides a learnable hybrid encoding that preserves quantum-inspired structure while remaining aligned with hardware-efficient circuit primitives and structurally compatible with future near-term quantum implementations. Experiments on WikiText-103 indicate that the proposed encoding achieves competitive perplexity, improved robustness to input scrambling, and stable training behavior relative to alternative quantum-inspired baselines under the evaluated settings. Preliminary circuit-level simulations further suggest favorable noise resilience of the associated encoding primitives. These findings support the potential utility of incorporating quantum-inspired design principles into deep learning architectures and provide a foundation for future exploration at the interface of quantum computing and transformer-based natural language processing (NLP).

Computers, materials & continua/Computers, materials & continua (Print)•Jan 1, 2026•0 citations

Foundations for Near-Term Quantum Natural Language Processing

Bob Coecke, Giovanni de Felice, Konstantinos Meichanetzidis +1 more

We provide conceptual and mathematical foundations for near-term quantum natural language processing (QNLP), and do so in quantum computer scientist friendly terms. We opted for an expository presentation style, and provide references for supporting empirical evidence and formal statements concerning mathematical generality. We recall how the quantum model for natural language that we employ canonically combines linguistic meanings with rich linguistic structure, most notably grammar. In particular, the fact that it takes a quantum-like model to combine meaning and structure, establishes QNLP as quantum-native, on par with simulation of quantum systems. Moreover, the now leading Noisy Intermediate-Scale Quantum (NISQ) paradigm for encoding classical data on quantum hardware, variational quantum circuits, makes NISQ exceptionally QNLP-friendly: linguistic structure can be encoded as a free lunch, in contrast to the apparently exponentially expensive classical encoding of grammar. Quantum speed-up for QNLP tasks has already been established in previous work with Will Zeng. Here we provide a broader range of tasks which all enjoy the same advantage. Diagrammatic reasoning is at the heart of QNLP. Firstly, the quantum model interprets language as quantum processes via the diagrammatic formalism of categorical quantum mechanics. Secondly, these diagrams are via ZX-calculus translated into quantum circuits. Parameterisations of meanings then become the circuit variables to be learned. Our encoding of linguistic structure within quantum circuits also embodies a novel approach for establishing word-meanings that goes beyond the current standards in mainstream AI, by placing linguistic structure at the heart of Wittgenstein's meaning-is-context.

arXiv:2012.03755•Dec 7, 2020•144 citations

Practical Hybrid Quantum Language Models with Observable Readout on Real Hardware

Stefan Balauca, Ada-Astrid Balauca, Adrian Iftene

Hybrid quantum-classical models represent a crucial step toward leveraging near-term quantum devices for sequential data processing. We present Quantum Recurrent Neural Networks (QRNNs) and Quantum Convolutional Neural Networks (QCNNs) as hybrid quantum language models, reporting the first empirical demonstration of generative language modeling trained and evaluated end-to-end on real quantum hardware. Our architecture combines hardware-optimized parametric quantum circuits with a lightweight classical projection layer, utilizing a multi-sample SPSA strategy to efficiently train quantum parameters despite hardware noise. To characterize the capabilities of these models, we introduce a synthetic dataset designed to isolate syntactic dependencies in a controlled, low-resource environment. Experiments on IBM Quantum processors reveal the critical trade-offs between circuit depth and trainability, demonstrating that while noise remains a significant factor, observable-based readout enables the successful learning of sequential patterns on NISQ devices. These results establish a rigorous engineering baseline for generative quantum natural language processing, validating the feasibility of training complex sequence models on current quantum hardware.

quant-pharXiv:2512.12710•Dec 16, 2025•2 citations

QuantumGPTMini: A Hybrid Quantum-Classical Transformer for Enhanced NLP

Ramazan Amire, Amire Ramazan

Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.

•Apr 28, 2025•0 citations

Quantum Natural Language Processing Using Hybrid AI Algorithms: A Variational Quantum–Classical Framework for Efficient Text Classification

P. Anitha

Quantum Natural Language Processing (QNLP) represents a novel interdisciplinaryparadigm that integrates Quantum Computing with Natural Language Processing toenhance semantic modeling and computational efficiency. Despite the remarkable successof transformer-based architectures such as BERT and GPT, classical NLP systems requirelarge-scale computational resources and extensive parameter tuning. This paper proposesa hybrid quantum–classical architecture employing parameterized quantum circuits (PQCs)integrated with classical embedding layers for text classification tasks. The frameworkutilizes classical preprocessing, quantum state encoding, variational optimization, andclassical post-measurement classification. Experimental results on benchmark sentimentanalysis datasets demonstrate competitive accuracy with significantly reduced parametercomplexity. The findings indicate that hybrid QNLP models provide enhanced semanticcompositionality and efficient representation in Hilbert space while remaining feasiblewithin Noisy Intermediate-Scale Quantum (NISQ) constraints

International Journal For Multidisciplinary Research•Mar 4, 2026•0 citations
We propose a compact NISQ-compatible quantum transformer architecture for synthetic QNLP sequence modelling. The model preserves the autoregressive next-token interface of a classical transformer, but replaces attention and feed-forward sublayers with variational quantum encoder blocks, connector circuits, decoder blocks and a direct two-qubit measurement readout. Token contexts are angle-encoded into small quantum registers, processed by parallel variational heads and encoder integration circuits and conditioned through decoder ancillae to produce a distribution over a four-token vocabulary. We evaluate several architecture variants on deterministic and lexicographic grammar-generation tasks against a compact classical transformer baseline. The quantum models are trainable end-to-end and learn nontrivial grammar structure, including perfect deterministic generation in individual runs and high lexicographic validity in the strongest variant. The classical baseline remains more accurate and stable and the quantum models are sensitive to initialization. The contribution is therefore not a claim of quantum advantage, but a concrete architecture and evaluation of transformer-inspired QNLP sequence modelling under near-term quantum constraints.
Can a compact variational quantum circuit implement transformer-inspired autoregressive sequence modelling? This paper constructs an encoder–connector–decoder architecture that maps four-token contexts to next-token probabilities over {A,B,C,space}\{A,B,C,\texttt{space}\}{A,B,C,space}. Tokens are angle-encoded, parallel variational heads and integration circuits compress context into two-qubit states, and decoder circuits condition a direct two-qubit measurement readout. The architecture uses at most six active qubits and 124–364 trainable parameters; it does not implement scaled dot-product attention.
Evaluation uses exact simulation on two synthetic grammars: the periodic cycle AAA BBB CCC and nondecreasing three-letter words. Models train on 25-word texts and generate 100 tokens greedily, with results averaged over three seeds. On the deterministic task, the highest mean quantum grammar score is 0.755±0.2130.755\pm0.2130.755±0.213, with token accuracy 0.557±0.3840.557\pm0.3840.557±0.384; individual runs achieve perfect continuation. On the lexicographic task, the strongest quantum variant reaches grammar score 0.828±0.2990.828\pm0.2990.828±0.299 and token accuracy 0.380±0.1560.380\pm0.1560.380±0.156. A 700-parameter classical transformer achieves perfect grammar scores on both tasks and deterministic token accuracy of 1. Quantum and classical training schedules differ substantially: 20 versus 800 epochs.
The stated limitations include initialization-sensitive optimization, synthetic tasks that do not test semantics or natural-language generalization, and no shot-based or noisy-hardware evaluation. Lexicographic validity can reflect repetitive outputs rather than distribution learning, while the two-qubit readout restricts the vocabulary to four tokens. The results establish small-scale feasibility, not quantum advantage.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
Transformers have become the dominant architecture for sequence modeling in natural language processing; however, their effectiveness critically depends on how positional information is encoded. Conventional positional encodings, while effective, may have limited structural flexibility for capturing complex global sequence relationships. Recent quantum-inspired approaches have sought to address this limitation, yet many either oversimplify quantum principles or introduce substantial computational or hardware overhead. We introduce a novel Quantum Fourier Transform (QFT)-inspired positional encoding scheme for transformers, motivated by the structured frequency representation of the QFT. Unlike prior approaches that either emulate quantum operations superficially or require complex circuit constructions, the proposed method provides a learnable hybrid encoding that preserves quantum-inspired structure while remaining aligned with hardware-efficient circuit primitives and structurally compatible with future near-term quantum implementations. Experiments on WikiText-103 indicate that the proposed encoding achieves competitive perplexity, improved robustness to input scrambling, and stable training behavior relative to alternative quantum-inspired baselines under the evaluated settings. Preliminary circuit-level simulations further suggest favorable noise resilience of the associated encoding primitives. These findings support the potential utility of incorporating quantum-inspired design principles into deep learning architectures and provide a foundation for future exploration at the interface of quantum computing and transformer-based natural language processing (NLP).
We provide conceptual and mathematical foundations for near-term quantum natural language processing (QNLP), and do so in quantum computer scientist friendly terms. We opted for an expository presentation style, and provide references for supporting empirical evidence and formal statements concerning mathematical generality. We recall how the quantum model for natural language that we employ canonically combines linguistic meanings with rich linguistic structure, most notably grammar. In particular, the fact that it takes a quantum-like model to combine meaning and structure, establishes QNLP as quantum-native, on par with simulation of quantum systems. Moreover, the now leading Noisy Intermediate-Scale Quantum (NISQ) paradigm for encoding classical data on quantum hardware, variational quantum circuits, makes NISQ exceptionally QNLP-friendly: linguistic structure can be encoded as a free lunch, in contrast to the apparently exponentially expensive classical encoding of grammar. Quantum speed-up for QNLP tasks has already been established in previous work with Will Zeng. Here we provide a broader range of tasks which all enjoy the same advantage. Diagrammatic reasoning is at the heart of QNLP. Firstly, the quantum model interprets language as quantum processes via the diagrammatic formalism of categorical quantum mechanics. Secondly, these diagrams are via ZX-calculus translated into quantum circuits. Parameterisations of meanings then become the circuit variables to be learned. Our encoding of linguistic structure within quantum circuits also embodies a novel approach for establishing word-meanings that goes beyond the current standards in mainstream AI, by placing linguistic structure at the heart of Wittgenstein's meaning-is-context.
Hybrid quantum-classical models represent a crucial step toward leveraging near-term quantum devices for sequential data processing. We present Quantum Recurrent Neural Networks (QRNNs) and Quantum Convolutional Neural Networks (QCNNs) as hybrid quantum language models, reporting the first empirical demonstration of generative language modeling trained and evaluated end-to-end on real quantum hardware. Our architecture combines hardware-optimized parametric quantum circuits with a lightweight classical projection layer, utilizing a multi-sample SPSA strategy to efficiently train quantum parameters despite hardware noise. To characterize the capabilities of these models, we introduce a synthetic dataset designed to isolate syntactic dependencies in a controlled, low-resource environment. Experiments on IBM Quantum processors reveal the critical trade-offs between circuit depth and trainability, demonstrating that while noise remains a significant factor, observable-based readout enables the successful learning of sequential patterns on NISQ devices. These results establish a rigorous engineering baseline for generative quantum natural language processing, validating the feasibility of training complex sequence models on current quantum hardware.
Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.
Quantum Natural Language Processing (QNLP) represents a novel interdisciplinaryparadigm that integrates Quantum Computing with Natural Language Processing toenhance semantic modeling and computational efficiency. Despite the remarkable successof transformer-based architectures such as BERT and GPT, classical NLP systems requirelarge-scale computational resources and extensive parameter tuning. This paper proposesa hybrid quantum–classical architecture employing parameterized quantum circuits (PQCs)integrated with classical embedding layers for text classification tasks. The frameworkutilizes classical preprocessing, quantum state encoding, variational optimization, andclassical post-measurement classification. Experimental results on benchmark sentimentanalysis datasets demonstrate competitive accuracy with significantly reduced parametercomplexity. The findings indicate that hybrid QNLP models provide enhanced semanticcompositionality and efficient representation in Hilbert space while remaining feasiblewithin Noisy Intermediate-Scale Quantum (NISQ) constraints