Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2610.12806

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

Deniz Yunus Göğüş

quant-phcs.LGSubmitted Oct 8, 2026• Announced Oct 12, 2026• 0 citations
arXivHTMLPDF

Abstract

We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.

AI Summary

The paper asks whether a small quantum register can serve as an autoregressive language model’s recurrent state, with learned token-conditioned channels and next-byte probabilities obtained directly from a POVM, without classical vocabulary logits. It compares unitary recurrence, learned token-independent dissipation, and token-conditioned dissipative channels, trained in simulation on byte-level *Alice’s Adventures in Wonderland*. Controls use identical parameterizations and counts but are retrained with one- or two-token memory. Unitary conjugation preserves trace distance and cannot reset or contract distinct states; experimentally, unitary recurrence loses to its one-token control in every tested variant. Token-conditioned dissipation improves test negative log-likelihood over one-token controls by $0.196\pm0.025$ nats/byte at context length 12 and $0.29$–$0.33$ at longer contexts, but does not reliably outperform two-token controls. A single-seed key–value benchmark crosses 50% answer accuracy for one stored pair at four qubits and two pairs at six qubits; three pairs remain unsolved through seven qubits. Accessible-information bounds are discussed as consistent with, not explanatory proof of, these limits. A secondary perturbed-coin experiment finds lower stationary quantum-memory entropy than a heuristically optimized classical frontier at matched prediction error. Compiled circuits on IBM Heron show computational-basis signal against a uniform null in 12/12 tests through 12-token prefixes; next-bit accuracy bars pass at four-token prefixes but not eight. Hardware-in-the-loop generation produces three bytes using measured probabilities. Limitations include few seeds, one corpus, small registers, heuristic classical entropy comparisons, and hardware validation without tomography. All models are classically simulable, remain worse than classical n-grams, and claim no computational advantage. Supporting code is private.

Little sign of AI-generated textIncrementalUnclear novelty

Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.

Comments
22 pages, 6 figures, 1 supplementary figure

Related papers

Coherent-feedback and dissipative training of quantum reservoir computers

Hikaru Wakaura, Taiki Tanimae

Abstract We introduce and test two measurement-free training primitives for quantum reservoir computers (QRC). Protocol~A encodes the loss gradient in an auxiliary phase that is fed back coherently before a dissipative reset; Protocol~B realizes gradient descent as relaxation of an engineered open system. Both match exact-gradient training in simulation---the ancilla and reset simulated as channels in a de-risk stage, their validated error model applied at the benchmark stage---on a minimal transverse-field Ising reservoir, with the coherent cost oracle emulated classically and its realization costed. A fully coherent update loop (quantum parameter register, unitary controlled update, no readout anywhere) descends the true landscape, its feedback drift attenuated by exactly $2F-1$ with the fixed point unchanged. Benchmarked against classical reservoir computing--- including the memory-augmented hybrid---they improve one-step chaotic prediction, though a thirty-seed study finds no autonomous-prediction advantage; we therefore make no quantum-advantage claim. The study instead addresses three questions that gate this route. First, within the channel model studied, the reset fidelity already demonstrated in measurement-free quantum error correction [Butt \textit{et al.}, Nat.\Commun.\\textbf{17}, 995 (2026)] suffices: reset is not the bottleneck. Second, the binding constraint is coherent cost-oracle precision, set by a finite-difference catastrophic cancellation that the exact parameter-shift rule removes, loosening the oracle-precision target $2.5$-fold in a like-for-like comparison. Third, a fault-tolerant realization---arithmetic and synthesis costs taken from real circuits---needs only $\sim\!10^{4}$ physical qubits and is runtime-, not qubit-bound. On three free IBM Quantum $156$-qubit Heron backends at three error-mitigation levels the simulation-optimized state survives device noise (RMSE $0.016$--$0.057$, i.e.\$7$--$26\%$ of the signal spread, correlation $\ge0.97$). All code, seeds, and job identifiers are released.

•Oct 6, 2026•0 citations

Quantum Generative Modeling with a Coherent Memory, Exact Training and a Memory-Dimension Advantage

Jamal Slim. Saverio Monaco, Ran Xue, Dirk Kruecker +1 more

We introduce the coherent memory Born machine (CoMB), an algorithm for quantum generative modeling on a register whose size does not depend on the dimension of the data. CoMB generates a sample one component per round. In each round, a work register is entangled with a memory register and measured, and the outcome gives one component. The memory is never measured and carries correlations between rounds. We show that the characteristic function of the generated distribution is a product of small matrices, one per round. This gives exact training, with maximum mean discrepancy losses and gradients costing time linear in the data dimension, without sampling or likelihood. We also compile every round onto one work qubit and evaluate device noise exactly. To isolate coherence, each CoMB is compared with its dephased twin and with equally trained classical hidden Markov models. We benchmark it on continuous data, $368$-cell calorimeter showers from five qubits, and on sequential data, handwritten digits from three qubits. The coherent memory provides a memory-dimension advantage. Four coherent levels do the work of about six classical states on both benchmarks, and of at least five with $95\%$ confidence on showers at every training budget, including convergence. On showers, removing their coherence raises the loss by $23$ to $31\%$. With eight levels, where all models approach a common floor, coherent and classical memories become indistinguishable with longer training. On an IBM Heron processor, the memory carries two thirds of the inter-round correlation through twelve mid-circuit measurements.

quant-pharXiv:2609.23050•Sep 22, 2026•1 citation

Controllable Quantum Memory Capacity in Quantum Reservoir Networks with Tunable partial-SWAPs

Erik L. Connerty, Ethan N. Evans

In the field of quantum reservoir computing (QRC), many different computational models and architectures have been proposed. From these models, we identify feedback-based models -- which use a feedback mechanism to re-embed classical measurements from the QRC -- and recurrent models -- which use a multi-register approach with memory and readout qubits -- as the two major competing architectures that have been discussed and validated on hardware. In this paper, we advance upon the recurrent architectures, which employ a two register approach to endow the QRC with a fading memory. While these approaches have been validated on hardware and have demonstrated great real-world performance on noisy-intermediate-scale-quantum (NISQ) quantum processing units (QPUs), the exact mechanism through which the memory capacity arises is not completely understood or fully controllable. With this, we augment the recurrent approaches and present a hardware-realizable mechanism, which we call a tunable partial-SWAP, that allows for the direct control of the rate of memory dissipation from a QRN implemented on a gate-based QPU. The theory behind this mechanism is discussed in terms of a controlled amplitude-damping channel and validation experiments using a randomized short-term memory capacity (STMC) recall benchmark and the NARMA-5 dataset are conducted using simulation and IBM QPUs, respectively.

quant-pharXiv:2605.12713•May 14, 2026•2 citations

Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models

Nathan Roll

Quantum language models have shown competitive performance on sequential tasks, yet whether trained quantum circuits exploit genuinely quantum resources -- or merely embed classical computation in quantum hardware -- remains unknown. Prior work has evaluated these models through endpoint metrics alone, without examining the memory strategies they actually learn internally. We introduce the first mechanistic interpretability study of quantum language models, combining causal gate ablation, entanglement tracking, and density-matrix interchange interventions on a controlled long-range dependency task. We find that single-qubit models are exactly classically simulable and converge to the same geometric strategy as matched classical baselines, while two-qubit models with entangling gates learn a representationally distinct strategy that encodes context in inter-qubit entanglement -- confirmed by three independent causal tests (p < 0.0001, d = 0.89). On real quantum hardware, only the classical geometric strategy survives device noise; the entanglement strategy degrades to chance. These findings open mechanistic interpretability as a tool for the science of quantum language models and reveal a noise-expressivity tradeoff governing which learned strategies survive deployment.

quant-pharXiv:2603.26494•Mar 30, 2026•1 citation

Hybrid Quantum-Classical Recurrent Neural Networks

Wenduan Xu

We present a hybrid quantum-classical recurrent neural network (QRNN) architecture in which the recurrent core is realized as a parametrized quantum circuit (PQC) controlled by a classical feedforward network. The hidden state is the quantum state of an $n$-qubit PQC in an exponentially large Hilbert space $\mathbb{C}^{2^n}$, which serves as a coherent recurrent quantum memory. The PQC is unitary by construction, making the hidden-state evolution norm-preserving without external constraints. At each timestep, mid-circuit Pauli expectation-value readouts are combined with the input embedding and processed by the feedforward network, which provides explicit classical nonlinearity. The outputs parametrize the PQC, which updates the hidden state via unitary dynamics. The QRNN is compact and physically consistent, and it unifies (i) unitary recurrence as a high-capacity memory, (ii) partial observation via mid-circuit readouts, and (iii) nonlinear classical control for input-conditioned parametrization. We evaluate the model in simulation with up to 14 qubits on sentiment analysis, MNIST, permuted MNIST, copying memory, and language modeling. For sequence-to-sequence learning, we further devise a soft attention mechanism over the mid-circuit readouts and show its effectiveness for machine translation. To our knowledge, this is the first model (RNN or otherwise) grounded in quantum operations to achieve competitive performance against strong classical baselines across a broad class of sequence-learning tasks.

arXiv:2510.25557•Oct 29, 2025•2 citations

Learnability and Complexity of Quantum Samples

Murphy Yuezhen Niu, Andrew M. Dai, Li Li +5 more

Given a quantum circuit, a quantum computer can sample the output distribution exponentially faster in the number of bits than classical computers. A similar exponential separation has yet to be established in generative models through quantum sample learning: given samples from an n-qubit computation, can we learn the underlying quantum distribution using models with training parameters that scale polynomial in n under a fixed training time? We study four kinds of generative models: Deep Boltzmann machine (DBM), Generative Adversarial Networks (GANs), Long Short-Term Memory (LSTM) and Autoregressive GAN, on learning quantum data set generated by deep random circuits. We demonstrate the leading performance of LSTM in learning quantum samples, and thus the autoregressive structure present in the underlying quantum distribution from random quantum circuits. Both numerical experiments and a theoretical proof in the case of the DBM show exponentially growing complexity of learning-agent parameters required for achieving a fixed accuracy as n increases. Finally, we establish a connection between learnability and the complexity of generative models by benchmarking learnability against different sets of samples drawn from probability distributions of variable degrees of complexities in their quantum and classical representations.

arXiv:2010.11983•Bulletin of the American Physical Society•Oct 22, 2020•13 citations
We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.
The paper asks whether a small quantum register can serve as an autoregressive language model’s recurrent state, with learned token-conditioned channels and next-byte probabilities obtained directly from a POVM, without classical vocabulary logits. It compares unitary recurrence, learned token-independent dissipation, and token-conditioned dissipative channels, trained in simulation on byte-level Alice’s Adventures in Wonderland. Controls use identical parameterizations and counts but are retrained with one- or two-token memory.
Unitary conjugation preserves trace distance and cannot reset or contract distinct states; experimentally, unitary recurrence loses to its one-token control in every tested variant. Token-conditioned dissipation improves test negative log-likelihood over one-token controls by 0.196±0.0250.196\pm0.0250.196±0.025 nats/byte at context length 12 and 0.290.290.29–0.330.330.33 at longer contexts, but does not reliably outperform two-token controls. A single-seed key–value benchmark crosses 50% answer accuracy for one stored pair at four qubits and two pairs at six qubits; three pairs remain unsolved through seven qubits. Accessible-information bounds are discussed as consistent with, not explanatory proof of, these limits.
A secondary perturbed-coin experiment finds lower stationary quantum-memory entropy than a heuristically optimized classical frontier at matched prediction error. Compiled circuits on IBM Heron show computational-basis signal against a uniform null in 12/12 tests through 12-token prefixes; next-bit accuracy bars pass at four-token prefixes but not eight. Hardware-in-the-loop generation produces three bytes using measured probabilities.
Limitations include few seeds, one corpus, small registers, heuristic classical entropy comparisons, and hardware validation without tomography. All models are classically simulable, remain worse than classical n-grams, and claim no computational advantage. Supporting code is private.
Abstract We introduce and test two measurement-free training primitives for quantum reservoir computers (QRC). Protocol~A encodes the loss gradient in an auxiliary phase that is fed back coherently before a dissipative reset; Protocol~B realizes gradient descent as relaxation of an engineered open system. Both match exact-gradient training in simulation---the ancilla and reset simulated as channels in a de-risk stage, their validated error model applied at the benchmark stage---on a minimal transverse-field Ising reservoir, with the coherent cost oracle emulated classically and its realization costed. A fully coherent update loop (quantum parameter register, unitary controlled update, no readout anywhere) descends the true landscape, its feedback drift attenuated by exactly 2F−12F-12F−1 with the fixed point unchanged. Benchmarked against classical reservoir computing--- including the memory-augmented hybrid---they improve one-step chaotic prediction, though a thirty-seed study finds no autonomous-prediction advantage; we therefore make no quantum-advantage claim. The study instead addresses three questions that gate this route. First, within the channel model studied, the reset fidelity already demonstrated in measurement-free quantum error correction [Butt \textit{et al.}, Nat.\Commun.\\textbf{17}, 995 (2026)] suffices: reset is not the bottleneck. Second, the binding constraint is coherent cost-oracle precision, set by a finite-difference catastrophic cancellation that the exact parameter-shift rule removes, loosening the oracle-precision target 2.52.52.5-fold in a like-for-like comparison. Third, a fault-tolerant realization---arithmetic and synthesis costs taken from real circuits---needs only ∼ ⁣104\sim\!10^{4}∼104 physical qubits and is runtime-, not qubit-bound. On three free IBM Quantum 156156156-qubit Heron backends at three error-mitigation levels the simulation-optimized state survives device noise (RMSE 0.0160.0160.016--0.0570.0570.057, i.e.$7$--26%26\%26% of the signal spread, correlation ≥0.97\ge0.97≥0.97). All code, seeds, and job identifiers are released.
We introduce the coherent memory Born machine (CoMB), an algorithm for quantum generative modeling on a register whose size does not depend on the dimension of the data. CoMB generates a sample one component per round. In each round, a work register is entangled with a memory register and measured, and the outcome gives one component. The memory is never measured and carries correlations between rounds. We show that the characteristic function of the generated distribution is a product of small matrices, one per round. This gives exact training, with maximum mean discrepancy losses and gradients costing time linear in the data dimension, without sampling or likelihood. We also compile every round onto one work qubit and evaluate device noise exactly. To isolate coherence, each CoMB is compared with its dephased twin and with equally trained classical hidden Markov models. We benchmark it on continuous data, 368368368-cell calorimeter showers from five qubits, and on sequential data, handwritten digits from three qubits. The coherent memory provides a memory-dimension advantage. Four coherent levels do the work of about six classical states on both benchmarks, and of at least five with 95%95\%95% confidence on showers at every training budget, including convergence. On showers, removing their coherence raises the loss by 232323 to 31%31\%31%. With eight levels, where all models approach a common floor, coherent and classical memories become indistinguishable with longer training. On an IBM Heron processor, the memory carries two thirds of the inter-round correlation through twelve mid-circuit measurements.
In the field of quantum reservoir computing (QRC), many different computational models and architectures have been proposed. From these models, we identify feedback-based models -- which use a feedback mechanism to re-embed classical measurements from the QRC -- and recurrent models -- which use a multi-register approach with memory and readout qubits -- as the two major competing architectures that have been discussed and validated on hardware. In this paper, we advance upon the recurrent architectures, which employ a two register approach to endow the QRC with a fading memory. While these approaches have been validated on hardware and have demonstrated great real-world performance on noisy-intermediate-scale-quantum (NISQ) quantum processing units (QPUs), the exact mechanism through which the memory capacity arises is not completely understood or fully controllable. With this, we augment the recurrent approaches and present a hardware-realizable mechanism, which we call a tunable partial-SWAP, that allows for the direct control of the rate of memory dissipation from a QRN implemented on a gate-based QPU. The theory behind this mechanism is discussed in terms of a controlled amplitude-damping channel and validation experiments using a randomized short-term memory capacity (STMC) recall benchmark and the NARMA-5 dataset are conducted using simulation and IBM QPUs, respectively.
Quantum language models have shown competitive performance on sequential tasks, yet whether trained quantum circuits exploit genuinely quantum resources -- or merely embed classical computation in quantum hardware -- remains unknown. Prior work has evaluated these models through endpoint metrics alone, without examining the memory strategies they actually learn internally. We introduce the first mechanistic interpretability study of quantum language models, combining causal gate ablation, entanglement tracking, and density-matrix interchange interventions on a controlled long-range dependency task. We find that single-qubit models are exactly classically simulable and converge to the same geometric strategy as matched classical baselines, while two-qubit models with entangling gates learn a representationally distinct strategy that encodes context in inter-qubit entanglement -- confirmed by three independent causal tests (p < 0.0001, d = 0.89). On real quantum hardware, only the classical geometric strategy survives device noise; the entanglement strategy degrades to chance. These findings open mechanistic interpretability as a tool for the science of quantum language models and reveal a noise-expressivity tradeoff governing which learned strategies survive deployment.
We present a hybrid quantum-classical recurrent neural network (QRNN) architecture in which the recurrent core is realized as a parametrized quantum circuit (PQC) controlled by a classical feedforward network. The hidden state is the quantum state of an nnn-qubit PQC in an exponentially large Hilbert space C2n\mathbb{C}^{2^n}C2n, which serves as a coherent recurrent quantum memory. The PQC is unitary by construction, making the hidden-state evolution norm-preserving without external constraints. At each timestep, mid-circuit Pauli expectation-value readouts are combined with the input embedding and processed by the feedforward network, which provides explicit classical nonlinearity. The outputs parametrize the PQC, which updates the hidden state via unitary dynamics. The QRNN is compact and physically consistent, and it unifies (i) unitary recurrence as a high-capacity memory, (ii) partial observation via mid-circuit readouts, and (iii) nonlinear classical control for input-conditioned parametrization. We evaluate the model in simulation with up to 14 qubits on sentiment analysis, MNIST, permuted MNIST, copying memory, and language modeling. For sequence-to-sequence learning, we further devise a soft attention mechanism over the mid-circuit readouts and show its effectiveness for machine translation. To our knowledge, this is the first model (RNN or otherwise) grounded in quantum operations to achieve competitive performance against strong classical baselines across a broad class of sequence-learning tasks.
Given a quantum circuit, a quantum computer can sample the output distribution exponentially faster in the number of bits than classical computers. A similar exponential separation has yet to be established in generative models through quantum sample learning: given samples from an n-qubit computation, can we learn the underlying quantum distribution using models with training parameters that scale polynomial in n under a fixed training time? We study four kinds of generative models: Deep Boltzmann machine (DBM), Generative Adversarial Networks (GANs), Long Short-Term Memory (LSTM) and Autoregressive GAN, on learning quantum data set generated by deep random circuits. We demonstrate the leading performance of LSTM in learning quantum samples, and thus the autoregressive structure present in the underlying quantum distribution from random quantum circuits. Both numerical experiments and a theoretical proof in the case of the DBM show exponentially growing complexity of learning-agent parameters required for achieving a fixed accuracy as n increases. Finally, we establish a connection between learnability and the complexity of generative models by benchmarking learnability against different sets of samples drawn from probability distributions of variable degrees of complexities in their quantum and classical representations.