arXiv:2610.12806
Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation
Deniz Yunus Göğüş
Abstract
We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.
AI Summary
The paper asks whether a small quantum register can serve as an autoregressive language model’s recurrent state, with learned token-conditioned channels and next-byte probabilities obtained directly from a POVM, without classical vocabulary logits. It compares unitary recurrence, learned token-independent dissipation, and token-conditioned dissipative channels, trained in simulation on byte-level *Alice’s Adventures in Wonderland*. Controls use identical parameterizations and counts but are retrained with one- or two-token memory. Unitary conjugation preserves trace distance and cannot reset or contract distinct states; experimentally, unitary recurrence loses to its one-token control in every tested variant. Token-conditioned dissipation improves test negative log-likelihood over one-token controls by $0.196\pm0.025$ nats/byte at context length 12 and $0.29$–$0.33$ at longer contexts, but does not reliably outperform two-token controls. A single-seed key–value benchmark crosses 50% answer accuracy for one stored pair at four qubits and two pairs at six qubits; three pairs remain unsolved through seven qubits. Accessible-information bounds are discussed as consistent with, not explanatory proof of, these limits. A secondary perturbed-coin experiment finds lower stationary quantum-memory entropy than a heuristically optimized classical frontier at matched prediction error. Compiled circuits on IBM Heron show computational-basis signal against a uniform null in 12/12 tests through 12-token prefixes; next-bit accuracy bars pass at four-token prefixes but not eight. Hardware-in-the-loop generation produces three bytes using measured probabilities. Limitations include few seeds, one corpus, small registers, heuristic classical entropy comparisons, and hardware validation without tomography. All models are classically simulable, remain worse than classical n-grams, and claim no computational advantage. Supporting code is private.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
- Comments
- 22 pages, 6 figures, 1 supplementary figure