The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of $π$/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is $ε$-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of π/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is ε-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
When a language model answers, we can usually only estimate why. We build a transformer in which every nonlinearity is a quadratic detection - an interference, a balanced reading or a camera - and the output is a Born probability: attention ||^2, the form of the Born rule, normalised per row, a measured product for the feed-forward nonlinearity, and a Born readout; between detections, classical control rescales and re-encodes. With its denominators held at the input, each class probability is a polynomial in the word amplitudes, so its integrated-gradient evidence is computed by a finite quadrature with no quadrature error: the words' contributions sum to the class probability above its no-information share to 10^-13. On e-SNLI natural language inference it reaches 77.45 +- 0.91 % accuracy, level with a same-size softmax transformer (76.62 +- 0.36 %) and an order-free bag of embeddings (77.79 %). Exactness buys the sum, not the best ranking. Deleting the word its evidence ranks first changes the answer in 44 % of pairs, of the 53 % some deletion changes, against 31-32 % for the softmax model's integrated gradients; the model's own gradient x input does as well (43 %), as does a bag of embeddings' (43 %, seed 0), and leave-one-out does better. Holding the denominators helps the low end: the five words ranked last change the answer in 1 %, against 6 % free. The same weights compile exactly to simulated optoelectronics, and their attention to qubits. On ibm_kingston, reading only the attention, the device gave the model's answers in all three Hadamard-test runs (the two shortest pairs and one without its deciding word) and both compute-uncompute runs (one circuit per overlap); uniform attention answers otherwise on two of the three inputs. With the device's row sums the evidence correlated with the exact one at 0.997 and 0.962 by Hadamard tests; with compute-uncompute, 0.971 and 0.48.
We derive two attention operators from generalized statistical entropies. Kaniadakis entropy yields an exact full-support normalization whose weights and low-score sensitivities decay algebraically, rather than exponentially as in Softmax or by exact truncation as in entmax. Classical Abe entropy yields an implicit reciprocal-symmetric operator. With q=eε, the involution q↔q−1 removes every odd correction about Softmax; we obtain the normalized second- and fourth-order terms, including the deformation of the normalization multiplier. These stationary laws follow from a Fisher-metric Lagrangian on the probability simplex, whose Shannon sector recovers scaled dot-product Softmax. We also give a tangent-gradient test for deciding whether changing the entropy changes the attention profile or only its scale. Rényi and two-parameter Sharma--Mittal entropies retain the Tsallis--entmax inverse-gradient shape, but their global moments make the effective temperature input dependent when the external temperature is fixed. Distinguishing profile-shape equivalence from fixed-parameter operator equivalence separates new normalization shapes from adaptive rescalings and organizes the operators by support, tail behavior, and realization complexity.
Given the query, key and value matrices Q,K,V∈Rn×d, the attention module is defined as Att(Q,K,V)=D−1AV where A=exp(QK⊤/d) with exp(⋅) applied entrywise, D=diag(A1n). The attention module is the backbone of modern transformers and large language models, but explicitly forming the softmax matrix D−1A incurs Ω(n2) time, motivating numerous approximation schemes that reduce runtime to O(nd) via sparsity or low-rank factorization.
We propose a quantum data structure that approximates any row of Att(Q,K,V) using only row queries to Q,K,V. Our algorithm preprocesses these matrices in O(ε−1n0.5(sλ2.5+sλ1.5d+α0.5d)) time, where ε is the target accuracy, sλ is the λ-statistical dimension of the exponential kernel defined by Q and K, and α measures the row distortion of V that is at most d/srank(V), the stable rank of V. Each row query can be answered in O(sλ2+sλd) time.
To our knowledge, this is the first quantum data structure that approximates rows of the attention matrix in sublinear time with respect to n. Our approach relies on a quantum Nyström approximation of the exponential kernel, quantum multivariate mean estimation for computing D, and quantum leverage score sampling for the multiplication with V.
Abstract An established exponential Bernoulli race samples exact softmax labels from residual quantum overlaps under an ideal independent-measurement model. We derive its early-stopped shot cost and prove that nested classical projections monotonically reduce expected costs and computable bounds. A first-proposal coupling gives an unbiased value estimator, with a coefficient computed from score envelopes that guarantees no variance increase over ordinary averaging. Tests use 192 pretrained-model attention rows with contexts up to 512 tokens. Retaining 32 of 64 coordinates reduces the primary median expected shots per label from 1.72 × 10 5 to 2.81. No hardware speedup is established.
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.
Abstract We formalize Quantum Hybrid Modules (QHM) to evaluate the insertion of quantum subroutines into classical AI pipelines with explicit assumptions about data loading, noise, and measurement overhead. The framework comprises three components: (1) a Hadamard Quantum Self-Attention (HQSA) module computing sparse attention via Hadamard-test inner products in O(nn/k⋅logd/ϵ) time assuming quantum random access memory (qRAM); (2) a Quantum Approximate Optimization Algorithm (QAOA) module for combinatorial optimization sub-problems; and (3) a Grover-accelerated neurosymbolic verification loop for quadratically faster constraint checking. We evaluate QHM through 22 experiments spanning statevector and GPU simulation up to 30 qubits, cloud simulators, and five quantum processing unit (QPU) experiments on Rigetti Cepheus and IonQ Forte, including 8--100-qubit superconducting and 8--30-qubit trapped-ion QAOA. Simulations confirm the intended mechanisms: QAOA exceeds the Goemans--Williamson bound on selected 3-regular instances, Hadamard-test attention reproduces classical softmax, and Grover verification reaches a 20,862× query reduction at 30 qubits. Classical, dequantized, and qRAM-free baselines expose current limits: small quantum kernels lag tuned classical baselines, hybrid variational quantum circuit layers add training cost without consistent accuracy gains, and shot-based noisy intermediate-scale quantum (NISQ) attention is limited by systematic bias. Hardware results are depth- and platform-dependent: shallow circuits retain measurable signal, Rigetti QAOA depolarizes to the random baseline and is not recovered by post-hoc mitigation, and IonQ Forte preserves QAOA signal but remains below noiseless performance. We do not claim near-term quantum advantage; we characterize the gap between asymptotic theory and present-day hardware.