Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the $\mathcal{O}(N^2D)$ attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3\% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.
AI Summary
QiT asks whether periodic encodings and multiplicative interactions motivated by quantum models can serve as inductive biases in a scalable classical vision Transformer, without quantum circuits or measurements.
The architecture maps tokens to learned angles $\theta(h)=\pi\tanh(Wh+b)$ and sine–cosine features, applies standard softmax attention to learned projections of these features, and replaces the feed-forward block with gated multiplicative channel mixing. The unprojected features induce a positive-semidefinite average-cosine kernel. Independently learned query and key projections generally remove this kernel interpretation; the scores are not quantum fidelities, and the mixer does not represent physical entanglement. Per-block classical cost is $\mathcal{O}(BN^2d+BNQd+BNdr)$, retaining quadratic token-count dependence.
Evaluation comprises a one-epoch CIFAR simulator diagnostic, training across eight image datasets, and ImageNet-1K experiments. In the diagnostic, QiT is $882$–$997\times$ faster than the tested four-qubit PennyLane Transformer, but $1.58$–$1.76\times$ slower than the small classical baseline. One fixed configuration trains across eight datasets; QiT-B reaches 78.3% ImageNet top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. Published-model comparisons use different training recipes.
The paper states that these results establish execution feasibility, not quantum advantage or superiority over a converged, parameter-matched Transformer. The simulator models have unmatched parameter counts, and the classical baseline remains at chance after one epoch. Component ablations, repeated seeds, uncertainty intervals, complete optimizer settings, reduced-class split definitions, and an energy-measurement procedure are absent; simulator timings cannot be generalized to other simulators or quantum hardware.
Central claim may not hold: The abstract claims competitiveness with a matched classical Transformer, but no converged, parameter-matched comparison is reported, and the paper explicitly acknowledges that this conclusion is not established.
The abstract presents QiT's cosine kernel as approximating quantum fidelity kernels, although the derivations distinguish these functions and establish no approximation relationship.
The unified complexity table incorrectly gives QiT's dense gated mixer an O(QL) cost instead of O(NDr) and omits projection and mixer terms from total complexity without dominance assumptions.
The unified comparison table's runtime slowdowns contradict the recorded wall-clock measurements, which show roughly 1.58–1.76× for QiT and 1396–1758× for PennyLane relative to CViT.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the O(N2D) attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.
QiT asks whether periodic encodings and multiplicative interactions motivated by quantum models can serve as inductive biases in a scalable classical vision Transformer, without quantum circuits or measurements.
The architecture maps tokens to learned angles θ(h)=πtanh(Wh+b) and sine–cosine features, applies standard softmax attention to learned projections of these features, and replaces the feed-forward block with gated multiplicative channel mixing. The unprojected features induce a positive-semidefinite average-cosine kernel. Independently learned query and key projections generally remove this kernel interpretation; the scores are not quantum fidelities, and the mixer does not represent physical entanglement. Per-block classical cost is O(BN2d+BNQd+BNdr), retaining quadratic token-count dependence.
Evaluation comprises a one-epoch CIFAR simulator diagnostic, training across eight image datasets, and ImageNet-1K experiments. In the diagnostic, QiT is 882–997× faster than the tested four-qubit PennyLane Transformer, but 1.58–1.76× slower than the small classical baseline. One fixed configuration trains across eight datasets; QiT-B reaches 78.3% ImageNet top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. Published-model comparisons use different training recipes.
The paper states that these results establish execution feasibility, not quantum advantage or superiority over a converged, parameter-matched Transformer. The simulator models have unmatched parameter counts, and the classical baseline remains at chance after one epoch. Component ablations, repeated seeds, uncertainty intervals, complete optimizer settings, reduced-class split definitions, and an energy-measurement procedure are absent; simulator timings cannot be generalized to other simulators or quantum hardware.
Classical Vision Transformers (CViTs) have demonstrated strong performance in medical image analysis, supporting a wide range of digital health applications. However, their self-attention mechanisms can require substantial computational resources and model parameters. In parallel, quantum machine learning (QML) has attracted increasing interest in healthcare and other scientific domains, particularly because quantum-enhanced models may provide expressive representations for small datasets. In this paper, we propose MedQViT, a robust and parameter-efficient Quantum Vision Transformer for medical image classification. To reduce the computational cost of global self-attention, MedQViT combines a shared four-qubit parameterized quantum circuit with a quantum-feature state memory and trainable landmark tokens. This hybrid architecture encodes image patches into compact quantum-enhanced representations while using classical layers for feature aggregation and classification. Experiments on established medical image datasets evaluate the effectiveness and feasibility of MedQViT against classical and quantum baselines. MedQViT achieves competitive performance with substantially fewer parameters than conventional vision models and outperforms several classical and quantum methods in terms of test accuracy (ACC) and area under the receiver operating characteristic curve (AUC).
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.
Abstract We introduce Quantum Kernel Transformer (QKT), a hybrid quantumclassical architecture that inserts a trainable parameterized quantum circuit between a classical backbone and an attention-based classifier head. Backbone features are projected to a small qubit input, encoded by a quantum feature map, transformed by a shallow variational ansatz, and decoded by attention over the resulting basis-state probability distribution. This keeps highdimensional representation learning on the classical side while using the quantum neural network as a non-linear embedding whose qubit count is independent of the raw input dimension. We evaluate QKT on five classification benchmarks spanning text (AG News), tabular biomedical data (Breast Cancer Wisconsin), bioacoustic spectrograms (BirdCLEF), images (CIFAR-10), and high-energy physics (SUSY). Against classical, classical-attention, and linear hybrid-QNN baselines, QKT is competitive on AG News and BCW, obtains the highest mean accuracy on BirdCLEF, and reduces the severe instability observed for the linear hybrid-QNN baseline on SUSY. It underperforms classical baselines on CIFAR-10, identifying an important failure mode for image-domain features. The evidence does not establish a universal quantum advantage; instead, it identifies where a trainable quantum-kernel bottleneck with distribution attention is robust under a reproducible workstation-scale protocol. Code, configuration files, generated tables, figures, and LaTeX sources are released to support independent reproduction.
The integration of quantum computing principles with classical deep learning architectures has emerged as a promising direction for advancing image recognition systems beyond the limitations of purely classical convolutional neural networks (CNNs). This study proposes and analyzes a hybrid quantum–classical deep learning framework based on quanvolutional neural network (QNN) principles, where quantum circuits are embedded within convolutional feature extraction layers to enhance representation capability. The research investigates how quantum-enhanced filters, particularly quanvolutional layers, contribute to improved feature diversity, non-linear transformation capacity, and classification accuracy in image recognition tasks. Building upon established convolutional architectures and quantum machine learning theories, the proposed framework integrates quantum random circuit encoding with classical deep neural networks to evaluate performance gains in comparison to conventional CNN models. The study further incorporates theoretical insights from hybrid quantum-classical learning systems, emphasizing the role of parameterized quantum circuits in feature mapping and optimization stability. The methodology is grounded in NISQ-era quantum computing constraints and leverages hybrid optimization strategies to ensure computational feasibility. Experimental evaluation demonstrates that quanvolution-based models can provide improved robustness in feature extraction, particularly in high-dimensional image spaces, while maintaining competitive computational efficiency. However, scalability limitations and noise sensitivity in quantum circuits remain key challenges. Overall, this work contributes to the growing field of quantum-enhanced machine learning by providing a structured performance analysis of hybrid quanvolutional architectures and highlighting their potential role in next-generation intelligent visual recognition systems.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
This working paper introduces the Quantum Interference Transformer (QIT), a sequence model in which token relationships are resolved by constructive and destructive interference in a parameterised quantum circuit, rather than by classical dot-product attention. We present QIT-0, a minimal PennyLane/PyTorch prototype. Tokens are encoded as rotation angles, cross-token correlations are seeded by a ring CNOT ladder, a learned strongly-entangling unitary acts as the attention operator, and Pauli-Z expectation values are decoded by a linear layer. On a 4-bit parity memorisation benchmark (16 inputs; 5 random seeds), QIT-0 (78 parameters) reaches 100% accuracy in 6.2 ± 1.7 epochs. Classical baselines with more parameters converge more slowly or fail within the epoch budget. Additional experiments include ablations, a first-token negative control, and related structured tasks. This is an independent technical report, not a peer-reviewed publication. We do not claim general quantum advantage for machine learning. Code: https://github.com/MihailStancescu/qit