Transformers excel in various domains but often demand massive computational resources. Quantum Neural Networks (QNNs) combine classical and quantum computing to mitigate these costs. However, QNNs typically fix the number of qubits, forcing complete retraining whenever the qubit count changes and leading to substantial inefficiency. We propose an Adaptive Quantum Transformer (AQT) to resolve this limitation by introducing an adapter module — originally designed for large language models (LLMs) — into a quantum vision Transformer. This module reuses parameters when the qubit count expands or contracts, preserving learned states. We validate AQT in an autoscaling cloud environment, demonstrating dynamic qubit scaling without sacrificing accuracy or retraining from scratch. Results on vision tasks reveal that AQT achieves higher accuracy than a conventional quantum vision Transformer while converging faster and requiring fewer computational resources. Notably, AQT provides over triple the efficiency in training time and better stability under fluctuating qubit availability, suggesting a promising avenue for practical, cost-effective quantum–classical hybrid intelligence.
Transformers excel in various domains but often demand massive computational resources. Quantum Neural Networks (QNNs) combine classical and quantum computing to mitigate these costs. However, QNNs typically fix the number of qubits, forcing complete retraining whenever the qubit count changes and leading to substantial inefficiency. We propose an Adaptive Quantum Transformer (AQT) to resolve this limitation by introducing an adapter module — originally designed for large language models (LLMs) — into a quantum vision Transformer. This module reuses parameters when the qubit count expands or contracts, preserving learned states. We validate AQT in an autoscaling cloud environment, demonstrating dynamic qubit scaling without sacrificing accuracy or retraining from scratch. Results on vision tasks reveal that AQT achieves higher accuracy than a conventional quantum vision Transformer while converging faster and requiring fewer computational resources. Notably, AQT provides over triple the efficiency in training time and better stability under fluctuating qubit availability, suggesting a promising avenue for practical, cost-effective quantum–classical hybrid intelligence.
Quantum machine learning (QML) is a promising application of quantum computing, yet when and how practical learning models can achieve quantum advantage remains poorly understood. Even a polynomial improvement in the resources needed to attain a given predictive performance would be significant. In this work, we investigate empirical scaling laws of quantum vision transformers (QViTs), which connect QML to the transformer architecture underlying modern large language models. We ask whether QViTs exhibit scaling behavior analogous to classical models and how this behavior changes when qubit number is included alongside dataset size and model parameters. The validation losses in our numerical image-classification experiments are well described by a joint shifted power-law model in qubit number, transformer depth, and training dataset size. Expressing this fit in terms of the classical encoder parameter budget identifies a regime in which increasing the number of qubits can compensate for fewer classical encoder parameters at comparable predicted performance. This finding is consistent with earlier observations of parameter efficiency in quantum learning models and suggests a direction for exploring possible resource savings in QML.
In this work, quantum transformers are designed and analysed in detail by extending the state-of-the-art classical transformer neural network architectures known to be very performant in natural language processing and image analysis. Building upon the previous work, which uses parametrised quantum circuits for data loading and orthogonal neural layers, we introduce three types of quantum transformers for training and inference, including a quantum transformer based on compound matrices, which guarantees a theoretical advantage of the quantum attention mechanism compared to their classical counterpart both in terms of asymptotic run time and the number of model parameters. These quantum architectures can be built using shallow quantum circuits and produce qualitatively different classification models. The three proposed quantum attention layers vary on the spectrum between closely following the classical transformers and exhibiting more quantum characteristics. As building blocks of the quantum transformer, we propose a novel method for loading a matrix as quantum states as well as two new trainable quantum orthogonal layers adaptable to different levels of connectivity and quality of quantum computers. We performed extensive simulations of the quantum transformers on standard medical image datasets that showed competitively, and at times better performance compared to the classical benchmarks, including the best-in-class classical vision transformers. The quantum transformers we trained on these small-scale datasets require fewer parameters compared to standard classical benchmarks. Finally, we implemented our quantum transformers on superconducting quantum computers and obtained encouraging results for up to six qubit experiments.
Classical Vision Transformers (CViTs) have demonstrated strong performance in medical image analysis, supporting a wide range of digital health applications. However, their self-attention mechanisms can require substantial computational resources and model parameters. In parallel, quantum machine learning (QML) has attracted increasing interest in healthcare and other scientific domains, particularly because quantum-enhanced models may provide expressive representations for small datasets. In this paper, we propose MedQViT, a robust and parameter-efficient Quantum Vision Transformer for medical image classification. To reduce the computational cost of global self-attention, MedQViT combines a shared four-qubit parameterized quantum circuit with a quantum-feature state memory and trainable landmark tokens. This hybrid architecture encodes image patches into compact quantum-enhanced representations while using classical layers for feature aggregation and classification. Experiments on established medical image datasets evaluate the effectiveness and feasibility of MedQViT against classical and quantum baselines. MedQViT achieves competitive performance with substantially fewer parameters than conventional vision models and outperforms several classical and quantum methods in terms of test accuracy (ACC) and area under the receiver operating characteristic curve (AUC).
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%, and to 16.84% for Gemma-2, with a marginal performance improvement of 0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
The remarkable success of Transformer architectures in Large Language Models (LLMs) has revolutionized natural language processing, yet the transition to quantum computing for next-generation language models remains an open challenge. While quantum computing promises exponential advantages, a fundamental gap exists between classical deep learning and quantum computing paradigms, particularly given the severe constraints of Noisy Intermediate-Scale Quantum (NISQ) devices, including barren plateaus, limited qubit coherence, and circuit depth restrictions. We present Adaptive Quantum-Classical Fusion (AQCF), the first framework to bridge classical and quantum computing for language models by reimagining Transformer architectures through quantum-classical co-design. Our key insight is that effective bridging requires dynamic adaptation rather than static translation—the framework analyzes input complexity in real-time to orchestrate seamless transitions between classical and quantum processing. AQCF introduces entropy-driven adaptive circuits that circumvent barren plateaus, quantum memory banks that unify classical attention with quantum state-based similarity retrieval, and intelligent fusion controllers that ensure each computational paradigm handles tasks where it naturally excels. This bridging architecture maintains full compatibility with existing classical Transformers while progressively incorporating quantum advantages as they become accessible. Experiments on sentiment analysis demonstrate that AQCF achieves competitive performance while significantly improving quantum resource efficiency, operating successfully within typical NISQ constraints. By establishing a seamless integration pathway from today's classical LLMs to tomorrow's quantum-enhanced models, our framework provides both immediate practical value on current quantum hardware and a clear evolution path toward full Quantum LLMs as technology matures.
Quantum neural architectures are rapidly emerging as a new paradigm for efficient representation learning. Their intrinsic structure, built upon superposition and entanglement, enables expressive transformations with remarkably few parameters. Meanwhile, parameter-efficient fine-tuning (PEFT) has become a key approach for adapting large language models (LLMs), typically by inserting lightweight adapters or low-rank modules such as LoRA. However, strict low-rank constraints may limit representational capacity under complex domain shifts. This raises a natural question: what if quantum parameterization is introduced into the adapter framework? To address this, we propose a Quantum-Enhanced Adapter (QEA), which adds a parameterized quantum circuit branch in parallel to each Transformer feed-forward network while keeping all pretrained weights frozen. QEA encodes token features as rotation angles, reads out Pauli-Z expectations, and fuses quantum and classical paths via a token-wise gate fusion with a soft-to-hard schedule for stable training. On LLaMA-7B and LLaMA-3.2-1B across WikiText-2 and WikiText-103, QEA achieves comparable or lower perplexity than strong PEFT baselines while using only about 0.066-0.068% of the model parameters. These results highlight QEA’s ability to deliver substantial parameter savings without sacrificing performance. Our work demonstrates the promise of quantum-enhanced, parameter-efficient fine-tuning for LLMs, providing a scalable quantum-classical solution.