Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2410.17397

Quantum Large Language Models via Tensor Network Disentanglers

Borja Aizpurua, Loren, Fernando, Saeed S. Jahromi, Sukhbinder Singh, Román Orús

Published Oct 22, 2024• 22 citations
arXivHTMLPDF

Abstract

We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to construct a hybrid quantum-classical representation that exactly reproduces the original model, providing a principled starting point from which quantum resources can only improve performance. Our approach replaces the weight matrices in self-attention and multilayer perceptron layers with two variational quantum circuits coupled to a matrix product operator (MPO). Tensor network disentanglers transfer much of each layer's information into the quantum circuits, enabling the remaining tensor network to be compressed to a bond-dimension-one MPO with over three orders of magnitude fewer classical parameters (in our experiments, from 110,592 to approximately 36 for the replaced layer) and less than a 0.3\% increase in perplexity. Training an added unitary adapter on top of this representation then surpasses the original model, reducing perplexity by up to 1.6\%. Finally, we validate the hybrid architecture on a real quantum processor, demonstrating a practical route towards quantum-enhanced language models.

DOI
10.48550/arxiv.2410.17397

Related papers

Classical neural networks on quantum devices via tensor network disentanglers: A case study in image classification

Borja Aizpurua, Sukhbinder Singh, Román Orús

We address the problem of implementing bottleneck layers from classical pretrained neural networks on a quantum computer, with the goal of exploring intrinsically quantum ansatz for representing large linear layers within hybrid classical-quantum models. Our approach begins with a compression step in which the target linear layer is represented as an effective matrix product operator (MPO) without degrading model performance. The MPO is then further disentangled into a more compact form. This enables a hybrid classical-quantum execution scheme, where the disentangling circuits are deployed on a quantum computer while the remainder of the network—including the disentangled MPO—runs on classical hardware. We introduce two complementary algorithms for MPO disentangling: (1) an disentangling variational method leveraging standard tensor network optimization techniques and (2) an disentangling gradient-descent-based approach. We validate these methods through a proof-of-concept translation of simple classical neural networks for MNIST and CIFAR-10 image classification into a hybrid classical-quantum form.

arXiv:2509.06653•Physical Review Research•Mar 12, 2026•0 citations

Hybrid Quantum-Classical Transfer Learning for Parameter-Efficient Code Generation

Su-Chang Lim, Jong-Chan Kim

The remarkable success of large language models (LLMs) in code generation has transformed software development, with systems like Codex, CodeGen, and StarCoder demonstrating strong capabilities in synthesizing functional code from natural language descriptions. However, this progress comes at a steep computational cost. Modern code generation models require hundreds of millions to billions of parameters—GPT-3 contains 175 billion parameters, CodeGen-16B uses 16 billion, and the trend continues toward ever-larger models. This scale creates fundamental challenges in training cost, inference latency, memory consumption, and environmental impact. We present a quantum transfer learning framework as an alternative approach to classical model compression for code generation. By replacing selected MLP layers with variational quantum circuits (VQCs), we exploit the exponential representational capacity of quantum systems to achieve $\Theta (D \log D)$ parameter scaling instead of classical $\Theta (D^{2})$ . Our hybrid classical-quantum architecture balances three competing objectives: leveraging existing pre-trained knowledge, achieving parameter compression through quantum representation, and maintaining compatibility with near-term quantum hardware constraints. We demonstrate our approach on CodeGen-350M, a production-scale code generation model, adopting a bottleneck architecture that substantially reduces parameters compared to the original MLP layer. Through a fair comparison against a classical bottleneck baseline with near-identical parameter count, we show that the VQC achieves superior representational quality—lower perplexity and higher code generation accuracy—validating the quantum layer’s expressivity advantage in the compressed latent space. Our design employs data re-uploading, where classical input features are re-encoded at each variational layer via trainable RY gates, enhancing the circuit’s expressive power on near-term devices.

IEEE Access•Jan 1, 2026•0 citations

Hybrid Quantum Transformer for Language Generation

Kong, Desheng, Xiaohai Cui, Jin, Jiaying +2 more

Although quantum computing has been increasingly applied to replace classical computation, most existing quantum or hybrid models remain confined to simple tasks, with no successful application to large-scale natural language generation to date. In this work, we present the first hybrid quantum-classical large language model (LLM) for natural language generation, HyQuT, capable of performing coherent and context-aware dialogue. The proposed architecture integrates variational quantum circuits (VQCs) into the Transformer framework at both 8M and 150M parameter scales. Experimental results show that a minimal number of qubits (10 qubits with 80 quantum gates) can replace about 10% of the classical parameters in the 150M-parameter model, while achieving comparable convergence stability and generation quality. This study provides an early demonstration of the feasibility of integrating quantum computing to large-scale generative language models.

arXiv:2511.10653•Nov 2, 2025•6 citations

Elevating LLMs Via Synergy of Quantum Computing and Classical Transformer Models

Shashanka Shekhar Sharma, Ritisha Katiyar, Hrishikesh Sarma +1 more

Quantum computation offers new directions in computer science that go beyond classical systems' state-of-the-art abilities. The combination of quantum technologies with Large Language Models (LLMs) provides promising avenues for the creation of more efficient and powerful NLP architectures. In this research, we examine quantum-enhanced transformers and further develop the Self-Attention Sequential Quantum Transformer Channel (SASQuatch) framework to enhance inference time and perplexity. The models proposed are tested on sentence completion tasks and compared against a baseline transformer, showing significant gains in efficiency and predictive power. We further compare the method with other optimization methods, such as the Quantum Approximate Optimization Algorithm (QAOA), thus highlighting the necessity of further investigation of hybrid quantum architectures in improving the performance of large-scale language models.

International Conference on AI-ML Systems (AIMLSystems)•Oct 8, 2025•1 citation

A Quantum Circuit-Based Compression Perspective for Parameter-Efficient Learning

Chen-Yu Liu, Chao-Han Huck Yang, Hsi‐Sheng Goan +1 more

Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to $52.06\%$ of the original LoRA for GPT-2 with a slight performance gain of $0.75\%$, and to $16.84\%$ for Gemma-2, with a marginal performance improvement of $0.07\%$. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.

arXiv:2410.09846•Oct 13, 2024•15 citations

Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters

Borja Aizpurua, Sukhbinder Singh, Augustine Kshetrimayum +2 more

Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.

quant-pharXiv:2605.05914•May 8, 2026•6 citations
We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to construct a hybrid quantum-classical representation that exactly reproduces the original model, providing a principled starting point from which quantum resources can only improve performance. Our approach replaces the weight matrices in self-attention and multilayer perceptron layers with two variational quantum circuits coupled to a matrix product operator (MPO). Tensor network disentanglers transfer much of each layer's information into the quantum circuits, enabling the remaining tensor network to be compressed to a bond-dimension-one MPO with over three orders of magnitude fewer classical parameters (in our experiments, from 110,592 to approximately 36 for the replaced layer) and less than a 0.3% increase in perplexity. Training an added unitary adapter on top of this representation then surpasses the original model, reducing perplexity by up to 1.6%. Finally, we validate the hybrid architecture on a real quantum processor, demonstrating a practical route towards quantum-enhanced language models.
We address the problem of implementing bottleneck layers from classical pretrained neural networks on a quantum computer, with the goal of exploring intrinsically quantum ansatz for representing large linear layers within hybrid classical-quantum models. Our approach begins with a compression step in which the target linear layer is represented as an effective matrix product operator (MPO) without degrading model performance. The MPO is then further disentangled into a more compact form. This enables a hybrid classical-quantum execution scheme, where the disentangling circuits are deployed on a quantum computer while the remainder of the network—including the disentangled MPO—runs on classical hardware. We introduce two complementary algorithms for MPO disentangling: (1) an disentangling variational method leveraging standard tensor network optimization techniques and (2) an disentangling gradient-descent-based approach. We validate these methods through a proof-of-concept translation of simple classical neural networks for MNIST and CIFAR-10 image classification into a hybrid classical-quantum form.
The remarkable success of large language models (LLMs) in code generation has transformed software development, with systems like Codex, CodeGen, and StarCoder demonstrating strong capabilities in synthesizing functional code from natural language descriptions. However, this progress comes at a steep computational cost. Modern code generation models require hundreds of millions to billions of parameters—GPT-3 contains 175 billion parameters, CodeGen-16B uses 16 billion, and the trend continues toward ever-larger models. This scale creates fundamental challenges in training cost, inference latency, memory consumption, and environmental impact. We present a quantum transfer learning framework as an alternative approach to classical model compression for code generation. By replacing selected MLP layers with variational quantum circuits (VQCs), we exploit the exponential representational capacity of quantum systems to achieve Θ(Dlog⁡D)\Theta (D \log D)Θ(DlogD) parameter scaling instead of classical Θ(D2)\Theta (D^{2})Θ(D2) . Our hybrid classical-quantum architecture balances three competing objectives: leveraging existing pre-trained knowledge, achieving parameter compression through quantum representation, and maintaining compatibility with near-term quantum hardware constraints. We demonstrate our approach on CodeGen-350M, a production-scale code generation model, adopting a bottleneck architecture that substantially reduces parameters compared to the original MLP layer. Through a fair comparison against a classical bottleneck baseline with near-identical parameter count, we show that the VQC achieves superior representational quality—lower perplexity and higher code generation accuracy—validating the quantum layer’s expressivity advantage in the compressed latent space. Our design employs data re-uploading, where classical input features are re-encoded at each variational layer via trainable RY gates, enhancing the circuit’s expressive power on near-term devices.
Although quantum computing has been increasingly applied to replace classical computation, most existing quantum or hybrid models remain confined to simple tasks, with no successful application to large-scale natural language generation to date. In this work, we present the first hybrid quantum-classical large language model (LLM) for natural language generation, HyQuT, capable of performing coherent and context-aware dialogue. The proposed architecture integrates variational quantum circuits (VQCs) into the Transformer framework at both 8M and 150M parameter scales. Experimental results show that a minimal number of qubits (10 qubits with 80 quantum gates) can replace about 10% of the classical parameters in the 150M-parameter model, while achieving comparable convergence stability and generation quality. This study provides an early demonstration of the feasibility of integrating quantum computing to large-scale generative language models.
Quantum computation offers new directions in computer science that go beyond classical systems' state-of-the-art abilities. The combination of quantum technologies with Large Language Models (LLMs) provides promising avenues for the creation of more efficient and powerful NLP architectures. In this research, we examine quantum-enhanced transformers and further develop the Self-Attention Sequential Quantum Transformer Channel (SASQuatch) framework to enhance inference time and perplexity. The models proposed are tested on sentence completion tasks and compared against a baseline transformer, showing significant gains in efficiency and predictive power. We further compare the method with other optimization methods, such as the Quantum Approximate Optimization Algorithm (QAOA), thus highlighting the necessity of further investigation of hybrid quantum architectures in improving the performance of large-scale language models.
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06%52.06\%52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%0.75\%0.75%, and to 16.84%16.84\%16.84% for Gemma-2, with a marginal performance improvement of 0.07%0.07\%0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.