Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2609.40146

QuLoC: Photonic Quantum-Assisted Low-Rank LLM Compression

Xiao-Hui Ni, Yu-Han Yao, Yu-Ze Zhu, Hang Song, Xiang Zhao, Lin Yang, Xian-Min Jin

quant-phSubmitted Sep 30, 2026• Announced Oct 1, 2026• 1 citation• Revised Oct 2, 2026 (v2)
v1 · Sep 30, 2026v2 · Oct 2, 2026
arXivHTMLPDF

Abstract

As LLMs grow in size, compression becomes increasingly important for efficient deployment. SVD-based low-rank compression reduces parameter counts but can degrade downstream performance. To improve performance after compression, we introduce QuLoC, a photonic quantum-assisted LLM compression algorithm that uses quantum circuit outputs to gate the retained low-rank components during training. Model performance is recovered through local functional reconstruction followed by end-to-end knowledge distillation. After training, the gating coefficients are absorbed into the low-rank factors, allowing the compressed model to run on classical hardware without executing quantum circuits during inference. Experiments on Qwen3.5-4B show that QuLoC achieves a 9.73\% relative improvement in average accuracy over state-of-the-art baselines across multiple downstream tasks, demonstrating its effectiveness. We further evaluate QuLoC on LLaMA-7B at different parameter compression ratios. It achieves comparable or higher average downstream accuracy than state-of-the-art baselines, supporting its applicability to a larger model across different compression settings. These results motivate further exploration of photonic quantum-assisted compression for larger models and more complex agentic tasks.

AI Summary

QuLoC investigates whether photonic-circuit-generated gates can improve recovery after low-rank compression of LLM MLP projections. Each SwiGLU projection is parameterized as $P\operatorname{diag}(g)B$, with SVD-initialized trainable factors and input-independent channel gains obtained by mapping photonic response features through a trainable classical matrix. Training combines layer-wise functional reconstruction with end-to-end next-token supervision and knowledge distillation. The final gates are folded exactly into the factors, eliminating photonic computation at inference. Experiments use numerically simulated 16-mode single-photon circuits and, separately, prerecorded response curves from seven four-mode squeezed-vacuum photonic configurations. On Qwen3.5-4B, compressing 16 middle MLPs removes 20.05% of language-model parameters. With a 27B distillation teacher, six-task mean accuracy is 54.03%, versus 49.24% for Swift-SVD and 48.54% for a matched two-stage, ungated SVD baseline; the uncompressed model scores 63.32%. WikiText-2/C4 perplexities are 12.173/24.644, versus 14.133/26.027 for the ungated baseline. Hardware-derived features yield 53.23% mean accuracy. On LLaMA-7B, mean accuracy is 52% at 80% parameter retention and 44% at 60%, versus 51% and 44% for Swift-SVD*, with mixed perplexity comparisons. A reconstruction ablation improves three-task mean accuracy by 0.83 percentage points but adds training compute. The paper explicitly does not establish quantum computational advantage or inference speedup. Stated limitations include token-independent gains, restricted architectures and compression settings, and the need for additional random seeds and broader hardware-condition testing.

Some signs of AI-generated textAI use disclosedMissing comparisonUnclear noveltyIncremental
ChatGPT (OpenAI, GPT-6 Astra) was used to improve the language, clarity, and consistency of the manuscript, particularly in the methods, discussion, and conclusion sections.

Error found: The experiments lack a trainable classical-gating control, so they establish improvements from the complete QuLoC parameterization but do not establish that photonic features contribute beyond ordinary trainable gates.

Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.

Related papers

Quantum Knowledge Distillation for Large Language Models

Lingxiao Li, Wang, Yihao, Jiacheng Fan +4 more

As foundational tools in natural language processing, Large Language Models (LLMs) have immense parameter scales, which makes deployment and inference increasingly prohibitive, especially in resource-constrained devices. Therefore, knowledge distillation for LLMs, i.e., compressing the LLM to a smaller model, is meaningful. With strong parameter representation capacity, quantum computing is regarded as a promising solution. Here, we propose a Quantum knowledge Distillation model for LLMs (QD-LLM) that leverages variational quantum circuits to learn from LLMs. In classical simulation, QD-LLM outperforms several mainstream distillation methods on multiple text classification tasks in terms of both accuracy and efficiency using only 11 qubits. The results reveal an interesting phenomenon that the simulation of quantum student models may be regarded as a new class of quantum-inspired classical algorithms. Remarkably, we deploy the obtained circuits on the Baihua superconducting quantum processor via the Quafu platform to assess practical feasibility. The model maintains stable inference performance despite hardware constraints such as decoherence and finite sampling. In summary, QD-LLM marks a foundational step in connecting quantum computing with LLMs, demonstrating the feasibility of quantum-native approaches that aim to compress and deploy models of increasingly larger scales. The code of this article has been open-sourced at https://github.com/Lilingxiao-bupt/QD-LLM.

arXiv:2505.13205•May 19, 2025•24 citations

A Quantum Circuit-Based Compression Perspective for Parameter-Efficient Learning

Chen-Yu Liu, Chao-Han Huck Yang, Hsi‐Sheng Goan +1 more

Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to $52.06\%$ of the original LoRA for GPT-2 with a slight performance gain of $0.75\%$, and to $16.84\%$ for Gemma-2, with a marginal performance improvement of $0.07\%$. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.

arXiv:2410.09846•Oct 13, 2024•15 citations

CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

Saeed S. Jahromi, Uygar Kurt, Sukhi Singh +3 more

Large Language Models (LLMs) like ChatGPT and LlaMA offer immense opportunities but face challenges due to their vast size, leading to high training/inference costs and energy demands.Traditional compression methods focus on reducing neurons or precision.We introduce CompactifAI, a novel LLM compression using quantum-inspired Tensor Networks to compress the model's correlation space.Testing on LlaMA-2 7B showed a 93% memory and 70% parameter reduction, with minimal accuracy loss (2-3%) and significant training (50%) and inference (25%) speedups.

European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)•Jan 1, 2025•0 citations

QCHFT: Quantum Cross-Hybrid Fine-Tuning for LLMs

Yuanjie Li, Kyungmin Lim, Jinsuk Baek +1 more

When full-parameter updates are impractical for large language models (LLMs), parameter-efficient fine-tuning (PEFT) is commonly employed to reduce the number of trainable parameters. Classical PEFT methods, such as low-rank adaptation (LoRA), are limited to linear transformations and may not capture complex, high-order feature interactions under stringent parameter constraints. In contrast, parameterized quantum circuits (PQCs) provide high-order expressivity per qubit rotation degree of freedom. Inspired by PQCs, in this paper, we propose quantum cross-hybrid fine-tuning (QCHFT), a suite of four PQC-augmented adapters: Matrix Product State PQC, Alternating and Hierarchical Pairwise Contraction PQC, and LinearA–Quantum–LinearB PQC. To evaluate our QCHFT adapters, we conduct experiments on a public multi-tier IoT resource-allocation task that serves as a challenging benchmark with intricate cross-tier interactions data and limited parameter budgets. We conduct experiments on a multi-tier IoT resource-allocation task across five causal LLM backbones scaling from 125M to 2B parameters, including Mamba2, GPT-Neo, Pythia, Llama-3.2, and Qwen-3.5. Our results demonstrate that QCHFT consistently enhances performance compared to both standard LoRA. On Mamba2-130M, the APC–PQC variant improves accuracy from 67.33% (LoRA) to 72.66%. These findings suggest that the high expressivity of quantum entanglement can be practically harnessed through shallow circuits, providing a superior, parameter-efficient pathway for adapting foundation models to complex, high-feature tasks.

IEEE Transactions on Quantum Engineering•Jan 1, 2026•0 citations

QEA: A Quantum-Enhanced Adapter for Parameter-Efficient Fine-Tuning of LLMs

Fanqi· Dong, Chenhao Zhang, Yuxin Deng +1 more

Quantum neural architectures are rapidly emerging as a new paradigm for efficient representation learning. Their intrinsic structure, built upon superposition and entanglement, enables expressive transformations with remarkably few parameters. Meanwhile, parameter-efficient fine-tuning (PEFT) has become a key approach for adapting large language models (LLMs), typically by inserting lightweight adapters or low-rank modules such as LoRA. However, strict low-rank constraints may limit representational capacity under complex domain shifts. This raises a natural question: what if quantum parameterization is introduced into the adapter framework? To address this, we propose a Quantum-Enhanced Adapter (QEA), which adds a parameterized quantum circuit branch in parallel to each Transformer feed-forward network while keeping all pretrained weights frozen. QEA encodes token features as rotation angles, reads out Pauli-Z expectations, and fuses quantum and classical paths via a token-wise gate fusion with a soft-to-hard schedule for stable training. On LLaMA-7B and LLaMA-3.2-1B across WikiText-2 and WikiText-103, QEA achieves comparable or lower perplexity than strong PEFT baselines while using only about 0.066-0.068% of the model parameters. These results highlight QEA’s ability to deliver substantial parameter savings without sacrificing performance. Our work demonstrates the promise of quantum-enhanced, parameter-efficient fine-tuning for LLMs, providing a scalable quantum-classical solution.

International Joint Conference on Neural Networks (IJCNN)•Jun 1, 2026•0 citations

Photonic Quantum-Enhanced Knowledge Distillation

Kuan-Cheng Chen, Shang Yu, Chen-Yu Liu +10 more

Photonic quantum processors naturally produce intrinsically stochastic measurement outcomes, offering a hardware-native source of structured randomness that can be exploited during machine-learning training. Here we introduce Photonic Quantum-Enhanced Knowledge Distillation (PQKD), a hybrid quantum photonic--classical framework in which a programmable photonic circuit generates a compact conditioning signal that constrains and guides a parameter-efficient student network during distillation from a high-capacity teacher. PQKD replaces fully trainable convolutional kernels with dictionary convolutions: each layer learns only a small set of shared spatial basis filters, while sample-dependent channel-mixing weights are derived from shot-limited photonic features and mapped through a fixed linear transform. Training alternates between standard gradient-based optimisation of the student and sampling-robust, gradient-free updates of photonic parameters, avoiding differentiation through photonic hardware. Across MNIST, Fashion-MNIST and CIFAR-10, PQKD traces a controllable compression--accuracy frontier, remaining close to teacher performance on simpler benchmarks under aggressive convolutional compression. Performance degrades predictably with finite sampling, consistent with shot-noise scaling, and exponential moving-average feature smoothing suppresses high-frequency shot-noise fluctuations, extending the practical operating regime at moderate shot budgets.

quant-pharXiv:2603.14898•Mar 17, 2026•2 citations
As LLMs grow in size, compression becomes increasingly important for efficient deployment. SVD-based low-rank compression reduces parameter counts but can degrade downstream performance. To improve performance after compression, we introduce QuLoC, a photonic quantum-assisted LLM compression algorithm that uses quantum circuit outputs to gate the retained low-rank components during training. Model performance is recovered through local functional reconstruction followed by end-to-end knowledge distillation. After training, the gating coefficients are absorbed into the low-rank factors, allowing the compressed model to run on classical hardware without executing quantum circuits during inference. Experiments on Qwen3.5-4B show that QuLoC achieves a 9.73% relative improvement in average accuracy over state-of-the-art baselines across multiple downstream tasks, demonstrating its effectiveness. We further evaluate QuLoC on LLaMA-7B at different parameter compression ratios. It achieves comparable or higher average downstream accuracy than state-of-the-art baselines, supporting its applicability to a larger model across different compression settings. These results motivate further exploration of photonic quantum-assisted compression for larger models and more complex agentic tasks.
QuLoC investigates whether photonic-circuit-generated gates can improve recovery after low-rank compression of LLM MLP projections. Each SwiGLU projection is parameterized as Pdiag⁡(g)BP\operatorname{diag}(g)BPdiag(g)B, with SVD-initialized trainable factors and input-independent channel gains obtained by mapping photonic response features through a trainable classical matrix. Training combines layer-wise functional reconstruction with end-to-end next-token supervision and knowledge distillation. The final gates are folded exactly into the factors, eliminating photonic computation at inference.
Experiments use numerically simulated 16-mode single-photon circuits and, separately, prerecorded response curves from seven four-mode squeezed-vacuum photonic configurations. On Qwen3.5-4B, compressing 16 middle MLPs removes 20.05% of language-model parameters. With a 27B distillation teacher, six-task mean accuracy is 54.03%, versus 49.24% for Swift-SVD and 48.54% for a matched two-stage, ungated SVD baseline; the uncompressed model scores 63.32%. WikiText-2/C4 perplexities are 12.173/24.644, versus 14.133/26.027 for the ungated baseline. Hardware-derived features yield 53.23% mean accuracy. On LLaMA-7B, mean accuracy is 52% at 80% parameter retention and 44% at 60%, versus 51% and 44% for Swift-SVD*, with mixed perplexity comparisons.
A reconstruction ablation improves three-task mean accuracy by 0.83 percentage points but adds training compute. The paper explicitly does not establish quantum computational advantage or inference speedup. Stated limitations include token-independent gains, restricted architectures and compression settings, and the need for additional random seeds and broader hardware-condition testing.
As foundational tools in natural language processing, Large Language Models (LLMs) have immense parameter scales, which makes deployment and inference increasingly prohibitive, especially in resource-constrained devices. Therefore, knowledge distillation for LLMs, i.e., compressing the LLM to a smaller model, is meaningful. With strong parameter representation capacity, quantum computing is regarded as a promising solution. Here, we propose a Quantum knowledge Distillation model for LLMs (QD-LLM) that leverages variational quantum circuits to learn from LLMs. In classical simulation, QD-LLM outperforms several mainstream distillation methods on multiple text classification tasks in terms of both accuracy and efficiency using only 11 qubits. The results reveal an interesting phenomenon that the simulation of quantum student models may be regarded as a new class of quantum-inspired classical algorithms. Remarkably, we deploy the obtained circuits on the Baihua superconducting quantum processor via the Quafu platform to assess practical feasibility. The model maintains stable inference performance despite hardware constraints such as decoherence and finite sampling. In summary, QD-LLM marks a foundational step in connecting quantum computing with LLMs, demonstrating the feasibility of quantum-native approaches that aim to compress and deploy models of increasingly larger scales. The code of this article has been open-sourced at https://github.com/Lilingxiao-bupt/QD-LLM.
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06%52.06\%52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%0.75\%0.75%, and to 16.84%16.84\%16.84% for Gemma-2, with a marginal performance improvement of 0.07%0.07\%0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
Large Language Models (LLMs) like ChatGPT and LlaMA offer immense opportunities but face challenges due to their vast size, leading to high training/inference costs and energy demands.Traditional compression methods focus on reducing neurons or precision.We introduce CompactifAI, a novel LLM compression using quantum-inspired Tensor Networks to compress the model's correlation space.Testing on LlaMA-2 7B showed a 93% memory and 70% parameter reduction, with minimal accuracy loss (2-3%) and significant training (50%) and inference (25%) speedups.
When full-parameter updates are impractical for large language models (LLMs), parameter-efficient fine-tuning (PEFT) is commonly employed to reduce the number of trainable parameters. Classical PEFT methods, such as low-rank adaptation (LoRA), are limited to linear transformations and may not capture complex, high-order feature interactions under stringent parameter constraints. In contrast, parameterized quantum circuits (PQCs) provide high-order expressivity per qubit rotation degree of freedom. Inspired by PQCs, in this paper, we propose quantum cross-hybrid fine-tuning (QCHFT), a suite of four PQC-augmented adapters: Matrix Product State PQC, Alternating and Hierarchical Pairwise Contraction PQC, and LinearA–Quantum–LinearB PQC. To evaluate our QCHFT adapters, we conduct experiments on a public multi-tier IoT resource-allocation task that serves as a challenging benchmark with intricate cross-tier interactions data and limited parameter budgets. We conduct experiments on a multi-tier IoT resource-allocation task across five causal LLM backbones scaling from 125M to 2B parameters, including Mamba2, GPT-Neo, Pythia, Llama-3.2, and Qwen-3.5. Our results demonstrate that QCHFT consistently enhances performance compared to both standard LoRA. On Mamba2-130M, the APC–PQC variant improves accuracy from 67.33% (LoRA) to 72.66%. These findings suggest that the high expressivity of quantum entanglement can be practically harnessed through shallow circuits, providing a superior, parameter-efficient pathway for adapting foundation models to complex, high-feature tasks.
Quantum neural architectures are rapidly emerging as a new paradigm for efficient representation learning. Their intrinsic structure, built upon superposition and entanglement, enables expressive transformations with remarkably few parameters. Meanwhile, parameter-efficient fine-tuning (PEFT) has become a key approach for adapting large language models (LLMs), typically by inserting lightweight adapters or low-rank modules such as LoRA. However, strict low-rank constraints may limit representational capacity under complex domain shifts. This raises a natural question: what if quantum parameterization is introduced into the adapter framework? To address this, we propose a Quantum-Enhanced Adapter (QEA), which adds a parameterized quantum circuit branch in parallel to each Transformer feed-forward network while keeping all pretrained weights frozen. QEA encodes token features as rotation angles, reads out Pauli-Z expectations, and fuses quantum and classical paths via a token-wise gate fusion with a soft-to-hard schedule for stable training. On LLaMA-7B and LLaMA-3.2-1B across WikiText-2 and WikiText-103, QEA achieves comparable or lower perplexity than strong PEFT baselines while using only about 0.066-0.068% of the model parameters. These results highlight QEA’s ability to deliver substantial parameter savings without sacrificing performance. Our work demonstrates the promise of quantum-enhanced, parameter-efficient fine-tuning for LLMs, providing a scalable quantum-classical solution.
Photonic quantum processors naturally produce intrinsically stochastic measurement outcomes, offering a hardware-native source of structured randomness that can be exploited during machine-learning training. Here we introduce Photonic Quantum-Enhanced Knowledge Distillation (PQKD), a hybrid quantum photonic--classical framework in which a programmable photonic circuit generates a compact conditioning signal that constrains and guides a parameter-efficient student network during distillation from a high-capacity teacher. PQKD replaces fully trainable convolutional kernels with dictionary convolutions: each layer learns only a small set of shared spatial basis filters, while sample-dependent channel-mixing weights are derived from shot-limited photonic features and mapped through a fixed linear transform. Training alternates between standard gradient-based optimisation of the student and sampling-robust, gradient-free updates of photonic parameters, avoiding differentiation through photonic hardware. Across MNIST, Fashion-MNIST and CIFAR-10, PQKD traces a controllable compression--accuracy frontier, remaining close to teacher performance on simpler benchmarks under aggressive convolutional compression. Performance degrades predictably with finite sampling, consistent with shot-noise scaling, and exponential moving-average feature smoothing suppresses high-frequency shot-noise fluctuations, extending the practical operating regime at moderate shot budgets.