IEEE Transactions on Quantum Engineering
QCHFT: Quantum Cross-Hybrid Fine-Tuning for LLMs
Yuanjie Li, Kyungmin Lim, Jinsuk Baek, Minho Jo
Abstract
When full-parameter updates are impractical for large language models (LLMs), parameter-efficient fine-tuning (PEFT) is commonly employed to reduce the number of trainable parameters. Classical PEFT methods, such as low-rank adaptation (LoRA), are limited to linear transformations and may not capture complex, high-order feature interactions under stringent parameter constraints. In contrast, parameterized quantum circuits (PQCs) provide high-order expressivity per qubit rotation degree of freedom. Inspired by PQCs, in this paper, we propose quantum cross-hybrid fine-tuning (QCHFT), a suite of four PQC-augmented adapters: Matrix Product State PQC, Alternating and Hierarchical Pairwise Contraction PQC, and LinearA–Quantum–LinearB PQC. To evaluate our QCHFT adapters, we conduct experiments on a public multi-tier IoT resource-allocation task that serves as a challenging benchmark with intricate cross-tier interactions data and limited parameter budgets. We conduct experiments on a multi-tier IoT resource-allocation task across five causal LLM backbones scaling from 125M to 2B parameters, including Mamba2, GPT-Neo, Pythia, Llama-3.2, and Qwen-3.5. Our results demonstrate that QCHFT consistently enhances performance compared to both standard LoRA. On Mamba2-130M, the APC–PQC variant improves accuracy from 67.33% (LoRA) to 72.66%. These findings suggest that the high expressivity of quantum entanglement can be practically harnessed through shallow circuits, providing a superior, parameter-efficient pathway for adapting foundation models to complex, high-feature tasks.