arXiv:2609.19392
Transformers as Intrinsic Optimizers for Quantum Approximate Optimization Algorithm
Kuan-Cheng Chen, Xiaotian Xu, Hiromichi Matsuyama, Wei-Hao Huang, Haomu Yuan, Yu Yamashiro
Abstract
The Quantum Approximate Optimization Algorithm (QAOA) is a leading variational framework for combinatorial optimization on noisy intermediate-scale quantum hardware, but its practical performance depends strongly on the classical optimizer used to train its variational parameters. This outer-loop optimization is often nonconvex, initialization-sensitive, and costly when repeated across large families of related problem instances. In this work, we propose a Transformer-based intrinsic optimization framework for QAOA, in which the optimizer itself is learned and embedded directly into the hybrid quantum-classical loop. The proposed graph-conditioned Transformer processes problem structure, current QAOA parameters, measurement feedback, and recent optimization history to predict the next variational-parameter update, thereby reformulating instance-wise classical optimization as an amortized learned policy. We develop a mathematical formulation of this intrinsic-optimization perspective and evaluate the method on QAOA-based MaxCut benchmarks across multiple problem settings, with comparisons against representative classical and learned optimization baselines. The results demonstrate that Transformer-based intrinsic optimization can provide a structured and transferable mechanism for improving the classical component of hybrid quantum optimization algorithms.
AI Summary
The paper asks whether a learned, graph-conditioned optimizer can amortize QAOA parameter training across related MaxCut instances. It uses a Transformer to predict angle updates from a graph embedding, current angles, previous updates, objective feedback, and optimization progress. One token represents each QAOA layer, with an additional graph-context token. Updates are bounded by a scaled $\tanh$ and clipped to the parameter domain. Meta-training optimizes performance along unrolled QAOA trajectories, with optional update regularization and imitation pretraining. Evaluation uses disjoint training, validation, and test graph sets, comparing against MLP and LSTM learned optimizers, Adam variants, COBYLA, Nelder–Mead, and SPSA. At depth $p=2$, the Transformer’s mean expected-cut approximation ratios are $0.8932$, $0.8766$, and $0.8767$ for $n=5,8,11$ vertices, respectively. Corresponding MLP ratios are $0.8923$, $0.8709$, and $0.8708$; multi-start Adam achieves $0.8914$, $0.8716$, and $0.8662$. Additional studies vary graph size up to 13 qubits and depth up to $p=10$. For six-qubit, depth-four instances, reported transfer performance approaches saturation with approximately 50 training graphs. A model-size sweep reports its highest ratios around $0.94$ with moderate-capacity Transformers. The stated scope is small MaxCut instances and primarily same-distribution transfer. With vectorized-adjacency inputs, results apply within fixed-size regimes; variable-size use requires a compatible graph encoder. Larger and heterogeneous graph families, out-of-distribution transfer, hardware execution, hardware-aware noise and cost features, and theoretical analysis of learned dynamics are left for future work.
Central claim may not hold: The claimed matched-budget comparison and optimization-efficiency benefits cannot be verified because actual baseline evaluation budgets, gradient costs, stopping criteria, and offline training costs are not reported. The Transformer exceeds the MLP's reported mean approximation ratios by only 0.0009, 0.0057, and 0.0059, and the benchmark does not provide enough information to quantify uncertainty in these differences. The supplied source contains no plotted data for the figure-based experiments, leaving their claimed transfer, scaling, depth, model-size, and ablation trends supported only by narrative descriptions. The zero-update ablation does not establish that graph conditioning or measurement/history feedback contributes to performance, because it compares learned updates only against leaving the initial parameters unchanged. The initialization-noise experiment is not reproducibly specified or explicitly reconciled with the methodology's zero-angle initialization.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.