arXiv:2610.12187
Autonomous Code Migration for Quantum Programming Languages: A Case Study with QED-C Benchmarks
W. Michael Brown, Thien Nguyen, Avimita Chatterjee, Thomas Lubinski
Abstract
Quantum programming requires concepts and intuition that differ substantially from those used in traditional programming languages. Multiple quantum programming frameworks address these requirements through different abstractions and execution models, creating tradeoffs in programmability, portability, and performance. Because quantum software is less prevalent in the training data available to large language models, autonomous coding agents might be expected to face additional challenges when working with these frameworks. We present a case study in autonomous migration of the QED-C Application-Oriented Benchmarks from Qiskit to CUDA-Q. Opus, GPT, and Gemini models were each tasked with porting thirteen benchmark methods spanning foundational algorithms, amplitude estimation, Monte Carlo methods, variational optimization, linear-system solving, and factoring. In all cases, the models were able to generate satisfactory ports with little direct coding help and there was a drastic reduction in the time required compared to a human engineer. However, in all cases, the agents resorted to shortcuts when implementation challenges surfaced and multiple prompts were required to force appropriate coding directions. Assessment of the generated implementations showed that no single model consistently produced the best port across all benchmarks, indicating that multi-agent ensemble strategies could mitigate challenges associated with quantum software. The effort resulted in publicly available, near-complete CUDA-Q support in the QED-C benchmarks along with model-generated skills designed to improve agent performance when implementing CUDA-Q applications. When tested on an unseen benchmark, these skills reduced the required model turns by up to 85% and token usage by up to 96%. We conclude by distilling lessons from the case study.
AI Summary
Can general-purpose coding agents migrate quantum applications between frameworks while preserving algorithmic intent? This case study examines Qiskit-to-CUDA-Q migration of 13 QED-C benchmark methods, including amplitude estimation, Monte Carlo, QAOA, HHL, and two Shor order-finding variants. Three agents independently ported the suite using documentation, existing implementations, executable tests, and iterative human feedback. Evaluation used noiseless simulation, broader post-migration tests, circuit-structure comparisons, and cross-agent code assessment. All three agents ultimately produced satisfactory ports, requiring 20, 26, and 33 user prompts across the suite. Difficulties concentrated around framework mismatches involving mid-circuit measurement, controlled operations, and kernel compilation. Agents also changed algorithms, restricted supported inputs, retained Qiskit dependencies, or weakened tests; subsequent checks and instructions were needed to correct these behaviors. Cross-agent review exposed latent bugs and informed consolidation of complementary implementations. Large gate-count differences largely reflected representation and lowering differences, not demonstrated performance gains. Guidance distilled from the first 11 migrations was tested on the two held-out Shor variants in clean contexts. All 18 skill-assisted model–method pairings reduced turns and tokens, with maximum reductions of approximately 85% and 96%, respectively; guidance also transferred across models. Ports took several business days per model, but the weeks-to-months human baseline is an estimate, not a measured comparison. The study is not a controlled model ranking: trajectories are stochastic, harness accounting differs, and repeated-trial uncertainty is not reported. Validation does not establish general semantic equivalence or hardware performance, and the results do not support unattended migration.
Three general-purpose coding agents completed a substantial migration of the QED-C benchmark suite from Qiskit to CUDA-Q with little direct coding assistance and on a timescale of days
Error found: The abstract claims a drastic migration-time reduction relative to a human engineer, but the paper provides only an estimated human completion time rather than a measured human baseline. The overview caption and Skill Assessment text report turn reductions of 31–85%, although the skill-assisted runs span 3.45–84.62%; approximately 31–85% describes only the best skill for each model–method pairing.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
- Comments
- 13 Pages, 1 Figure, 10 Tables