arXiv:2611.02833
Reimagining computational quantum research with AI scientists
Yuxuan Du, Hanqing Zhao, Xinbiao Wang, Yiming Lu, Kaining Zhang, Dacheng Tao
Abstract
AI scientists are expanding automated research across scientific domains, yet their ability to turn the scientific record into a foundation for autonomous discovery remains elusive. Here we uncover a fundamental barrier to this transition and demonstrate a route to overcome it in computational quantum science, where advances shape both what we can learn from complex quantum systems and what quantum computing can ultimately deliver. To identify this barrier, we introduce OmniQ, an expert-audited virtual research environment that converts published studies into executable reproduction tasks. Across 59 papers spanning classical simulation, quantum algorithms and quantum-property characterization, we systematically evaluate five agentic systems powered by large language models and six third-party agent harnesses. Our findings reveal that current AI scientists often fail to reproduce published results, exposing a fundamental mismatch between what scientific papers communicate and what autonomous agents require. By contrast, human expert intervention can turn these failures into successful reproductions, showing that the missing information can be identified and converted into reusable research experience. Guided by this observation, we develop Residual Agents (RA), a quantum-tailored self-improving AI scientist that distils externally verified workflow repairs into such experience. RA substantially improves research performance under fixed base models and limited adaptation budgets. For example, RA raises DeepSeek V4.1 Flash performance from 68% to 85%, matching Claude Code with Opus 5 at a third of its cost despite using a weaker base model. We integrate all reproduced results into an extensible platform built on the model context protocol, making verified research workflows directly accessible to AI scientists and allowing new scientific papers to be continuously incorporated.
AI Summary
The paper asks whether LLM-based agents can reconstruct computational quantum-science results from published papers, and whether verified reproduction failures can supply reusable research experience. It introduces OmniQ, an expert-audited environment containing figure-reproduction tasks from 60 papers across classical simulation, quantum algorithms, and quantum-property characterization, selected for execution within approximately one day per task. Evaluation combines automated inspection of code, logs, data, and numerical checkpoints with cross-agent auditing and expert review. Issues are identified in 18 tasks, ranging from missing methodological details and errors in benchmark specifications to paper–code discrepancies. One unresolved case is excluded, leaving 59 tasks after corrections. In that case, a reported kernel variance reaches $0.49$, exceeding the $0.25$ upper bound for a quantity in $[0,1]$; the released code instead computes a distance variance. On the cleaned benchmark, reported agent success rates range from approximately 19% to 93%. Residual Agents adds planner–executor–critic coordination and external memories distilled from expert-verified repairs, without changing model weights. Developed using five challenging tasks, with 54 remaining unseen, it raises DeepSeek V4.1 Flash success from 68% to approximately 85%, at an average inference cost of $5.6 per task. The aggregate score includes development tasks. The study addresses reproduction rather than new algorithm discovery. Its scope excludes many computationally intensive studies, adaptation requires human approval, and some corrected tasks assess qualitative conclusions rather than original numerical values. Fully reliable reproduction, broader disciplinary coverage, and extension to autonomous discovery remain future directions.
Central claim may not hold: The paper characterizes all 18 audited studies as having scientific-record inconsistencies even though its audit table includes errors introduced by its own task specifications and grading criteria. The headline 68%-to-85% comparison includes five development tasks rather than reporting an exclusively held-out rate, although the aggregate counts still imply substantial held-out improvement. The presented RA-versus-DSH comparison changes both orchestration and experience memory, so it does not isolate reusable verified repairs as the cause of the performance gain.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
- Comments
- 57 pages, 19 figures. Comments are welcome