Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2610.32651

CC4Q Workbench: From Quantum Repositories to Analyzable Code Comments

Zenghui Zhou, Yuechen Li, Zheng Zheng

Published Oct 23, 2026• 0 citations
arXivHTMLPDF

Abstract

Quantum software development kits (SDKs) expose high-level APIs whose use requires an understanding of underlying mathematical principles, circuit semantics, and hardware properties. Code comments can provide natural-language or structural explanations that are not explicit in API calls. However, studying comments in a new quantum SDK still requires researchers to coordinate separate steps for comment extraction, quantum-specific classification, and manual verification, resulting in repeated integration effort. To address this problem, we present CC4Q Workbench, an open-source workbench that integrates these steps into a reusable workflow. Given a Python repository at a specified revision, CC4Q Workbench automatically extracts code-comment pairs, segments comments into sentence-level units, and applies two trained classifiers to identify quantum-specific content and assign quantum-knowledge categories. Its web interface supports exploration of bundled datasets, inspection of code snippets, corpus construction from new repositories, manual annotation with model predictions hidden during binary labeling, and classifier fine-tuning using exported annotations. The release provides aggregate statistics for six SDK datasets comprising 39,593 sentence-level code comment units (SCCUs), of which 17,517 are browsable in the web interface. Applied to three additional application-layer Qiskit repositories outside the original CC4Q dataset, the pipeline ran without code changes and produced 11,517 SCCUs. Manual verification of 379 SCCUs showed 85.0% agreement between human annotations and the binary classifier's predictions.

DOI
10.48550/arxiv.2610.32651

Related papers

Code Comments for Quantum Software Development Kits: An Empirical Study on Qiskit

Zenghui Zhou, Yuechen Li, Yi Cai +4 more

Quantum computing is gaining attention from academia and industry. With the quantum Software Development Kits (SDKs), programmers can develop quantum software to explore the power of quantum computing. However, programmers may face challenges in understanding quantum software due to the non-intuitive quantum mechanics. To facilitate software development and maintenance, code comments offered in quantum SDKs serve as a natural language explanation of program functionalities and logical flows. Despite their importance, scarce research systematically reports their value and provides constructive guidelines for programmers. To address this gap, our paper focuses on Qiskit, one of the most popular quantum SDKs, and presents CC4Q, the first dataset of code comments for quantum computing. CC4Q incorporates 12,812 code comment pairs and 28,076 sentence-level code comment units, the latter of which involve heavy human annotation. Regarding the annotation, we validate the applicability of the developer-intent taxonomy used in classical programs, and also propose a new taxonomy considering quantum-specific knowledge. We conduct an empirical study comprehensively interpreting code comments from three perspectives: comment structure and coverage, developers’ intentions, and associated quantum topics. Our findings uncover key differences in code comments between classical and quantum software, and also outline quantum-specific knowledge relevant to quantum software development.

arXiv:2512.00766•ACM Transactions on Software Engineering and Methodology•Sep 15, 2026•1 citation

Large Language Models for Quantum Code Translation: An Exploratory Study

Xiyuan Li, Yuechen Li, Jianjun Zhao

Quantum software development is challenged by quantum-mechanical semantics and fragmented software development kits (SDKs), which complicate cross-SDK code translation and execution. In this paper, we propose an LLM-based approach for translating executable programs between high-level quantum SDKs. We develop a quantum-aware translation workflow whose prompt design and validation reflect quantum semantics and accommodate the version evolution of quantum ecosystems. We conduct a preliminary evaluation of 4 LLM configurations on 43 tasks, translating Qiskit programs to PennyLane and Q#. LLMs achieve a zero-shot Pass@1 of up to 72.33%, while one-shot prompting further improves the translation effectiveness. These results suggest potential for quantum code translation by LLMs.

cs.SECross-listedarXiv:2610.34841•Oct 27, 2026•0 citations

C2|Q>: A Robust Framework for Bridging Classical and Quantum Software Development

Boshuai Ye, Arif Ali Khan, Teemu Pihkakoski +4 more

Quantum Software Engineering (QSE) is emerging as a critical discipline to make quantum computing accessible to a broader developer community; however, most quantum development environments still require developers to engage with low-level details across the software stack—including problem encoding, circuit construction, algorithm configuration, hardware selection, and result interpretation—making them difficult for classical software engineers to use. To bridge this gap, we present C2 \(\lvert Q \rangle\) , a hardware-agnostic quantum software development framework that translates specific types of classical specifications into quantum-executable programs while preserving methodological rigor. The framework applies modular software engineering principles by classifying the workflow into three core modules: an encoder that classifies problems, produces Quantum-Compatible Formats (QCFs), and constructs quantum circuits, a deployment module that generates circuits and recommends hardware based on fidelity, runtime, and cost, and a decoder that interprets quantum outputs into classical solutions. This architecture supports systematic evaluation across simulators and Noisy Intermediate-Scale Quantum (NISQ) quantum devices, remaining scalable to new problem classes and algorithms. In evaluation, the encoder module achieved a 93.8% completion rate, the hardware recommendation module consistently selected the appropriate quantum devices for workloads scaling up to 56 qubits. End-to-end experiments on 434 Python programs and 100 JSON problem instances demonstrate that the full C2 \(\lvert Q \rangle\) workflow executes reliably on simulators and can be deployed successfully on representative real quantum hardware, with empirical runs limited to small- and medium-sized instances consistent with current NISQ capabilities. A proxy-based usability analysis further indicates substantial reductions in handwritten lines of code and explicit configuration steps compared to conventional quantum SDK workflows. These results indicate that C2 \(\lvert Q \rangle\) lowers the entry barrier to quantum software development by providing a reproducible, extensible toolchain that connects classical specifications to quantum execution. The open-source implementation of C2 \(\lvert Q \rangle\) is available at https://github.com/C2-Q/C2Q and as a ready-to-use Python package at https://pypi.org/project/c2q-framework/ .

arXiv:2510.02854•ACM Transactions on Software Engineering and Methodology•Mar 25, 2026•1 citation

Exploring Developers Discussion Forums for Quantum Software Engineering: A Fine-Grained Classification Approach Using Large Language Model (ChatGPT)

Mobashir Husain, Muhammad Sohail Khan, Javed Ali Khan +3 more

Quantum Software Engineering (QSE) has recently emerged as a potential research and development direction frequently practiced by many tech joints. However, quantum developers face challenges in optimizing quantum computing and QSE concepts. Quantum developers use the Stack Overflow (SO) platform to report and discuss quantum-related challenges. Also, quantum practitioners use specialized quantum tags to label quantum-related posts in developers' forums. However, these quantum tags referred to more technical quantum aspects than the developer posts. Therefore, categorizing quantum practitioners' questions based on quantum concepts can help quantum developers better identify frequently occurring challenges to QSE. For this purpose, we conducted qualitative and quantitative studies to classify quantum developers' questions into various frequently occurring quantum-related challenges. We extracted 2829 developers' questions from various Q&A platforms using queries and filters based on quantum-related tags. Next, the developers' posts on the Q&A forums were critically analyzed to identify frequently discussed quantum-related challenges and develop a novel grounded theory. The frequent quantum developer challenges identified by analyzing practitioners' discussions in the Q&A forums include Tooling, Theoretical, Learning, Conceptual, Errors, and API Usage. Moreover, using content analysis and grounded theory, the developers' discussions were annotated with commonly reported quantum challenges to develop a ground truth and a novel dataset. A Large Language model (ChatGPT) was used to validate the human annotation and overcome disagreements. Finally, various fine-tuned Deep and Machine learning (D&ML) classifiers automatically classify developer discussions into commonly reported quantum challenges. Additionally, to improve the classification results, we utilized textual data augmentation approaches, such as random deletion, swapping, and insertion with the D&ML classifiers. We obtained average accuracies of 89%, 86%, 84%, 84%, and 80% with FNN, CNN, LSTM, GRU, and RNN classifiers, respectively. This helps quantum researchers and vendors propose solutions and tools to frequently occurring issues for quantum developers.

ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE)•Jun 23, 2025•5 citations

Mining Quantum Software Patterns in Open-Source Projects

Neilson Carlos Leite Ramalho, Erico A. da Silva, Higor Amario de Souza +1 more

Quantum computing has become an active research field in recent years, as its applications in fields such as cryptography, optimization, and materials science are promising. Along with these developments, challenges and opportunities exist in the field of Quantum Software Engineering, as the development of frameworks and higher-level abstractions has attracted practitioners from diverse backgrounds. Unlike initial quantum frameworks based on the circuit model, recent frameworks and libraries leverage higher-level abstractions for creating quantum programs. This paper presents an empirical study of 985 Jupyter Notebooks from 80 open-source projects to investigate how quantum patterns are applied in practice. Our work involved two main stages. First, we built a knowledge base from three quantum computing frameworks (Qiskit, PennyLane, and Classiq). This process led us to identify and document 9 new patterns that refine and extend the existing quantum computing pattern catalog. Second, we developed a reusable semantic search tool to automatically detect these patterns across our large-scale dataset, providing a practitioner-focused analysis. Our results show that developers use patterns in three levels: from foundational circuit utilities, to common algorithmic primitives (e.g., Amplitude Amplification), up to domain-specific applications for finance and optimization. This indicates a maturing field where developers are increasingly using high-level building blocks to solve real-world problems.

cs.SECross-listedarXiv:2601.06281•International Conference on Mining Software Repositories (MSR)•Jan 13, 2026•2 citations

Benchmarking the Benchmarks for Quantum Code Generation with LLMs

Yunhe Feng, Xiaokun Yang, Kewei Sha +1 more

Large Language Models (LLMs) are increasingly deployed as quantum coding copilots, assisting researchers by generating complex programs for frameworks such as Qiskit, PennyLane, and OpenQASM. However, accurately evaluating the practical utility of these models remains a significant challenge. Existing benchmarks are highly fragmented, ranging from syntax-centric unit tests to simulator- and hardware-aware metrics, making results difficult to compare across different languages and hardware targets. To address this fragmentation, this paper systematically reviews recent evaluation suites and post-training methodologies for quantum code generation. We distill current evaluation criteria into four practical dimensions: syntax and API validity, semantic and physical correctness, resource and hardware feasibility, and execution-based feedback. Building upon these dimensions, we present a unified taxonomy that maps current benchmarks to emerging model adaptation techniques, specifically retrieval-augmented generation (RAG), fine-tuning, multi-agent workflows, and reinforcement learning (RL). Our analysis highlights a clear shift: moving away from prompt-only baselines and toward agentic workflows driven by quantum-verifiable rewards. This shift directly targets the semantic gap between superficially compilable code and physically executable quantum programs. In addition, we identify key research opportunities-such as standardized semantic metrics and integration with quantum error correction-aimed at fostering a more reproducible, verifiable, and hardware-grounded ecosystem for automated quantum software development.

IEEE Dallas Circuits and Systems Conference (DCAS)•Apr 10, 2026•0 citations
Quantum software development kits (SDKs) expose high-level APIs whose use requires an understanding of underlying mathematical principles, circuit semantics, and hardware properties. Code comments can provide natural-language or structural explanations that are not explicit in API calls. However, studying comments in a new quantum SDK still requires researchers to coordinate separate steps for comment extraction, quantum-specific classification, and manual verification, resulting in repeated integration effort. To address this problem, we present CC4Q Workbench, an open-source workbench that integrates these steps into a reusable workflow. Given a Python repository at a specified revision, CC4Q Workbench automatically extracts code-comment pairs, segments comments into sentence-level units, and applies two trained classifiers to identify quantum-specific content and assign quantum-knowledge categories. Its web interface supports exploration of bundled datasets, inspection of code snippets, corpus construction from new repositories, manual annotation with model predictions hidden during binary labeling, and classifier fine-tuning using exported annotations. The release provides aggregate statistics for six SDK datasets comprising 39,593 sentence-level code comment units (SCCUs), of which 17,517 are browsable in the web interface. Applied to three additional application-layer Qiskit repositories outside the original CC4Q dataset, the pipeline ran without code changes and produced 11,517 SCCUs. Manual verification of 379 SCCUs showed 85.0% agreement between human annotations and the binary classifier's predictions.
Quantum computing is gaining attention from academia and industry. With the quantum Software Development Kits (SDKs), programmers can develop quantum software to explore the power of quantum computing. However, programmers may face challenges in understanding quantum software due to the non-intuitive quantum mechanics. To facilitate software development and maintenance, code comments offered in quantum SDKs serve as a natural language explanation of program functionalities and logical flows. Despite their importance, scarce research systematically reports their value and provides constructive guidelines for programmers. To address this gap, our paper focuses on Qiskit, one of the most popular quantum SDKs, and presents CC4Q, the first dataset of code comments for quantum computing. CC4Q incorporates 12,812 code comment pairs and 28,076 sentence-level code comment units, the latter of which involve heavy human annotation. Regarding the annotation, we validate the applicability of the developer-intent taxonomy used in classical programs, and also propose a new taxonomy considering quantum-specific knowledge. We conduct an empirical study comprehensively interpreting code comments from three perspectives: comment structure and coverage, developers’ intentions, and associated quantum topics. Our findings uncover key differences in code comments between classical and quantum software, and also outline quantum-specific knowledge relevant to quantum software development.
Quantum software development is challenged by quantum-mechanical semantics and fragmented software development kits (SDKs), which complicate cross-SDK code translation and execution. In this paper, we propose an LLM-based approach for translating executable programs between high-level quantum SDKs. We develop a quantum-aware translation workflow whose prompt design and validation reflect quantum semantics and accommodate the version evolution of quantum ecosystems. We conduct a preliminary evaluation of 4 LLM configurations on 43 tasks, translating Qiskit programs to PennyLane and Q#. LLMs achieve a zero-shot Pass@1 of up to 72.33%, while one-shot prompting further improves the translation effectiveness. These results suggest potential for quantum code translation by LLMs.
Quantum Software Engineering (QSE) is emerging as a critical discipline to make quantum computing accessible to a broader developer community; however, most quantum development environments still require developers to engage with low-level details across the software stack—including problem encoding, circuit construction, algorithm configuration, hardware selection, and result interpretation—making them difficult for classical software engineers to use. To bridge this gap, we present C2 ∣Q⟩\lvert Q \rangle∣Q⟩ , a hardware-agnostic quantum software development framework that translates specific types of classical specifications into quantum-executable programs while preserving methodological rigor. The framework applies modular software engineering principles by classifying the workflow into three core modules: an encoder that classifies problems, produces Quantum-Compatible Formats (QCFs), and constructs quantum circuits, a deployment module that generates circuits and recommends hardware based on fidelity, runtime, and cost, and a decoder that interprets quantum outputs into classical solutions. This architecture supports systematic evaluation across simulators and Noisy Intermediate-Scale Quantum (NISQ) quantum devices, remaining scalable to new problem classes and algorithms. In evaluation, the encoder module achieved a 93.8% completion rate, the hardware recommendation module consistently selected the appropriate quantum devices for workloads scaling up to 56 qubits. End-to-end experiments on 434 Python programs and 100 JSON problem instances demonstrate that the full C2 ∣Q⟩\lvert Q \rangle∣Q⟩ workflow executes reliably on simulators and can be deployed successfully on representative real quantum hardware, with empirical runs limited to small- and medium-sized instances consistent with current NISQ capabilities. A proxy-based usability analysis further indicates substantial reductions in handwritten lines of code and explicit configuration steps compared to conventional quantum SDK workflows. These results indicate that C2 ∣Q⟩\lvert Q \rangle∣Q⟩ lowers the entry barrier to quantum software development by providing a reproducible, extensible toolchain that connects classical specifications to quantum execution. The open-source implementation of C2 ∣Q⟩\lvert Q \rangle∣Q⟩ is available at https://github.com/C2-Q/C2Q and as a ready-to-use Python package at https://pypi.org/project/c2q-framework/ .
Quantum Software Engineering (QSE) has recently emerged as a potential research and development direction frequently practiced by many tech joints. However, quantum developers face challenges in optimizing quantum computing and QSE concepts. Quantum developers use the Stack Overflow (SO) platform to report and discuss quantum-related challenges. Also, quantum practitioners use specialized quantum tags to label quantum-related posts in developers' forums. However, these quantum tags referred to more technical quantum aspects than the developer posts. Therefore, categorizing quantum practitioners' questions based on quantum concepts can help quantum developers better identify frequently occurring challenges to QSE. For this purpose, we conducted qualitative and quantitative studies to classify quantum developers' questions into various frequently occurring quantum-related challenges. We extracted 2829 developers' questions from various Q&A platforms using queries and filters based on quantum-related tags. Next, the developers' posts on the Q&A forums were critically analyzed to identify frequently discussed quantum-related challenges and develop a novel grounded theory. The frequent quantum developer challenges identified by analyzing practitioners' discussions in the Q&A forums include Tooling, Theoretical, Learning, Conceptual, Errors, and API Usage. Moreover, using content analysis and grounded theory, the developers' discussions were annotated with commonly reported quantum challenges to develop a ground truth and a novel dataset. A Large Language model (ChatGPT) was used to validate the human annotation and overcome disagreements. Finally, various fine-tuned Deep and Machine learning (D&ML) classifiers automatically classify developer discussions into commonly reported quantum challenges. Additionally, to improve the classification results, we utilized textual data augmentation approaches, such as random deletion, swapping, and insertion with the D&ML classifiers. We obtained average accuracies of 89%, 86%, 84%, 84%, and 80% with FNN, CNN, LSTM, GRU, and RNN classifiers, respectively. This helps quantum researchers and vendors propose solutions and tools to frequently occurring issues for quantum developers.
Quantum computing has become an active research field in recent years, as its applications in fields such as cryptography, optimization, and materials science are promising. Along with these developments, challenges and opportunities exist in the field of Quantum Software Engineering, as the development of frameworks and higher-level abstractions has attracted practitioners from diverse backgrounds. Unlike initial quantum frameworks based on the circuit model, recent frameworks and libraries leverage higher-level abstractions for creating quantum programs. This paper presents an empirical study of 985 Jupyter Notebooks from 80 open-source projects to investigate how quantum patterns are applied in practice. Our work involved two main stages. First, we built a knowledge base from three quantum computing frameworks (Qiskit, PennyLane, and Classiq). This process led us to identify and document 9 new patterns that refine and extend the existing quantum computing pattern catalog. Second, we developed a reusable semantic search tool to automatically detect these patterns across our large-scale dataset, providing a practitioner-focused analysis. Our results show that developers use patterns in three levels: from foundational circuit utilities, to common algorithmic primitives (e.g., Amplitude Amplification), up to domain-specific applications for finance and optimization. This indicates a maturing field where developers are increasingly using high-level building blocks to solve real-world problems.
Large Language Models (LLMs) are increasingly deployed as quantum coding copilots, assisting researchers by generating complex programs for frameworks such as Qiskit, PennyLane, and OpenQASM. However, accurately evaluating the practical utility of these models remains a significant challenge. Existing benchmarks are highly fragmented, ranging from syntax-centric unit tests to simulator- and hardware-aware metrics, making results difficult to compare across different languages and hardware targets. To address this fragmentation, this paper systematically reviews recent evaluation suites and post-training methodologies for quantum code generation. We distill current evaluation criteria into four practical dimensions: syntax and API validity, semantic and physical correctness, resource and hardware feasibility, and execution-based feedback. Building upon these dimensions, we present a unified taxonomy that maps current benchmarks to emerging model adaptation techniques, specifically retrieval-augmented generation (RAG), fine-tuning, multi-agent workflows, and reinforcement learning (RL). Our analysis highlights a clear shift: moving away from prompt-only baselines and toward agentic workflows driven by quantum-verifiable rewards. This shift directly targets the semantic gap between superficially compilable code and physically executable quantum programs. In addition, we identify key research opportunities-such as standardized semantic metrics and integration with quantum error correction-aimed at fostering a more reproducible, verifiable, and hardware-grounded ecosystem for automated quantum software development.