Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2405.15450 · ACM Transactions on Software Engineering and Methodology

Faster and Better Quantum Software Testing through Specification Reduction and Projective Measurements

Noah H. Oldfield, Christoph Laaber, Tao Yue, Shaukat Ali

Published Jan 22, 2025• 11 citations
arXivHTMLPDF

Abstract

Quantum computing (QC) promises polynomial and exponential speedups in many domains, such as unstructured search and prime number factoring. However, quantum programs yield probabilistic outputs from exponentially growing distributions and are vulnerable to quantum-specific faults. Existing quantum software testing (QST) approaches treat quantum superpositions as classical distributions. This leads to two major limitations when applied to quantum programs: (1) an exponentially growing sample space distribution and (2) failing to detect quantum-specific faults such as phase flips. To overcome these limitations, we introduce a QST approach, which applies a reduction algorithm to a quantum program specification. The reduced specification alleviates the limitations (1) by enabling faster sampling through quantum parallelism and (2) by performing projective measurements in the mixed Hadamard basis. Our evaluation of 143 quantum programs across four categories demonstrates significant improvements in test runtimes and fault detection with our reduction approach. Average test runtimes improved from 169.9 s to 11.8 s, with notable enhancements in programs with large circuit depths (383.1 s to 33.4 s) and large program specifications (464.8 s to 7.7 s). Furthermore, our approach increases mutation scores from \(54.5\%\) to \(74.7\%\) , effectively detecting phase flip faults that non-reduced specifications miss. These results underline our approach's importance to improve QST efficiency and effectiveness.

DOI
10.1145/3714468

Related papers

Statistical Hypothesis Testing for Efficient Black-Box Quantum Program Testing

Tiancheng Jin, Bo Yang, Haibo Yu +1 more

Quantum computing offers exponential computational advantages across diverse application domains, yet the inherent probabilistic nature of quantum measurements poses challenges for program testing. Existing quantum black-box testing approaches rely on ad-hoc tolerance checks applied to empirical probability distributions estimated from large measurement samples, often requiring a large number of measurements, because of lacking formal Type I error control, power guarantees, or principled sample size determination. We present a statistical hypothesis-testing method for quantum program testing, encompassing two paradigms: marginal probability testing and joint distribution testing over the full$2^{d}$-dimensional state space. By combining exact and asymptotic tests, bootstrap procedures, sequential designs, and kernel-based two-sample testing, our method provides rigorous error control with substantially improved sample efficiency. Experiments on quantum programs with 2 to 12 qubits show measurement reductions from 97 to 99 percent over the conventional practice of ten thousand shots against strong synthetic faults. A structural mutation study separates the faults that every test detects at one hundred measurements from those that no measurement-based test can detect. The exact marginal test, the exact branch of the sequential design, and the permutation-calibrated joint tests control the Type I error rate at or below the nominal level of 0.05. The adaptive chi-square branch is mildly anticonservative on random states, with a median of 0.060, and conservative on the named benchmarks. On QFT, the sequential design rejects a faulty program with 32 to 85 percent fewer measurements than a properly powered fixed-sample test at two to twelve qubits on its adaptive branch, and with 19 to 60 percent fewer on its exact branch, which is valid under any dependence and reaches the target power from four qubits upward. Power exceeds 0.80 above a characterized fault strength, and faults that preserve the output distribution are shown to be undetectable by any measurement-based test. These results establish a principled and practical foundation for quantum program testing.

IEEE Access•Jan 1, 2026•0 citations

Benchmarking Quantum Software Testing with Scalable Quantum Programs

Yuechen Li, Minqi Shao, Xiyuan Li +2 more

Quantum software testing (QST) checks whether quantum programs behave according to their intended specifications. A key requirement for QST research is a benchmark that supports rigorous empirical evaluation on programs that are testable and better reflect current software development practices. However, existing studies heavily rely on small hard-coded or circuit-level benchmarks, while available quantum programs are scattered across repositories without clear selection criteria, which limits fair comparison and systematic reproducibility. To this end, we present Qolumbina, a benchmark infrastructure for controlled QST experiments on scalable quantum programs. Qolumbina curates 40 programs from open-source repositories, turns them into test-ready subjects through systematic selection, refactoring, specifications, test case examples, unit tests, and standardized interfaces. We also propose QST-oriented criteria to characterize quantum programs along functionality, output behavior, development complexity, and quantum-specific execution complexity. Using these criteria, our empirical study shows that Qolumbina covers diverse testing-relevant properties and supports scalability analysis beyond fixed-size circuit benchmarks. Through controlled experiments with two recent QST approaches, we demonstrate the feasibility of using Qolumbina for execution-cost and fault-detection studies, and highlight backend-dependent effects that can influence QST result interpretation.

cs.SECross-listedarXiv:2607.02029•Jul 3, 2026•3 citations

Search-Based Quantum Program Testing via Commuting Pauli String

Asmar Muqeet, Shaukat Ali, Paolo Arcaini

Quantum software testing is important for reliable quantum software engineering. Despite recent advances, existing quantum software testing approaches rely on simple test inputs and statistical oracles, costly program specifications, and limited validation on real quantum computers. To address these challenges, we propose SB-QOPS, a search-based quantum program testing approach via commuting Pauli strings. SB-QOPS, as a direct extension to a previously proposed QOPS approach, redefines test cases in terms of Pauli strings and introduces a measurement-centric oracle that exploits their commutation properties, enabling effective testing of quantum programs while reducing the need for full program specifications. By systematically exploring the search space through an expectation-value-based fitness function, SB-QOPS improves test budget utilization and increases the likelihood of uncovering subtle faults. We conduct a large-scale empirical evaluation on quantum circuits of up to 29 qubits on real quantum computers and emulators. We assess three search strategies: Genetic Algorithm, Hill Climbing, and the (1+1) Evolutionary Algorithm, and evaluate SB-QOPS under both simulated and real noisy conditions. Experiments span three quantum computing platforms: IBM, IQM, and Quantinuum. Results show that SB-QOPS significantly outperforms QOPS, achieving a fault-detection score of 100% for circuits up to 29 qubits, and demonstrating portability across quantum platforms.

arXiv:2602.11487•Feb 12, 2026•0 citations

QuraTest: Integrating Quantum Specific Features in Quantum Program Testing

Jiaming Ye, Shangzhou Xia, Fuyuan Zhang +4 more

The recent fast development of quantum computers breaks several computation limitations that are difficult for conventional computers. Up to the present, although many approaches and tools have been proposed to test quantum programs, the fundamental features of quantum programs, i.e., magnitude, phase, and entanglement, have been largely overlooked, leading to limited fault detection capability and reduced testing effectiveness. To address this problem, we propose an automated testing framework named QURATEST, equipped with three test case generators (including two newly proposed techniques, UCNOT and IQFT in this paper, as well as one based on Random techniques) to test quantum programs. Overall, the proposed generators enable the generation of diverse test inputs by considering the quantum features of quantum programs. In the experiments, we perform an in-depth evaluation of QURATEST from three aspects: generated test case diversity, output coverage of the program under test, and fault detection capability. The results demonstrate the potential of our newly proposed techniques in that IQFT can generate the most diverse test cases regarding magnitude, phase, and entanglement, with 66% cell coverage. Comparatively, the Random approach only has 10% cell coverage. Regarding the evaluations of the output coverage, IQFT can achieve the highest output coverage in 70.2% (33 out of 47) of all quantum programs. In terms of fault detection, UCNOT outperforms the other two techniques. Specifically, the test cases generated by UCNOT have the best mutation score in 88.4% (23 out of 26) quantum programs.

IEEE/ACM International Conference on Automated Software Engineering (ASE)•Sep 11, 2023•39 citations

QuCheck: A Property-based Testing Framework for Quantum Programs in Qiskit

Gabriel Pontolillo, Mohammad Reza Mousavi, Marek Grzesiuk

Property-based testing has been previously proposed for quantum programs in Q# with QSharpCheck; however, this implementation was limited in functionality, lacked extensibility, and was evaluated on a narrow range of programs using a single property. To address these limitations, we propose QuCheck, an improved property-based testing framework for Qiskit. By leveraging Qiskit and the broader Python ecosystem, QuCheck facilitates property construction, introduces flexible input generators and assertions, and supports expressive preconditions. We assessed its effectiveness through mutation analysis on five quantum programs (2-10 qubits), varying the number of properties, inputs, and measurement shots to assess their impact on fault detection and demonstrate the effectiveness of property-based testing across a range of conditions. Results show a strong positive correlation between the mutation score (a measure of fault detection) and number of properties evaluated, with a moderate negative correlation between the false positive rate and number of measurement shots. The most thorough test configurations achieved a mean mutation score of 0.90 averaged across all five algorithms, with a false positive rate between 0 and 0.06. QuCheck identified 47.8% more faults than QSharpCheck, with execution time reduced by 67.4%. These findings highlight the viability of property-based testing for verifying quantum systems.

arXiv:2503.22641•ACM Transactions on Quantum Computing•May 8, 2026•17 citations

Quantum Squeeziness: Information-Theoretic Foundations of Quantum Software Testability

Avner Bensoussan, Héctor D. Menéndez, Mohammadreza Ahmadi Mousavi

Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability. We propose an efficient method to compute Quantum Squeeziness and validate it through large-scale quantum mutation analysis. Across more than 50k mutants, we observe a strong negative correlation between squeeziness and mutation score, showing that higher squeeziness predicts lower testability. We further introduce a technique that constrains where and how perturbations are injected, reducing cost while preserving accuracy. The computation is 15% faster than a standard error-propagation baseline, while the optimised variant, FastSqueeziness, reduces mean runtime from 3200 s to 189 s (16.9×speedup) with 95% accuracy. By establishing a rigorous foundation of testability for quantum software, this work opens new directions at the intersection of software engineering, quantum information theory, and testing. Beyond testing and debugging, it also supports applications in quantum error mitigation and correction, enabling principled assessment and improvement of program robustness.

•Jul 16, 2026•0 citations
Quantum computing (QC) promises polynomial and exponential speedups in many domains, such as unstructured search and prime number factoring. However, quantum programs yield probabilistic outputs from exponentially growing distributions and are vulnerable to quantum-specific faults. Existing quantum software testing (QST) approaches treat quantum superpositions as classical distributions. This leads to two major limitations when applied to quantum programs: (1) an exponentially growing sample space distribution and (2) failing to detect quantum-specific faults such as phase flips. To overcome these limitations, we introduce a QST approach, which applies a reduction algorithm to a quantum program specification. The reduced specification alleviates the limitations (1) by enabling faster sampling through quantum parallelism and (2) by performing projective measurements in the mixed Hadamard basis. Our evaluation of 143 quantum programs across four categories demonstrates significant improvements in test runtimes and fault detection with our reduction approach. Average test runtimes improved from 169.9 s to 11.8 s, with notable enhancements in programs with large circuit depths (383.1 s to 33.4 s) and large program specifications (464.8 s to 7.7 s). Furthermore, our approach increases mutation scores from (54.5%) to (74.7%) , effectively detecting phase flip faults that non-reduced specifications miss. These results underline our approach's importance to improve QST efficiency and effectiveness.
Quantum computing offers exponential computational advantages across diverse application domains, yet the inherent probabilistic nature of quantum measurements poses challenges for program testing. Existing quantum black-box testing approaches rely on ad-hoc tolerance checks applied to empirical probability distributions estimated from large measurement samples, often requiring a large number of measurements, because of lacking formal Type I error control, power guarantees, or principled sample size determination. We present a statistical hypothesis-testing method for quantum program testing, encompassing two paradigms: marginal probability testing and joint distribution testing over the full2d2^{d}2d-dimensional state space. By combining exact and asymptotic tests, bootstrap procedures, sequential designs, and kernel-based two-sample testing, our method provides rigorous error control with substantially improved sample efficiency. Experiments on quantum programs with 2 to 12 qubits show measurement reductions from 97 to 99 percent over the conventional practice of ten thousand shots against strong synthetic faults. A structural mutation study separates the faults that every test detects at one hundred measurements from those that no measurement-based test can detect. The exact marginal test, the exact branch of the sequential design, and the permutation-calibrated joint tests control the Type I error rate at or below the nominal level of 0.05. The adaptive chi-square branch is mildly anticonservative on random states, with a median of 0.060, and conservative on the named benchmarks. On QFT, the sequential design rejects a faulty program with 32 to 85 percent fewer measurements than a properly powered fixed-sample test at two to twelve qubits on its adaptive branch, and with 19 to 60 percent fewer on its exact branch, which is valid under any dependence and reaches the target power from four qubits upward. Power exceeds 0.80 above a characterized fault strength, and faults that preserve the output distribution are shown to be undetectable by any measurement-based test. These results establish a principled and practical foundation for quantum program testing.
Quantum software testing (QST) checks whether quantum programs behave according to their intended specifications. A key requirement for QST research is a benchmark that supports rigorous empirical evaluation on programs that are testable and better reflect current software development practices. However, existing studies heavily rely on small hard-coded or circuit-level benchmarks, while available quantum programs are scattered across repositories without clear selection criteria, which limits fair comparison and systematic reproducibility. To this end, we present Qolumbina, a benchmark infrastructure for controlled QST experiments on scalable quantum programs. Qolumbina curates 40 programs from open-source repositories, turns them into test-ready subjects through systematic selection, refactoring, specifications, test case examples, unit tests, and standardized interfaces. We also propose QST-oriented criteria to characterize quantum programs along functionality, output behavior, development complexity, and quantum-specific execution complexity. Using these criteria, our empirical study shows that Qolumbina covers diverse testing-relevant properties and supports scalability analysis beyond fixed-size circuit benchmarks. Through controlled experiments with two recent QST approaches, we demonstrate the feasibility of using Qolumbina for execution-cost and fault-detection studies, and highlight backend-dependent effects that can influence QST result interpretation.
Quantum software testing is important for reliable quantum software engineering. Despite recent advances, existing quantum software testing approaches rely on simple test inputs and statistical oracles, costly program specifications, and limited validation on real quantum computers. To address these challenges, we propose SB-QOPS, a search-based quantum program testing approach via commuting Pauli strings. SB-QOPS, as a direct extension to a previously proposed QOPS approach, redefines test cases in terms of Pauli strings and introduces a measurement-centric oracle that exploits their commutation properties, enabling effective testing of quantum programs while reducing the need for full program specifications. By systematically exploring the search space through an expectation-value-based fitness function, SB-QOPS improves test budget utilization and increases the likelihood of uncovering subtle faults. We conduct a large-scale empirical evaluation on quantum circuits of up to 29 qubits on real quantum computers and emulators. We assess three search strategies: Genetic Algorithm, Hill Climbing, and the (1+1) Evolutionary Algorithm, and evaluate SB-QOPS under both simulated and real noisy conditions. Experiments span three quantum computing platforms: IBM, IQM, and Quantinuum. Results show that SB-QOPS significantly outperforms QOPS, achieving a fault-detection score of 100% for circuits up to 29 qubits, and demonstrating portability across quantum platforms.
The recent fast development of quantum computers breaks several computation limitations that are difficult for conventional computers. Up to the present, although many approaches and tools have been proposed to test quantum programs, the fundamental features of quantum programs, i.e., magnitude, phase, and entanglement, have been largely overlooked, leading to limited fault detection capability and reduced testing effectiveness. To address this problem, we propose an automated testing framework named QURATEST, equipped with three test case generators (including two newly proposed techniques, UCNOT and IQFT in this paper, as well as one based on Random techniques) to test quantum programs. Overall, the proposed generators enable the generation of diverse test inputs by considering the quantum features of quantum programs. In the experiments, we perform an in-depth evaluation of QURATEST from three aspects: generated test case diversity, output coverage of the program under test, and fault detection capability. The results demonstrate the potential of our newly proposed techniques in that IQFT can generate the most diverse test cases regarding magnitude, phase, and entanglement, with 66% cell coverage. Comparatively, the Random approach only has 10% cell coverage. Regarding the evaluations of the output coverage, IQFT can achieve the highest output coverage in 70.2% (33 out of 47) of all quantum programs. In terms of fault detection, UCNOT outperforms the other two techniques. Specifically, the test cases generated by UCNOT have the best mutation score in 88.4% (23 out of 26) quantum programs.
Property-based testing has been previously proposed for quantum programs in Q# with QSharpCheck; however, this implementation was limited in functionality, lacked extensibility, and was evaluated on a narrow range of programs using a single property. To address these limitations, we propose QuCheck, an improved property-based testing framework for Qiskit. By leveraging Qiskit and the broader Python ecosystem, QuCheck facilitates property construction, introduces flexible input generators and assertions, and supports expressive preconditions. We assessed its effectiveness through mutation analysis on five quantum programs (2-10 qubits), varying the number of properties, inputs, and measurement shots to assess their impact on fault detection and demonstrate the effectiveness of property-based testing across a range of conditions. Results show a strong positive correlation between the mutation score (a measure of fault detection) and number of properties evaluated, with a moderate negative correlation between the false positive rate and number of measurement shots. The most thorough test configurations achieved a mean mutation score of 0.90 averaged across all five algorithms, with a false positive rate between 0 and 0.06. QuCheck identified 47.8% more faults than QSharpCheck, with execution time reduced by 67.4%. These findings highlight the viability of property-based testing for verifying quantum systems.
Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability. We propose an efficient method to compute Quantum Squeeziness and validate it through large-scale quantum mutation analysis. Across more than 50k mutants, we observe a strong negative correlation between squeeziness and mutation score, showing that higher squeeziness predicts lower testability. We further introduce a technique that constrains where and how perturbations are injected, reducing cost while preserving accuracy. The computation is 15% faster than a standard error-propagation baseline, while the optimised variant, FastSqueeziness, reduces mean runtime from 3200 s to 189 s (16.9×speedup) with 95% accuracy. By establishing a rigorous foundation of testability for quantum software, this work opens new directions at the intersection of software engineering, quantum information theory, and testing. Beyond testing and debugging, it also supports applications in quantum error mitigation and correction, enabling principled assessment and improvement of program robustness.