Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE International Conference on Software Quality, Reliability and Security (QRS)

A Strategy of Dynamic Random Testing with Hybrid Distance Metrics for Quantum Programs

Lin Huang, Hanyu Pei, Yuechen Li, Beibei Yin, Kai‐Yuan Cai

Published Jul 1, 2024• 6 citations
Publisher

Abstract

Quantum Computing (QC) leverages quantum mechanics to manipulate quantum information, holding greater potential than classical computing. To fully exploit QC’s potential, it is crucial to ensure the reliability and quality of quantum programs. Research on quantum program testing is still at its early stage, in which some distinctive features of quantum programs, e.g., superposition and entanglement, may be overlooked, and the fault detection capability and testing effectiveness are rather limited. Besides, the input space of quantum programs may exponentially grow when the number of qubits increases, posing great challenges to testing quantum programs. It is imperative to develop a proper testing strategy to effectively select the potential failure-causing test cases and detect faults faster. In this paper, test cases with both basis states and superposition ones are considered and generated to cover more input space. A hybrid distance measurement method based on quantum fidelity and Hamming distance is presented for measuring the similarity among quantum test cases. Furthermore, a Dynamic Random Testing strategy based on Hybrid distance metrics (DRT-H) for quantum programs is proposed, which combines the hybrid distance metrics and the feedback mechanism of the classical Dynamic Random Testing (DRT) strategy to adjust the testing profile and guide the test case selection. Experimental studies demonstrate that the proposed DRT-H strategy outperforms the baseline testing strategies in most cases.

DOI
10.1109/qrs62785.2024.00011

Related papers

A Dynamic Testing Strategy With Incremental Learning Model for Quantum Programs

Lin Huang, Hanyu Pei, Yuechen Li +2 more

Quantum computing has emerged as an innovative computational paradigm with great potential in various domains. As quantum computing advances, the development of high-quality quantum programs has become crucial, making it essential to ensure their reliability. Software testing plays a vital role in achieving the reliability and quality of software systems. Various testing strategies and tools have been proposed for traditional programs; however, research on testing methodologies for quantum programs is still in the early stages. Traditional testing techniques, while effective for classical programs, struggle to address the unique challenges posed by quantum programs, including inherent characteristics of quantum systems (such as superposition and entanglement), and the exponentially expanding input space as the number of qubits increases. Moreover, traditional testing strategies typically do not account for the hidden and non-deterministic failure patterns associated with input quantum bits (qubits), which, if recognized, could potentially lead to more efficient fault detection. Therefore, in this paper, we introduce a novel approach—Quantum Dynamic-Testing with Incremental Learning (QDT-IL), which aims to effectively capture and adapt to the failure patterns in quantum programs. QDT-IL employs an incremental learning model to learn from executed test cases and continuously updates its predictions on the failure tendencies for new test cases. By utilizing distance-based diversity metrics, QDT-IL strategically increases the variety of test cases, enabling targeted exploration of failure-prone regions in the input space. This combination of adaptive learning and diverse test case selection noticeably enhances the effectiveness and efficiency of quantum program testing. Experimental studies show that QDT-IL outperforms the baseline strategies, providing a more effective testing process for quantum programs.

IEEE Transactions on Quantum Engineering•Oct 31, 2025•2 citations

Application of Combinatorial Testing to Quantum Programs

Xinyi Wang, Paolo Arcaini, Tao Yue +1 more

The capability of Quantum Computing (QC) in solving complex problems has been increasingly recognized. However, similar to classical computing, to fully exploit QC's potential, it is important to ensure the correctness of quantum programs. Doing so via software testing is, however, very challenging because of QC's inherent properties: superposition and entanglement. Towards the direction of ensuring the correctness of quantum programs, we propose an approach called QuCAT (QUantum CombinAtorial Testing) for systematic and automated testing of quantum programs by benefiting from combinatorial testing, which has been proven to be cost-effective in testing classical programs. QuCAT supports two combinatorial test suite generation scenarios, i.e., generating combinatorial test suites of a given strength, and incrementally generating and executing combinatorial test suites of increasing strength until a fault is found. The approach employs two types of test oracles to assess test results. We performed an empirical study with 18 faulty versions of quantum programs to evaluate QuCAT with strengths of two, three, and four in the two test generation scenarios. We compare the cost-effectiveness of combinatorial testing of various strengths and random testing (taken as baseline approach). Results show that combinatorial testing always performs better than random testing with the same cost and finds faults more quickly (in terms of required number of test cases). In addition, in most cases, combinatorial testing with a higher strength outperforms the lower strength in terms of effectiveness.

IEEE International Conference on Software Quality, Reliability and Security (QRS)•Dec 1, 2021•52 citations

QuraTest: Integrating Quantum Specific Features in Quantum Program Testing

Jiaming Ye, Shangzhou Xia, Fuyuan Zhang +4 more

The recent fast development of quantum computers breaks several computation limitations that are difficult for conventional computers. Up to the present, although many approaches and tools have been proposed to test quantum programs, the fundamental features of quantum programs, i.e., magnitude, phase, and entanglement, have been largely overlooked, leading to limited fault detection capability and reduced testing effectiveness. To address this problem, we propose an automated testing framework named QURATEST, equipped with three test case generators (including two newly proposed techniques, UCNOT and IQFT in this paper, as well as one based on Random techniques) to test quantum programs. Overall, the proposed generators enable the generation of diverse test inputs by considering the quantum features of quantum programs. In the experiments, we perform an in-depth evaluation of QURATEST from three aspects: generated test case diversity, output coverage of the program under test, and fault detection capability. The results demonstrate the potential of our newly proposed techniques in that IQFT can generate the most diverse test cases regarding magnitude, phase, and entanglement, with 66% cell coverage. Comparatively, the Random approach only has 10% cell coverage. Regarding the evaluations of the output coverage, IQFT can achieve the highest output coverage in 70.2% (33 out of 47) of all quantum programs. In terms of fault detection, UCNOT outperforms the other two techniques. Specifically, the test cases generated by UCNOT have the best mutation score in 88.4% (23 out of 26) quantum programs.

IEEE/ACM International Conference on Automated Software Engineering (ASE)•Sep 11, 2023•39 citations

NovaQ: Improving Quantum Program Testing through Diversity-Guided Test Case Generation

Tiancheng Jin, Shangzhou Xia, Jianjun Zhao

Quantum programs are designed to run on quantum computers, leveraging quantum circuits to solve problems that are intractable for classical machines. As quantum computing advances, ensuring the reliability of quantum programs has become increasingly important. This paper introduces NovaQ, a diversity-guided testing framework for quantum programs. NovaQ combines a distribution-based test case generator with a novelty-driven evaluation module. The generator produces diverse quantum state inputs by mutating circuit parameters, while the evaluator quantifies behavioral novelty based on internal circuit state metrics, including magnitude, phase, and entanglement. By selecting inputs that map to infrequently covered regions in the metric space, NovaQ effectively explores under-tested program behaviors. We evaluate NovaQ on quantum programs of varying sizes and complexities. Experimental results show that NovaQ consistently achieves higher test input diversity and detects more bugs than existing baseline approaches.

arXiv:2509.04763•IEEE/ACM International Conference on Automated Software Engineering (ASE)•Nov 16, 2025•8 citations

A Dynamic Test Oracle for Quantum Programs With Separable Output States

Yuechen Li, Kai-Yuan Cai, Beibei Yin

As quantum software engineering advances, testing techniques are required to assess the quality of quantum programs (QPs). In the test process, the test oracle is vital for determining whether the test result indicates a success or a failure. Most related works directly measure the output states and acquire the corresponding test results by comparing the output distribution with the expected one. While attention has been paid to the capability of fault detection, the guarantee for the correctness of the produced test results remains limited. Unlike classical programs (CPs), the output quantum states of QPs should be transformed into probabilistic classical outcomes through quantum measurement. This additional operation of measurement could cause a test oracle to yield the wrong test results. Especially for high-dimensional output spaces, numerous measurement outcomes are required to capture the distribution characteristics, threatening the effectiveness and cost-efficiency of test oracles. Hence, this paper proposes a novel specified test oracle DOSS employing a dynamic scheme to integrate a quantum algorithm (i.e., swap test) with the direct measurement mode. This innovative approach enables the validation of individual outputs rather than their distribution during the testing phase. Considering acceptable cost, DOSS decomposes the fully or partially separable output states to lower the dimensionality and simplify the quantum circuit for testing. Empirical studies demonstrate that DOSS generally gives more correct test results than baselines, and maintains reasonable cost on an ideal simulator. Besides, DOSS’s effectiveness with quantum noise involved is validated via three noisy simulators.

IEEE Transactions on Software Engineering•Mar 4, 2026•2 citations

Statistical Hypothesis Testing for Efficient Black-Box Quantum Program Testing

Tiancheng Jin, Bo Yang, Haibo Yu +1 more

Quantum computing offers exponential computational advantages across diverse application domains, yet the inherent probabilistic nature of quantum measurements poses challenges for program testing. Existing quantum black-box testing approaches rely on ad-hoc tolerance checks applied to empirical probability distributions estimated from large measurement samples, often requiring a large number of measurements, because of lacking formal Type I error control, power guarantees, or principled sample size determination. We present a statistical hypothesis-testing method for quantum program testing, encompassing two paradigms: marginal probability testing and joint distribution testing over the full$2^{d}$-dimensional state space. By combining exact and asymptotic tests, bootstrap procedures, sequential designs, and kernel-based two-sample testing, our method provides rigorous error control with substantially improved sample efficiency. Experiments on quantum programs with 2 to 12 qubits show measurement reductions from 97 to 99 percent over the conventional practice of ten thousand shots against strong synthetic faults. A structural mutation study separates the faults that every test detects at one hundred measurements from those that no measurement-based test can detect. The exact marginal test, the exact branch of the sequential design, and the permutation-calibrated joint tests control the Type I error rate at or below the nominal level of 0.05. The adaptive chi-square branch is mildly anticonservative on random states, with a median of 0.060, and conservative on the named benchmarks. On QFT, the sequential design rejects a faulty program with 32 to 85 percent fewer measurements than a properly powered fixed-sample test at two to twelve qubits on its adaptive branch, and with 19 to 60 percent fewer on its exact branch, which is valid under any dependence and reaches the target power from four qubits upward. Power exceeds 0.80 above a characterized fault strength, and faults that preserve the output distribution are shown to be undetectable by any measurement-based test. These results establish a principled and practical foundation for quantum program testing.

IEEE Access•Jan 1, 2026•0 citations
Quantum Computing (QC) leverages quantum mechanics to manipulate quantum information, holding greater potential than classical computing. To fully exploit QC’s potential, it is crucial to ensure the reliability and quality of quantum programs. Research on quantum program testing is still at its early stage, in which some distinctive features of quantum programs, e.g., superposition and entanglement, may be overlooked, and the fault detection capability and testing effectiveness are rather limited. Besides, the input space of quantum programs may exponentially grow when the number of qubits increases, posing great challenges to testing quantum programs. It is imperative to develop a proper testing strategy to effectively select the potential failure-causing test cases and detect faults faster. In this paper, test cases with both basis states and superposition ones are considered and generated to cover more input space. A hybrid distance measurement method based on quantum fidelity and Hamming distance is presented for measuring the similarity among quantum test cases. Furthermore, a Dynamic Random Testing strategy based on Hybrid distance metrics (DRT-H) for quantum programs is proposed, which combines the hybrid distance metrics and the feedback mechanism of the classical Dynamic Random Testing (DRT) strategy to adjust the testing profile and guide the test case selection. Experimental studies demonstrate that the proposed DRT-H strategy outperforms the baseline testing strategies in most cases.
Quantum computing has emerged as an innovative computational paradigm with great potential in various domains. As quantum computing advances, the development of high-quality quantum programs has become crucial, making it essential to ensure their reliability. Software testing plays a vital role in achieving the reliability and quality of software systems. Various testing strategies and tools have been proposed for traditional programs; however, research on testing methodologies for quantum programs is still in the early stages. Traditional testing techniques, while effective for classical programs, struggle to address the unique challenges posed by quantum programs, including inherent characteristics of quantum systems (such as superposition and entanglement), and the exponentially expanding input space as the number of qubits increases. Moreover, traditional testing strategies typically do not account for the hidden and non-deterministic failure patterns associated with input quantum bits (qubits), which, if recognized, could potentially lead to more efficient fault detection. Therefore, in this paper, we introduce a novel approach—Quantum Dynamic-Testing with Incremental Learning (QDT-IL), which aims to effectively capture and adapt to the failure patterns in quantum programs. QDT-IL employs an incremental learning model to learn from executed test cases and continuously updates its predictions on the failure tendencies for new test cases. By utilizing distance-based diversity metrics, QDT-IL strategically increases the variety of test cases, enabling targeted exploration of failure-prone regions in the input space. This combination of adaptive learning and diverse test case selection noticeably enhances the effectiveness and efficiency of quantum program testing. Experimental studies show that QDT-IL outperforms the baseline strategies, providing a more effective testing process for quantum programs.
The capability of Quantum Computing (QC) in solving complex problems has been increasingly recognized. However, similar to classical computing, to fully exploit QC's potential, it is important to ensure the correctness of quantum programs. Doing so via software testing is, however, very challenging because of QC's inherent properties: superposition and entanglement. Towards the direction of ensuring the correctness of quantum programs, we propose an approach called QuCAT (QUantum CombinAtorial Testing) for systematic and automated testing of quantum programs by benefiting from combinatorial testing, which has been proven to be cost-effective in testing classical programs. QuCAT supports two combinatorial test suite generation scenarios, i.e., generating combinatorial test suites of a given strength, and incrementally generating and executing combinatorial test suites of increasing strength until a fault is found. The approach employs two types of test oracles to assess test results. We performed an empirical study with 18 faulty versions of quantum programs to evaluate QuCAT with strengths of two, three, and four in the two test generation scenarios. We compare the cost-effectiveness of combinatorial testing of various strengths and random testing (taken as baseline approach). Results show that combinatorial testing always performs better than random testing with the same cost and finds faults more quickly (in terms of required number of test cases). In addition, in most cases, combinatorial testing with a higher strength outperforms the lower strength in terms of effectiveness.
The recent fast development of quantum computers breaks several computation limitations that are difficult for conventional computers. Up to the present, although many approaches and tools have been proposed to test quantum programs, the fundamental features of quantum programs, i.e., magnitude, phase, and entanglement, have been largely overlooked, leading to limited fault detection capability and reduced testing effectiveness. To address this problem, we propose an automated testing framework named QURATEST, equipped with three test case generators (including two newly proposed techniques, UCNOT and IQFT in this paper, as well as one based on Random techniques) to test quantum programs. Overall, the proposed generators enable the generation of diverse test inputs by considering the quantum features of quantum programs. In the experiments, we perform an in-depth evaluation of QURATEST from three aspects: generated test case diversity, output coverage of the program under test, and fault detection capability. The results demonstrate the potential of our newly proposed techniques in that IQFT can generate the most diverse test cases regarding magnitude, phase, and entanglement, with 66% cell coverage. Comparatively, the Random approach only has 10% cell coverage. Regarding the evaluations of the output coverage, IQFT can achieve the highest output coverage in 70.2% (33 out of 47) of all quantum programs. In terms of fault detection, UCNOT outperforms the other two techniques. Specifically, the test cases generated by UCNOT have the best mutation score in 88.4% (23 out of 26) quantum programs.
Quantum programs are designed to run on quantum computers, leveraging quantum circuits to solve problems that are intractable for classical machines. As quantum computing advances, ensuring the reliability of quantum programs has become increasingly important. This paper introduces NovaQ, a diversity-guided testing framework for quantum programs. NovaQ combines a distribution-based test case generator with a novelty-driven evaluation module. The generator produces diverse quantum state inputs by mutating circuit parameters, while the evaluator quantifies behavioral novelty based on internal circuit state metrics, including magnitude, phase, and entanglement. By selecting inputs that map to infrequently covered regions in the metric space, NovaQ effectively explores under-tested program behaviors. We evaluate NovaQ on quantum programs of varying sizes and complexities. Experimental results show that NovaQ consistently achieves higher test input diversity and detects more bugs than existing baseline approaches.
As quantum software engineering advances, testing techniques are required to assess the quality of quantum programs (QPs). In the test process, the test oracle is vital for determining whether the test result indicates a success or a failure. Most related works directly measure the output states and acquire the corresponding test results by comparing the output distribution with the expected one. While attention has been paid to the capability of fault detection, the guarantee for the correctness of the produced test results remains limited. Unlike classical programs (CPs), the output quantum states of QPs should be transformed into probabilistic classical outcomes through quantum measurement. This additional operation of measurement could cause a test oracle to yield the wrong test results. Especially for high-dimensional output spaces, numerous measurement outcomes are required to capture the distribution characteristics, threatening the effectiveness and cost-efficiency of test oracles. Hence, this paper proposes a novel specified test oracle DOSS employing a dynamic scheme to integrate a quantum algorithm (i.e., swap test) with the direct measurement mode. This innovative approach enables the validation of individual outputs rather than their distribution during the testing phase. Considering acceptable cost, DOSS decomposes the fully or partially separable output states to lower the dimensionality and simplify the quantum circuit for testing. Empirical studies demonstrate that DOSS generally gives more correct test results than baselines, and maintains reasonable cost on an ideal simulator. Besides, DOSS’s effectiveness with quantum noise involved is validated via three noisy simulators.
Quantum computing offers exponential computational advantages across diverse application domains, yet the inherent probabilistic nature of quantum measurements poses challenges for program testing. Existing quantum black-box testing approaches rely on ad-hoc tolerance checks applied to empirical probability distributions estimated from large measurement samples, often requiring a large number of measurements, because of lacking formal Type I error control, power guarantees, or principled sample size determination. We present a statistical hypothesis-testing method for quantum program testing, encompassing two paradigms: marginal probability testing and joint distribution testing over the full2d2^{d}2d-dimensional state space. By combining exact and asymptotic tests, bootstrap procedures, sequential designs, and kernel-based two-sample testing, our method provides rigorous error control with substantially improved sample efficiency. Experiments on quantum programs with 2 to 12 qubits show measurement reductions from 97 to 99 percent over the conventional practice of ten thousand shots against strong synthetic faults. A structural mutation study separates the faults that every test detects at one hundred measurements from those that no measurement-based test can detect. The exact marginal test, the exact branch of the sequential design, and the permutation-calibrated joint tests control the Type I error rate at or below the nominal level of 0.05. The adaptive chi-square branch is mildly anticonservative on random states, with a median of 0.060, and conservative on the named benchmarks. On QFT, the sequential design rejects a faulty program with 32 to 85 percent fewer measurements than a properly powered fixed-sample test at two to twelve qubits on its adaptive branch, and with 19 to 60 percent fewer on its exact branch, which is valid under any dependence and reaches the target power from four qubits upward. Power exceeds 0.80 above a characterized fault strength, and faults that preserve the output distribution are shown to be undetectable by any measurement-based test. These results establish a principled and practical foundation for quantum program testing.