Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)

Cross-Platform Testing of Quantum Computing Platforms

Matteo Paltenghi

Published May 1, 2022• 6 citations
Publisher

Abstract

Quantum computing has been attracting the attention of both applied research and companies. Continuous progress on fundamental hardware technology promises to bring us more reliable and large-scale quantum computers on which to run the next generation of quantum algorithms. These programs are compiled and executed on dedicated platforms, and similarly to classical programs, a large effort is required to test these platforms and create a robust software toolchain.Unlike previous studies which focused on cross-optimization and cross-backend testing, this dissertation aims to create the first approach for cross-platform testing which compares execution on diverse quantum computing platforms. To inform the design of the method, we will first perform an empirical study of bugs in quantum computing platforms and a review of the characteristics of realistic quantum programs.The final approach for cross-platform testing will include three components: a learning-based method to generate realistic quantum programs, an approach to map and run them on multiple platforms, and finally a quantum-specific statistical test to compare two multivariate binary distributions returned as the output of quantum programs.

DOI
10.1145/3510454.3517061

Related papers

Bugs in Quantum computing platforms: an empirical study

Matteo Paltenghi, Michael Pradel

The interest in quantum computing is growing, and with it, the importance of software platforms to develop quantum programs. Ensuring the correctness of such platforms is important, and it requires a thorough understanding of the bugs they typically suffer from. To address this need, this paper presents the first in-depth study of bugs in quantum computing platforms. We gather and inspect a set of 223 real-world bugs from 18 open-source quantum computing platforms. Our study shows that a significant fraction of these bugs (39.9%) are quantum-specific, calling for dedicated approaches to prevent and find them. The bugs are spread across various components, but quantum-specific bugs occur particularly often in components that represent, compile, and optimize quantum programming abstractions. Many quantum-specific bugs manifest through unexpected outputs, rather than more obvious signs of misbehavior, such as crashes. Finally, we present a hierarchy of recurrent bug patterns, including ten novel, quantum-specific patterns. Our findings not only show the importance and prevalence bugs in quantum computing platforms, but they help developers to avoid common mistakes and tool builders to tackle the challenge of preventing, finding, and fixing these bugs.

arXiv:2110.14560•Proceedings of the ACM on Programming Languages•Apr 29, 2022•109 citations

IterTestQ: Assembly-Level, Cross-Platform Testing of Quantum Computing Platforms

Matteo Paltenghi, Michael Pradel

Quantum computing platforms are susceptible to quantum-specific bugs, such as incorrectly ordering qubits or incorrectly implementing quantum abstractions. These bugs are difficult to detect and require specialized expertise. The field faces challenges due to a fragmented landscape of platforms and rapid development cycles that often prioritize new features over thorough testing, severely hindering the reliability of quantum software. To address these challenges, we present IterTestQ, a novel cross-platform testing approach for quantum computing platforms. The key technical contribution is our novel ITE process, which generates equivalent quantum programs by iteratively (I)mporting them into platform-specific representations, (T)ransforming the program via optimizations and gate conversions, and (E)xporting the program again. To transfer programs across platforms and test cross-platform consistency, IterTestQ leverages QASM, an assembly-level representation supported by most platforms. The approach uses a crash oracle to detect failures during cross-platform transformations and an equivalence oracle to validate the semantic consistency of the generated assembly programs, which are expected to be equivalent by construction. We evaluate IterTestQ on widely-used quantum computing platforms, including Qiskit, PennyLane, Pytket, BQSKit, and Cirq, revealing 23 bugs, 17 of which are already confirmed or fixed. Our results also demonstrate that IterTestQ complements existing quantum fuzzers (covering tens of thousands of otherwise uncovered lines), is efficient (with 0.00089 seconds per generated program), and that the ITE process is crucial for its effectiveness.

cs.SECross-listedarXiv:2503.17322•Proceedings of the ACM on software engineering.•Mar 24, 2025•8 citations

A Survey on Testing and Analysis of Quantum Software

Matteo Paltenghi, Michael Pradel

Quantum computing is getting increasing interest from both academia and industry, and the quantum software landscape has been growing rapidly. The quantum software stack comprises quantum programs, implementing algorithms, and platforms like IBM Qiskit, Google Cirq, and Microsoft Q#, enabling their development. To ensure the reliability and performance of quantum software, various techniques for testing and analyzing it have been proposed, such as test generation, bug pattern detection, and circuit optimization. However, the large amount of work and the fact that work on quantum software is performed by several research communities, make it difficult to get a comprehensive overview of the existing techniques. In this work, we provide an extensive survey of the state of the art in testing and analysis of quantum software. We discuss literature from several research communities, including quantum computing, software engineering, programming languages, and formal methods. Our survey covers a wide range of topics, including expected and unexpected behavior of quantum programs, testing techniques, program analysis approaches, optimizations, and benchmarks for testing and analyzing quantum software. We create novel connections between the discussed topics and present them in an accessible way. Finally, we discuss key challenges and open problems to inspire future research.

arXiv:2410.00650•Oct 1, 2024•30 citations

Regression testing of an evolving quantum platform

Linsey Kitt

Quantum computing is a field that has drawn a lot of attention over the years as it has shown great potential to solve new problems and has seen rapid development. To meet the demands of this rapid development, software engineering researchers have been studying methods to address the unique challenges presented by the probabilistic nature of quantum computing in its software platforms. One important aspect of this is understanding how to test and validate quantum software platforms when the expected outputs when running a program are not always known. Recent research using metamorphic testing was able to detect 13 faults in the Qiskit quantum computing platform; however, this is a rapidly evolving platform. Since their work, Qiskit has already advanced ahead several versions, which brings up more questions regarding research for quantum software platforms: do these tools researchers create hold up across versions of these frequently-updated platforms, and how much work does it take to maintain and extend these tools? In this thesis, we study the ease of regression testing in the context of this quantum computing platform. We update to the latest version of Qiskit, extend two existing metamorphic transformations on a metamorphic testing platform for Qiskit, add six new metamorphic transformations, and perform a set of metamorphic testing runs to analyze how our changes perform. In doing so, we were able to discover five additional new faults on the Qiskit platform; however, we also uncovered several challenges, which we synthesize for the reader. We also offer discussion on where the future work of metamorphic testing on quantum software platforms may lead based on our challenges and successes during this process.

•Jan 1, 2024•0 citations

Quantum Computing Architecture: Integration and Optimization Across the Stack

Kaiwen Gui

Quantum computing is a revolutionary approach to computation that leverages the principles of quantum mechanics, enabling computers to process information in fundamentally different ways than classical computers. Unlike traditional bits, which represent data as either 0s or 1s, quantum bits, or qubits, can exist in multiple states simultaneously due to superposition. Additionally, qubits can be entangled, allowing them to be interconnected in ways that enhance computational power and efficiency. This unique behavior has the potential to run certain types of computation tasks much faster than classical computers can, marking a significant leap forward in technology. When building actual quantum computers to run practical quantum algorithms, one needs efficient ways to map the algorithms to the hardware components, similar to compiling classical algorithms onto CPU/GPU hardware. This thesis aims to tackle some of the important problems in the mapping process. First, can we address hardware errors and algorithm errors at the same time? Second, can we build a better circuit complexity metric to inspire better quantum architecture design, and can we design better quantum algorithms and compilation techniques using this improved metric? Lastly, can we use physics-level simulation results to inspire better circuit-level optimization techniques? We hope to provide positive answers to all three questions above through three separate architectural innovations.

•Dec 1, 2024•0 citations

QSPE: Enumerating Skeletal Quantum Programs for Quantum Library Testing

Jiaming Ye, Fuyuan Zhang, Shangzhou Xia +4 more

The rapid advancement of quantum computing has led to the development of various quantum libraries, empowering compilation, simulation, and hardware backend interfaces. However, ensuring the correctness of these libraries remains a fundamental challenge due to the lack of mature testing methodologies. The state-of-the-art tools often rely on domain-specific configurations and expert knowledge, which limits their accessibility and scalability in practice. Furthermore, although these tools demonstrate strong performance, they adopt measurement-based for output validation in testing, which makes them produce false positive reports. To alleviate these limitations, we propose QSPE, a practical approach that follows the differential testing principle and extends the existing approach, SPE, for quantum libraries. QSPE is fully automated, requiring no pre-set configurations or domain expertise, and can effectively generate a large set of diverse program variants that comprehensively explore the quantum compilation space. To mitigate the possible false positive reports, we propose statevector-based validation as an alternative to measurement-based validation. In our experiments, the QSPE approach demonstrates remarkable effectiveness in generating 22,770 program variants across multiple quantum computing platforms. By avoiding $α$-equivalence at the quantum and classical program wise, QSPE can reduce redundant generation and save more than 90\% of execution cost. Finally, the statevector-based validation method assists QSPE to reduce false alarms and effectively detects 708 miscompilations across multiple quantum libraries. Notably, 81 of the discovered bugs have been officially approved and acknowledged by the Qiskit development team, demonstrating the practical impact of our approach.

quant-pharXiv:2602.00024•Jan 20, 2026•5 citations
Quantum computing has been attracting the attention of both applied research and companies. Continuous progress on fundamental hardware technology promises to bring us more reliable and large-scale quantum computers on which to run the next generation of quantum algorithms. These programs are compiled and executed on dedicated platforms, and similarly to classical programs, a large effort is required to test these platforms and create a robust software toolchain.Unlike previous studies which focused on cross-optimization and cross-backend testing, this dissertation aims to create the first approach for cross-platform testing which compares execution on diverse quantum computing platforms. To inform the design of the method, we will first perform an empirical study of bugs in quantum computing platforms and a review of the characteristics of realistic quantum programs.The final approach for cross-platform testing will include three components: a learning-based method to generate realistic quantum programs, an approach to map and run them on multiple platforms, and finally a quantum-specific statistical test to compare two multivariate binary distributions returned as the output of quantum programs.
The interest in quantum computing is growing, and with it, the importance of software platforms to develop quantum programs. Ensuring the correctness of such platforms is important, and it requires a thorough understanding of the bugs they typically suffer from. To address this need, this paper presents the first in-depth study of bugs in quantum computing platforms. We gather and inspect a set of 223 real-world bugs from 18 open-source quantum computing platforms. Our study shows that a significant fraction of these bugs (39.9%) are quantum-specific, calling for dedicated approaches to prevent and find them. The bugs are spread across various components, but quantum-specific bugs occur particularly often in components that represent, compile, and optimize quantum programming abstractions. Many quantum-specific bugs manifest through unexpected outputs, rather than more obvious signs of misbehavior, such as crashes. Finally, we present a hierarchy of recurrent bug patterns, including ten novel, quantum-specific patterns. Our findings not only show the importance and prevalence bugs in quantum computing platforms, but they help developers to avoid common mistakes and tool builders to tackle the challenge of preventing, finding, and fixing these bugs.
Quantum computing platforms are susceptible to quantum-specific bugs, such as incorrectly ordering qubits or incorrectly implementing quantum abstractions. These bugs are difficult to detect and require specialized expertise. The field faces challenges due to a fragmented landscape of platforms and rapid development cycles that often prioritize new features over thorough testing, severely hindering the reliability of quantum software. To address these challenges, we present IterTestQ, a novel cross-platform testing approach for quantum computing platforms. The key technical contribution is our novel ITE process, which generates equivalent quantum programs by iteratively (I)mporting them into platform-specific representations, (T)ransforming the program via optimizations and gate conversions, and (E)xporting the program again. To transfer programs across platforms and test cross-platform consistency, IterTestQ leverages QASM, an assembly-level representation supported by most platforms. The approach uses a crash oracle to detect failures during cross-platform transformations and an equivalence oracle to validate the semantic consistency of the generated assembly programs, which are expected to be equivalent by construction. We evaluate IterTestQ on widely-used quantum computing platforms, including Qiskit, PennyLane, Pytket, BQSKit, and Cirq, revealing 23 bugs, 17 of which are already confirmed or fixed. Our results also demonstrate that IterTestQ complements existing quantum fuzzers (covering tens of thousands of otherwise uncovered lines), is efficient (with 0.00089 seconds per generated program), and that the ITE process is crucial for its effectiveness.
Quantum computing is getting increasing interest from both academia and industry, and the quantum software landscape has been growing rapidly. The quantum software stack comprises quantum programs, implementing algorithms, and platforms like IBM Qiskit, Google Cirq, and Microsoft Q#, enabling their development. To ensure the reliability and performance of quantum software, various techniques for testing and analyzing it have been proposed, such as test generation, bug pattern detection, and circuit optimization. However, the large amount of work and the fact that work on quantum software is performed by several research communities, make it difficult to get a comprehensive overview of the existing techniques. In this work, we provide an extensive survey of the state of the art in testing and analysis of quantum software. We discuss literature from several research communities, including quantum computing, software engineering, programming languages, and formal methods. Our survey covers a wide range of topics, including expected and unexpected behavior of quantum programs, testing techniques, program analysis approaches, optimizations, and benchmarks for testing and analyzing quantum software. We create novel connections between the discussed topics and present them in an accessible way. Finally, we discuss key challenges and open problems to inspire future research.
Quantum computing is a field that has drawn a lot of attention over the years as it has shown great potential to solve new problems and has seen rapid development. To meet the demands of this rapid development, software engineering researchers have been studying methods to address the unique challenges presented by the probabilistic nature of quantum computing in its software platforms. One important aspect of this is understanding how to test and validate quantum software platforms when the expected outputs when running a program are not always known. Recent research using metamorphic testing was able to detect 13 faults in the Qiskit quantum computing platform; however, this is a rapidly evolving platform. Since their work, Qiskit has already advanced ahead several versions, which brings up more questions regarding research for quantum software platforms: do these tools researchers create hold up across versions of these frequently-updated platforms, and how much work does it take to maintain and extend these tools? In this thesis, we study the ease of regression testing in the context of this quantum computing platform. We update to the latest version of Qiskit, extend two existing metamorphic transformations on a metamorphic testing platform for Qiskit, add six new metamorphic transformations, and perform a set of metamorphic testing runs to analyze how our changes perform. In doing so, we were able to discover five additional new faults on the Qiskit platform; however, we also uncovered several challenges, which we synthesize for the reader. We also offer discussion on where the future work of metamorphic testing on quantum software platforms may lead based on our challenges and successes during this process.
Quantum computing is a revolutionary approach to computation that leverages the principles of quantum mechanics, enabling computers to process information in fundamentally different ways than classical computers. Unlike traditional bits, which represent data as either 0s or 1s, quantum bits, or qubits, can exist in multiple states simultaneously due to superposition. Additionally, qubits can be entangled, allowing them to be interconnected in ways that enhance computational power and efficiency. This unique behavior has the potential to run certain types of computation tasks much faster than classical computers can, marking a significant leap forward in technology. When building actual quantum computers to run practical quantum algorithms, one needs efficient ways to map the algorithms to the hardware components, similar to compiling classical algorithms onto CPU/GPU hardware. This thesis aims to tackle some of the important problems in the mapping process. First, can we address hardware errors and algorithm errors at the same time? Second, can we build a better circuit complexity metric to inspire better quantum architecture design, and can we design better quantum algorithms and compilation techniques using this improved metric? Lastly, can we use physics-level simulation results to inspire better circuit-level optimization techniques? We hope to provide positive answers to all three questions above through three separate architectural innovations.
The rapid advancement of quantum computing has led to the development of various quantum libraries, empowering compilation, simulation, and hardware backend interfaces. However, ensuring the correctness of these libraries remains a fundamental challenge due to the lack of mature testing methodologies. The state-of-the-art tools often rely on domain-specific configurations and expert knowledge, which limits their accessibility and scalability in practice. Furthermore, although these tools demonstrate strong performance, they adopt measurement-based for output validation in testing, which makes them produce false positive reports. To alleviate these limitations, we propose QSPE, a practical approach that follows the differential testing principle and extends the existing approach, SPE, for quantum libraries. QSPE is fully automated, requiring no pre-set configurations or domain expertise, and can effectively generate a large set of diverse program variants that comprehensively explore the quantum compilation space. To mitigate the possible false positive reports, we propose statevector-based validation as an alternative to measurement-based validation. In our experiments, the QSPE approach demonstrates remarkable effectiveness in generating 22,770 program variants across multiple quantum computing platforms. By avoiding ααα-equivalence at the quantum and classical program wise, QSPE can reduce redundant generation and save more than 90\% of execution cost. Finally, the statevector-based validation method assists QSPE to reduce false alarms and effectively detects 708 miscompilations across multiple quantum libraries. Notably, 81 of the discovered bugs have been officially approved and acknowledged by the Qiskit development team, demonstrating the practical impact of our approach.