Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2511.18506

HQPEF-Py: Metrics, Python Patterns, and Guidance for Evaluating Hybrid Quantum Programs

Osei, Michael Adjei, Shapiro, Sidney

Published Nov 23, 2025• 0 citations
arXivHTMLPDF

Abstract

We study how to evaluate hybrid quantum programs as end-to-end workflows rather than as isolated devices or algorithms. Building on the Hybrid Quantum Program Evaluation Framework (HQPEF), we formalize a workflow-aware Quantum Readiness Level (QRL) score; define a normalized speedup under quality constraints for the Utility of Quantumness (UQ); and provide a timing-and-drift audit for hybrid pipelines. We complement these definitions with concise Python reference implementations that illustrate how to instantiate the metrics and audit procedures with state-of-the-art classical and quantum solvers (e.g., via Qiskit or PennyLane), while preserving matched-budget discipline and reproducibility.

DOI
10.48550/arxiv.2511.18506

Related papers

Quantum-ready software engineering: principles, frameworks, and hybrid development pipelines

Hasanain Hazim Azeez

Quantum computing is quickly becoming a potentially disruptive technology that can tackle the problems inaccessible to classical computers, but the use of quantum computing in real software systems has yet to be fully explored. Present day quantum studies are largely focused on the development of algorithms and hardware, whereas engineering workflows to be used to enable smooth hybrid quantum-classical operations attract limited attention. The paper suggests a concept of quantum-ready software engineering- a systematic approach to applying established principles of software engineering to hybrid systems. In order to meet maintainability, scalability, and reliability objectives in hybrid applications, propose the adoption of a multilayered design, such as abstraction, integration, orchestration, and verification. An example concrete programming template, written in Python in Qiskit, demonstrates how quantum functions being used as modular services can be simulated with classical software. Evaluation measures (maintainability index, latency, and modularity) demonstrate the feasibility of the framework. Results indicate that the hybrid pipelines can be designed in a systematic way using quantum-classical-adapted design patterns. The study introduces a repeatable methodology, program templates, and design pattern engineering principles, which open the door to additional development of quantum DevOps and design pattern formalization. This project, by bridging the gap between classical software engineering and quantum integration, gives a platform upon which to build extraordinarily reliable, scalable systems in the Noisy Intermediate-Scale Quantum and beyond.

International Journal of Information technology and Computer Engineering•Sep 13, 2025•1 citation

Fair Benchmarking of Optimisation Applications

Frank Phillipson

Quantum optimisation is emerging as a promising approach alongside classical heuristics and specialised hardware, yet its performance is often difficult to assess fairly. Traditional benchmarking methods, rooted in digital complexity theory, do not directly capture the continuous dynamics, probabilistic outcomes, and workflow overheads of quantum and hybrid systems. This paper proposes principles and protocols for fair benchmarking of quantum optimisation, emphasising end-to-end workflows, transparency in tuning and reporting, problem diversity, and avoidance of speculative claims. By extending lessons from classical benchmarking and incorporating application-driven and energy-aware metrics, we outline a framework that enables practitioners to evaluate quantum methods responsibly, ensuring reproducibility, comparability, and trust in reported results.

quant-pharXiv:2512.07915•Dec 9, 2025•1 citation

Optimization of Hybrid Quantum-Classical Algorithms

Lian Remme, Alexander Weinert, Andre Waschk

Quantum computers do not run in isolation; rather, they are embedded in quantum-classical hybrid architectures. In these setups, a quantum processing unit communicates with a classical device in near-real time. To enable efficient hybrid computations, it is mandatory to optimize quantum-classical hybrid code. To the best of our knowledge, no previous work on the optimization of hybrid code nor on metrics for which to optimize such code exists. In this work, we take a step towards optimization of hybrid programs by introducing seven optimization routines and three metrics to evaluate the effectiveness of the optimization. We implement these routines for the hybrid quantum language Quil and show that our optimizations improve programs according to our metrics. This lays the foundation for new kinds of hybrid optimizers that enable real-time collaboration between quantum and classical devices.

arXiv:2505.12853•IEEE International Conference on Quantum Software (QSW)•Jul 7, 2025•6 citations

A Dependability Profile for Hybrid Quantum–Classical Systems

Piyush A. Sonawane, Andrew Kelly, Naveed Mahmud

Hybrid quantum–classical computing is a key paradigm for executing quantum algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices. However, the dependability of quantum–classical workflows under realistic system conditions remains underexplored. These workflows often involve heterogeneous resources, including CPUs, GPUs, FPGAs, and cloud-accessible QPUs, introducing variability, resource constraints, and trust concerns. In this work, we propose a methodology for benchmarking the dependability of quantum–classical workflows and their execution on hybrid systems. Our methodology involves establishing a dependability profile for workloads and corresponding systems using empirical measurements and evaluations of standard metrics of speed, scalability, and security of the overall system. Experimental evaluation includes two variational algorithms as case studies deployed on a hybrid CPU–FPGA system with QPU emulation backends that replicate the noise characteristics and gate sets of real quantum processors. For the dependability profile, we define iteration-level metrics for speed: variance, jitter, worst-case latency, stability, and resource-aware scalability metrics: Circuit Layer Operations Per Second (CLOPS). Results demonstrate that the proposed dependability profile enables practical decision-making by linking system behavior to deployability and performance limitations.

International Conference on Dependable Systems and Networks workshops•Jun 22, 2026•0 citations

qWard: A Unified Toolkit for Pre- and Post-Runtime Quantum Circuit Metrics

Carmen Márquez, Daniel Sierra-Sosa, Kelly Garcés

As quantum computing (QC) matures towards practical applications, effective methods to evaluate quantum algorithms, circuits, and execution quality in quantum software are gaining more relevance. To apply such methods, a set of meaningful metrics must be defined, understood, calculated, and applied. In an effort to solve that problem, this paper categorizes quantum related metrics into pre-runtime (derived from the static analysis of circuits before execution) and post-runtime (derived from the analysis of execution results). It is worthwhile to mention that while popular quantum SDKs such as Qiskit, PennyLane, and Q\# offer some circuit characteristics, there is often a gap between these native capabilities and the broader set of insightful pre- and post-runtime metrics in the literature. This paper introduces$\boldsymbol{q}$Ward, a Python library developed to bridge this gap by offering an initial implementation for the Qiskit SDK. qWard provides a toolkit designed to compute and visualize key pre-runtime and post-runtime metrics, facilitating the analysis of quantum algorithms. To illustrate its application, an experiment using a bell state circuit is conducted, which includes different backend configurations within the simulator as well as its execution on the IBM “torino” quantum processor. It is important to note that the initial version of qWard is limited to the Qiskit SDK and the Qiskit AER simulator.

IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON)•Oct 28, 2025•0 citations

Extending the Q-Score to an Application-Level Quantum Metric Framework

Ward van der Schoot, Robert S. Wezeman, Niels M. P. Neumann +2 more

Evaluating the performance of quantum devices is an important step towards scaling quantum devices and eventually using them in practice. The great number of available quantum metrics and the different hardware technologies used to develop quantum computers complicate this evaluation. In addition, different computational paradigms implement quantum operations in different ways. A prominent quantum metric is given by the Q-score metric of Atos. This metric was originally introduced as a standalone way to benchmark devices using the Max-Cut problem. In this work, we show that the Q-score defines a framework of quantum metrics, which allows benchmarking using different problems, user settings and solvers. To showcase the applicability of the framework, we showcase a second Q-score in this framework, called the Q-score Max-Clique. This yields, to our knowledge, the first application-level metric capable of natively comparing three different paradigms of quantum computing. This metric is evaluated on these computational quantum paradigms – quantum annealing, gate-based quantum computing, and photonic quantum computing – and the results are compared to those obtained by classical solvers.

arXiv:2302.00639•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 15, 2024•20 citations
We study how to evaluate hybrid quantum programs as end-to-end workflows rather than as isolated devices or algorithms. Building on the Hybrid Quantum Program Evaluation Framework (HQPEF), we formalize a workflow-aware Quantum Readiness Level (QRL) score; define a normalized speedup under quality constraints for the Utility of Quantumness (UQ); and provide a timing-and-drift audit for hybrid pipelines. We complement these definitions with concise Python reference implementations that illustrate how to instantiate the metrics and audit procedures with state-of-the-art classical and quantum solvers (e.g., via Qiskit or PennyLane), while preserving matched-budget discipline and reproducibility.
Quantum computing is quickly becoming a potentially disruptive technology that can tackle the problems inaccessible to classical computers, but the use of quantum computing in real software systems has yet to be fully explored. Present day quantum studies are largely focused on the development of algorithms and hardware, whereas engineering workflows to be used to enable smooth hybrid quantum-classical operations attract limited attention. The paper suggests a concept of quantum-ready software engineering- a systematic approach to applying established principles of software engineering to hybrid systems. In order to meet maintainability, scalability, and reliability objectives in hybrid applications, propose the adoption of a multilayered design, such as abstraction, integration, orchestration, and verification. An example concrete programming template, written in Python in Qiskit, demonstrates how quantum functions being used as modular services can be simulated with classical software. Evaluation measures (maintainability index, latency, and modularity) demonstrate the feasibility of the framework. Results indicate that the hybrid pipelines can be designed in a systematic way using quantum-classical-adapted design patterns. The study introduces a repeatable methodology, program templates, and design pattern engineering principles, which open the door to additional development of quantum DevOps and design pattern formalization. This project, by bridging the gap between classical software engineering and quantum integration, gives a platform upon which to build extraordinarily reliable, scalable systems in the Noisy Intermediate-Scale Quantum and beyond.
Quantum optimisation is emerging as a promising approach alongside classical heuristics and specialised hardware, yet its performance is often difficult to assess fairly. Traditional benchmarking methods, rooted in digital complexity theory, do not directly capture the continuous dynamics, probabilistic outcomes, and workflow overheads of quantum and hybrid systems. This paper proposes principles and protocols for fair benchmarking of quantum optimisation, emphasising end-to-end workflows, transparency in tuning and reporting, problem diversity, and avoidance of speculative claims. By extending lessons from classical benchmarking and incorporating application-driven and energy-aware metrics, we outline a framework that enables practitioners to evaluate quantum methods responsibly, ensuring reproducibility, comparability, and trust in reported results.
Quantum computers do not run in isolation; rather, they are embedded in quantum-classical hybrid architectures. In these setups, a quantum processing unit communicates with a classical device in near-real time. To enable efficient hybrid computations, it is mandatory to optimize quantum-classical hybrid code. To the best of our knowledge, no previous work on the optimization of hybrid code nor on metrics for which to optimize such code exists. In this work, we take a step towards optimization of hybrid programs by introducing seven optimization routines and three metrics to evaluate the effectiveness of the optimization. We implement these routines for the hybrid quantum language Quil and show that our optimizations improve programs according to our metrics. This lays the foundation for new kinds of hybrid optimizers that enable real-time collaboration between quantum and classical devices.
Hybrid quantum–classical computing is a key paradigm for executing quantum algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices. However, the dependability of quantum–classical workflows under realistic system conditions remains underexplored. These workflows often involve heterogeneous resources, including CPUs, GPUs, FPGAs, and cloud-accessible QPUs, introducing variability, resource constraints, and trust concerns. In this work, we propose a methodology for benchmarking the dependability of quantum–classical workflows and their execution on hybrid systems. Our methodology involves establishing a dependability profile for workloads and corresponding systems using empirical measurements and evaluations of standard metrics of speed, scalability, and security of the overall system. Experimental evaluation includes two variational algorithms as case studies deployed on a hybrid CPU–FPGA system with QPU emulation backends that replicate the noise characteristics and gate sets of real quantum processors. For the dependability profile, we define iteration-level metrics for speed: variance, jitter, worst-case latency, stability, and resource-aware scalability metrics: Circuit Layer Operations Per Second (CLOPS). Results demonstrate that the proposed dependability profile enables practical decision-making by linking system behavior to deployability and performance limitations.
As quantum computing (QC) matures towards practical applications, effective methods to evaluate quantum algorithms, circuits, and execution quality in quantum software are gaining more relevance. To apply such methods, a set of meaningful metrics must be defined, understood, calculated, and applied. In an effort to solve that problem, this paper categorizes quantum related metrics into pre-runtime (derived from the static analysis of circuits before execution) and post-runtime (derived from the analysis of execution results). It is worthwhile to mention that while popular quantum SDKs such as Qiskit, PennyLane, and Q\# offer some circuit characteristics, there is often a gap between these native capabilities and the broader set of insightful pre- and post-runtime metrics in the literature. This paper introducesq\boldsymbol{q}qWard, a Python library developed to bridge this gap by offering an initial implementation for the Qiskit SDK. qWard provides a toolkit designed to compute and visualize key pre-runtime and post-runtime metrics, facilitating the analysis of quantum algorithms. To illustrate its application, an experiment using a bell state circuit is conducted, which includes different backend configurations within the simulator as well as its execution on the IBM “torino” quantum processor. It is important to note that the initial version of qWard is limited to the Qiskit SDK and the Qiskit AER simulator.
Evaluating the performance of quantum devices is an important step towards scaling quantum devices and eventually using them in practice. The great number of available quantum metrics and the different hardware technologies used to develop quantum computers complicate this evaluation. In addition, different computational paradigms implement quantum operations in different ways. A prominent quantum metric is given by the Q-score metric of Atos. This metric was originally introduced as a standalone way to benchmark devices using the Max-Cut problem. In this work, we show that the Q-score defines a framework of quantum metrics, which allows benchmarking using different problems, user settings and solvers. To showcase the applicability of the framework, we showcase a second Q-score in this framework, called the Q-score Max-Clique. This yields, to our knowledge, the first application-level metric capable of natively comparing three different paradigms of quantum computing. This metric is evaluated on these computational quantum paradigms – quantum annealing, gate-based quantum computing, and photonic quantum computing – and the results are compared to those obtained by classical solvers.