The benchmark of Noisy Intermediate-Scale Quantum (NISQ) computers plays a major role in the research and development of quantum devices and algorithms. Our study aims to investigate a single metric of benchmarking for the Noisy Intermediate-Scale Quantum device. We use state-of-the-art applications to generate a device-score according to our proposed formula. Our study utilizes Quantum Volume (QV) to determine the qubit circuit size for each device. Furthermore, we utilize SupermarQ to determine each application's benchmark score, utilizing their 5 feature vectors to indicate the circuit performance, and applying their circuit optimization techniques to configure the transpiler. Our experiment shows that device-score of the simulated IBM backend are directly correlated to Quantum Volume. However, it should be noted that topology and gate error are also embedded in the benchmark scores of the applications. Because of this, our study introduces the device-score, a single number metric that provides a more precise scale with the assistance of Quantum Volume and SupermarQ.
The benchmark of Noisy Intermediate-Scale Quantum (NISQ) computers plays a major role in the research and development of quantum devices and algorithms. Our study aims to investigate a single metric of benchmarking for the Noisy Intermediate-Scale Quantum device. We use state-of-the-art applications to generate a device-score according to our proposed formula. Our study utilizes Quantum Volume (QV) to determine the qubit circuit size for each device. Furthermore, we utilize SupermarQ to determine each application's benchmark score, utilizing their 5 feature vectors to indicate the circuit performance, and applying their circuit optimization techniques to configure the transpiler. Our experiment shows that device-score of the simulated IBM backend are directly correlated to Quantum Volume. However, it should be noted that topology and gate error are also embedded in the benchmark scores of the applications. Because of this, our study introduces the device-score, a single number metric that provides a more precise scale with the assistance of Quantum Volume and SupermarQ.
Quantum volume (QV) has become the de-facto standard benchmark to quantify the capability of Noisy Intermediate-Scale Quantum (NISQ) devices. While QV values are often reported by NISQ providers for their systems, we perform our own series of QV calculations on 24 NISQ devices currently offered by IBM Q, IonQ, Rigetti, Oxford Quantum Circuits, and Quantinuum (formerly Honeywell). Our approach characterizes the performances that an advanced user of these NISQ devices can expect to achieve with a reasonable amount of optimization, but without white-box access to the device. In particular, we compile QV circuits to standard gate sets of the vendor using compiler optimization routines where available, and we perform experiments across different qubit subsets. We find that running QV tests requires very significant compilation cycles, QV values achieved in our tests typically lag behind officially reported results and also depend significantly on the classical compilation effort invested.
Quantum computing promises to solve complex problems beyond the capabilities of classical computers. However, the practical implementation of quantum algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices is significantly challenged by various noise sources that degrade computational fidelity. Transpilation—the process of converting high-level quantum circuits into optimized, hardware-specific instructions—plays a crucial role in mitigating these noise effects. In this paper, we present a comprehensive study on enhancing noise resilience in quantum algorithms through advanced transpilation techniques, evaluated using the SupermarQ benchmarking suite. By systematically analyzing the impact of different optimization levels in Qiskit’s transpiler on a diverse set of quantum algorithms, we demonstrate how specific transpilation strategies influence key performance metrics related to noise resilience. Our results show that higher optimization levels can improve metrics such as Liveness (LV) and Parallelism (PL), leading to more reliable quantum computations on NISQ hardware. This work contributes to the development of noise-adaptive transpilation methods that incorporate noise characteristics into the optimization process, ultimately enhancing the practical viability of quantum algorithms.
The emergence of quantum computers as a new computational paradigm has been accompanied by speculation concerning the scope and timeline of their anticipated revolutionary changes. While quantum computing is still in its infancy, the variety of different architectures used to implement quantum computations make it difficult to reliably measure and compare performance. This problem motivates our introduction of SupermarQ, a scalable, hardware-agnostic quantum benchmark suite which uses application-level metrics to measure performance. SupermarQ is the first attempt to systematically apply techniques from classical benchmarking methodology to the quantum domain. We define a set of feature vectors to quantify coverage, select applications from a variety of domains to ensure the suite is representative of real workloads, and collect benchmark results from the IBM, IonQ, and AQT@LBNL platforms. Looking forward, we envision that quantum benchmarking will encompass a large cross-community effort built on open source, constantly evolving benchmark suites. We introduce SupermarQ as an important step in this direction.
Current advancements in technology have focused the attention of the quantum computing community toward exploring the potential of near-term devices whose computing power surpasses that of classical computers in practical applications. An unresolved central question revolves around whether the inherent noise in these devices can be overcome or whether any potential quantum advantage would be limited. There is no doubt that crosstalk is one of the main sources of noise in noisy intermediate-scale quantum (NISQ) systems, and it poses a fundamental challenge to hardware designs. Crosstalk between parallel instructions can corrupt quantum states and cause incorrect program execution. In this study, we present a necessary analysis of the crosstalk error effect on NISQ devices. Our approach is extremely straightforward and practical to estimate the crosstalk error of various multi-qubit devices. In particular, we combine the randomized benchmarking (RB) and simultaneous randomized benchmarking (SRB) protocol to estimate the crosstalk error from the correlation controlled-NOT (CNOT) gate. We demonstrate this protocol experimentally on 5-, 7-, \& 16-qubit devices. Our results demonstrate the crosstalk error model of three different IBM quantum devices over the experimental week and compare the error variation against the machine, number of qubits, quantum volume, processor, and topology. We then confirm the improvement in the circuit fidelity on different benchmarks by up to 3.06x via inserting an instruction barrier, as compared with an IBM quantum noisy device which offers near-optimal crosstalk mitigation in practice. Finally, we discuss the current system limitation, its tradeoff on fidelity and depth, noise beyond the NISQ system, and mitigation opportunities to ensure that the quantum operation can perform its quantum magic undisturbed.
As quantum computers continue to increase in size and topological complexity, benchmarking crosstalk becomes more complex and resource-intensive. This limits the ability to obtain relevant crosstalk metrics for applications such as error mitigation, quantum computer security, and circuit transpilation. There applications benefit from accessible metrics on how each gate contributes to crosstalk. However, crosstalk metrics are rarely provided by quantum computer providers and can be expensive to obtain on modern large NISQ devices. In this work, we propose CrossBench, a customizable system that generates crosstalk benchmarks for finitely large NISQ devices with arbitrary topologies. CrossBench uses a custom graph labeling algorithm to generate crosstalk benchmarks for a given quantum topology and gate set. These benchmarks can be used to estimate the average contribution of each gate's crosstalk to qubit error rates. We evaluate the effectiveness of CrossBench by generating and running benchmarks on multiple IBM quantum computers with different topologies. Our results show that CrossBench can identify gates that introduce significant crosstalk across all tested devices, with strong statistical significance (p < 0.05). These promising results show that CrossBench can give simple and accessible crosstalk benchmarks for modern NISQ systems.
The current phase of quantum computing is in the Noisy Intermediate-Scale Quantum (NISQ) era. On NISQ devices, two-qubit gates such as CNOTs are much noisier than single-qubit gates, so it is essential to minimize their count. Quantum circuit synthesis is a process of decomposing an arbitrary unitary into a sequence of quantum gates, and can be used as an optimization tool to produce shorter circuits to improve overall circuit fidelity. However, the time-to-solution of synthesis grows exponentially with the number of qubits. As a result, synthesis is intractable for circuits on a large qubit scale.
In this paper, we propose a hierarchical, block-by-block optimization framework, QGo, for quantum circuit optimization. Our approach allows an exponential cost optimization to scale to large circuits. QGo uses a combination of partitioning and synthesis: 1) partition the circuit into a sequence of independent circuit blocks; 2) re-generate and optimize each block using quantum synthesis; and 3) re-compose the final circuit by stitching all the blocks together. We perform our analysis and show the fidelity improvements in three different regimes: small-size circuits on real devices, medium-size circuits on noise simulations, and large-size circuits on analytical models. Using a set of NISQ benchmarks, we show that QGo can reduce the number of CNOT gates by 29.9% on average and up to 50% when compared with industrial compilers such as t|ket>. When executed on the IBM Athens system, shorter depth leads to higher circuit fidelity. We also demonstrate the scalability of our QGo technique to optimize circuits of 60+ qubits. Our technique is the first demonstration of successfully employing and scaling synthesis in the compilation toolchain for large circuits. Overall, our approach is robust for direct incorporation in production compiler toolchains.