We examine which conclusions from quantum-hardware benchmarks persist when the same programs are run again. On eight fixed routes of IQM Emerald, round-trip state echo (RTSE), a diagnostic of state return after a round trip, gave mean scores near $0.84$ that differed by only $0.00125$ between two executions. Individual route estimates nevertheless changed by up to $0.06$ in both directions. The mean difference between RTSE and measurement-free teleportation along the same routes shrank from $0.046$ to $0.014$ and was no longer statistically resolved on repetition; none of the four prespecified communication targets met its replication rule. Recovery with an existing four-qubit single-deletion code also changed on repetition: after discarding one encoded qubit in the circuit model, the IQM mean fell from $0.738$ to $0.624$ and failed the replication test against the two-thirds operational reference. In a separate IonQ experiment, recovery means of $0.91$ and $0.92$ passed tests against that reference and against a deliberately incorrect decoder in both executions. These results concern the tested workloads and executions. They show why a nearly unchanged aggregate benchmark score needs to be interpreted alongside individual routes, direct task comparisons, and repeated executions.
Comments
42 pages, 11 figures, 10 tables. Revised abstract, exposition and interpretation of repeated tests; expanded provenance and reproducibility checks. Original hardware data unchanged. Reproducibility package: doi:10.5281/zenodo.21969397
We examine which conclusions from quantum-hardware benchmarks persist when the same programs are run again. On eight fixed routes of IQM Emerald, round-trip state echo (RTSE), a diagnostic of state return after a round trip, gave mean scores near 0.84 that differed by only 0.00125 between two executions. Individual route estimates nevertheless changed by up to 0.06 in both directions. The mean difference between RTSE and measurement-free teleportation along the same routes shrank from 0.046 to 0.014 and was no longer statistically resolved on repetition; none of the four prespecified communication targets met its replication rule. Recovery with an existing four-qubit single-deletion code also changed on repetition: after discarding one encoded qubit in the circuit model, the IQM mean fell from 0.738 to 0.624 and failed the replication test against the two-thirds operational reference. In a separate IonQ experiment, recovery means of 0.91 and 0.92 passed tests against that reference and against a deliberately incorrect decoder in both executions. These results concern the tested workloads and executions. They show why a nearly unchanged aggregate benchmark score needs to be interpreted alongside individual routes, direct task comparisons, and repeated executions.
H. Billah and M.M.H. Sarker contributed equally to this work. Interpreting a fixed quantum circuit's output as evidence of a hardware effect requires ruling out two things usually conflated with it: finite-sample fluctuation, and structure already present in the circuit's own noiseless output. Less commonly checked is a third: whether the reference circuit compared against is the one actually executed, once compiler-level qubit routing is accounted for. We execute a fixed 35-qubit, 243-gate diagnostic circuit on IBM Heron-family hardware (three runs, two backends) and show this third check is essential: the routed circuit submitted to hardware has different noiseless predictions than the logical circuit intended, for several qubits. Against the correct, routed-circuit reference, the aggregate deviation is significant in two of three runs, and four registers (0, 9, 14, 27) show a significant output-probability deviation in every run — though we distinguish a high-confidence subset, {0, 9, 27}, with tightly time-matched calibration, from qubit 14, included only on an exploratory basis given its calibration was retrieved roughly two weeks post-execution. Same-backend replicates correlate more strongly by the Pearson measure, though a rank-based (Spearman) test shows this gap does not survive at the level of register ordering, indicating substantial leverage from a small number of registers rather than a broadly robust pattern; these three executions also share a byte-identical payload and are not independent replications in the strictest sense. The mechanism is only partially resolved: independent single-qubit readout error and raw gate count are ruled out as complete explanations for most of these registers, but the simplest version of the leading hypothesis for the single most-affected register does not reproduce the observed cross-backend pattern even with live, backend-specific timing data. A second, structurally different circuit validated only in simulation calibrates the statistical procedure's behavior under one synthetic noise model; it does not itself establish generality to real hardware, and the specific register set found here should not be expected to generalize beyond this study. Note on scope: this is a methodology and case-study contribution centered on executed-circuit references for fixed-circuit diagnostics, not a characterization of IBM Heron hardware in general or a randomized-benchmarking experiment in the formal sense. Supporting data, the routed OpenQASM circuits, raw measurement counts, and analysis code for this study are available on request from the corresponding author. Affiliation: Sonicium Quantum Lab Ltd., Dhaka, Bangladesh.
Quantum processors have crossed the one-hundred-qubit mark, but noise continues to limit circuit performance, while full quantum error correction remains too costly for routine use. Error suppression and mitigation therefore play an important role in extracting value from current hardware, yet independent comparisons of commercial solutions on identical workloads and devices remain scarce. We benchmark IBM Qiskit Runtime, Q-CTRL Performance Management, and Qedma QESEM on IBM Pittsburgh, a 156-qubit IBM Quantum Heron r3 processor. For Sampler workloads, we run Bernstein-Vazirani, quantum phase estimation, GHZ-state preparation, and randomized mirror circuits with up to 100 measured qubits, comparing raw execution, IBM measurement twirling, and Q-CTRL. For Estimator workloads, we measure chain-averaged magnetization and correlation observables of an eight-layer transverse-field Ising circuit at 25, 50, and 75 qubits against an exact matrix-product-state reference, comparing IBM raw execution, IBM TREX plus twirling, Q-CTRL, and QESEM. Q-CTRL produced the best results on the three structured Sampler workloads while keeping reported QPU times within the same order as the IBM configurations. Across six Ising observable and system-size cases, aggregate mean absolute error was 0.0883 for IBM raw execution, 0.0807 for IBM TREX plus twirling, 0.0285 for Q-CTRL, and 0.0188 for QESEM. Relative to raw execution, Q-CTRL and QESEM reduced aggregate error by factors of 3.10 and 4.70, respectively, while QESEM used 7.5 to 11.1 times the reported QPU time of Q-CTRL. These results show that managed error suppression and mitigation can substantially improve current hardware performance, but with distinct accuracy and execution-time tradeoffs.
Validating quantum processors at scale increasingly relies on approximate classical simulation, which adds truncation error to the comparison between ideal and observed behavior. We describe a 15-qubit dynamic-circuit benchmark that stays within exact state-vector simulation, so the ideal reference value is computed without approximation. The circuit scrambles a 6-qubit system, couples it to a 5-qubit engineered environment, reads out four weakly coupled probe qubits mid-circuit, and then applies the inverse of the scrambling and system–environment coupling to restore the system's coherence. Exact summation over the 16 probe outcomes gives an ideal recovery fidelity F_max = 0.9889 on the 11-qubit system–environment register; the shortfall from unity reflects information carried away by the probes, whose coupling is not inverted and which are discarded after readout. We demonstrate the pipeline with a single 400-shot run on the Quantinuum H2-1 emulator (quantinuum.sim.h2-1e) via Azure Quantum. A local surrogate built from three published H2-1 headline error rates predicted F_exp = 0.8353 ± 0.0054; the emulator returned a readout-mitigated F_rec = 0.7836 ± 0.0212, 2.36 nominal standard errors lower. Because the surrogate's quoted uncertainty reflects sampling error only, we interpret the shortfall as evidence of surrogate-model mismatch—expected, given the emulator's richer noise model—rather than as a statistically calibrated anomaly. No physical-hardware data are reported. Changes in v50: Retitled (formerly "A 15-Qubit Decoherence-Reversal Protocol for Benchmarking Mid-Circuit Measurements and Environmental Imprinting"). Corrected the echo formulation (v49 Eq. 4 was identically 1) and the reported raw frequency (77.5%, not 76.0%). Result reframed as a nominal surrogate-model mismatch on the H2-1 emulator rather than anomalous decoherence; SWAP-routing explanation withdrawn. References corrected; limitations and data/code availability added. Analysis code unchanged from v48. Code and data: https://github.com/dkahng/15-qubit-mcm-protocol
A compiler pass over a quantum error correction circuit must preserve what the circuit computes. The artefacts a practitioner reaches for when reasoning about such a circuit, detector samples and the detector error model, are defined relative to the circuit's own noiseless execution. This self-reference makes them exactly blind, rather than merely insensitive, in the regime where a pass has to be checked: before noise is added, which is also where the checking must happen if noise channels are not to survive the gates they describe. Across 335 mutants that provably change what a repetition-code memory circuit computes, detector sampling and detector error model comparison detect none. On a rotated surface code memory experiment, deleting every single-qubit gate leaves the detector error model byte-identical, 29 lines and 24 detectors either way, while 21 of 33 measurement outcomes change by up to 0.50. Two negative controls, each equivalent on every one of 200 trials with no oracle raising a false positive, establish that the measurement is not an artefact of the mutation set. The paper gives the measurement-record oracle that closes the gap and is exact for noiseless stabiliser circuits, and reports two negative results: one compiler test suite whose oracle detected every fault that could be constructed against it, and one transformation library that came back clean. All code and data are public.
Hardware noise and finite sampling perturb the fidelity estimates forming a quantum-kernel Gram matrix. We measured how far three hardware-reconstructed Gram matrices depart from an exact statevector reference for one frozen four-qubit ZZ feature map on N=24 indoor air-quality windows, executed on ibm_fez at 1024 shots per circuit in three single, non-interleaved jobs: baseline, dynamical decoupling alone, and gate twirling alone. All were complete, finite, and positive-semidefinite. Off-diagonal root-mean-squared error (RMSE) against the reference was 0.0878, 0.0864, and 0.0427; full-matrix centered kernel alignment (CKA) ranged 0.933-0.989 and the post hoc diagonal-excluded (U-centered) CKA 0.816-0.986. The gate-twirled job deviated least on every reported geometry axis; its baseline contrasts are deletion-stable for the Spearman, mean-absolute-error, RMSE, and full-matrix CKA diagnostics, while the Pearson and diagonal-excluded contrasts fall just below that convention. Dynamical decoupling was not separated from the baseline. The observed error exceeded both finite-shot reference scales, so, under those sampling-only models, sampling does not explain it. Centered kernel-target alignment did not track reconstruction fidelity and stayed at or below each label-permutation reference: implementation fidelity and task relevance are distinct diagnostic axes. All configuration-level statements describe three realized jobs on one backend; no mitigation-efficacy, classifier-superiority, forecasting, or quantum-advantage claim is made.
We derive an explicit test of whether a reduced quantum-memory model overstates recoverability. For channels with orthogonal logical outputs in distinguishable sectors, the minimum recovery error follows directly from differences between environmental record amplitudes, without numerical recovery optimization. Applying this test to two-excitation oscillator–qubit and four-qubit encodings reveals that matching single-loss responses need not preserve finite-time recoverability. Under collective loss, the four-qubit encoding is perfectly recoverable, whereas oscillator emission records reveal logical information. The resulting unavoidable error initially grows as the cube of storage time. The test quantifies omitted-level effects and benchmarks reduced models and recovery algorithms.