Benchmarking of quantum machine learning (QML) algorithms is challenging due to the complexity and variability of QML systems, e. g., regarding model ansatzes, data sets, training techniques, and hyper-parameter selection. The QUantum computing Application benchmaRK (QUARK) framework simplifies and standardizes benchmarking studies for quantum computing applications. Here, we propose several extensions of QUARK to include the ability to evaluate the training and deployment of quantum generative models. We describe the updated software architecture and illustrate its flexibility through several example applications: (1) We trained different quantum generative models using several circuit ansatzes, data sets, and data transformations. (2) We evaluated our models on GPU and real quantum hardware. (3) We assessed the generalization capabilities of our generative models using a broad set of metrics that capture, e. g., the novelty and validity of the generated data.
Benchmarking of quantum machine learning (QML) algorithms is challenging due to the complexity and variability of QML systems, e. g., regarding model ansatzes, data sets, training techniques, and hyper-parameter selection. The QUantum computing Application benchmaRK (QUARK) framework simplifies and standardizes benchmarking studies for quantum computing applications. Here, we propose several extensions of QUARK to include the ability to evaluate the training and deployment of quantum generative models. We describe the updated software architecture and illustrate its flexibility through several example applications: (1) We trained different quantum generative models using several circuit ansatzes, data sets, and data transformations. (2) We evaluated our models on GPU and real quantum hardware. (3) We assessed the generalization capabilities of our generative models using a broad set of metrics that capture, e. g., the novelty and validity of the generated data.
Abstract Quantum computing promises a disruptive impact on machine learning algorithms, taking advantage of the exponentially large Hilbert space available. However, it is not clear how to scale quantum machine learning (QML) to industrial-level applications. This paper investigates the scalability and noise resilience of quantum generative learning applications. We consider the training performance in the presence of statistical noise due to finite-shot noise statistics and quantum noise due to decoherence to analyze the scalability of QML methods. We employ rigorous benchmarking techniques to track progress and identify challenges in scaling QML algorithms, and show how characterization of QML systems can be accelerated, simplified, and made reproducible when the QUARK framework is used. We show that QGANs are not as affected by the curse of dimensionality as QCBMs and to which extent QCBMs are resilient to noise.
Quantum generative modeling is a growing area of interest for industry-relevant applications. This work systematically compares a broad range of techniques to guide quantum computing practitioners when deciding which models and methods to use in their applications. We compare fundamentally different architectural ansatzes of parametric quantum circuits: (1) A continuous architecture, which produces continuous-valued data samples, and (2) a discrete architecture, which samples on a discrete grid. We also compare the performance of different data transformations: the min-max and the probability integral transforms. We use two popular training methods: (1) quantum circuit Born machines (QCBM), and (2) quantum generative adversarial networks (QGAN). We study their performance and tradeoffs as the number of model parameters increases, with a baseline comparison of similarly trained classical neural networks. The study is performed on six low-dimensional synthetic and two real financial data sets. Our two key findings are that: (1) For all data sets, our quantum models require similar or fewer parameters than their classical counterparts. In the extreme case, the quantum models require two orders of magnitude less parameters. (2) We empirically find that a variant of the discrete architecture, which learns the copula of the probability distribution, outperforms all other methods.
Quantum Machine Learning (QML) has attracted widespread notice as an innovative approach that combines the computational capabilities of quantum computing with modern data-driven training methodologies. Despite its potential, practical implementation remains challenging due to restricted access to quantum hardware and the limitations inherent in Noisy Intermediate-Scale Quantum (NISQ) devices. Cloud-enabled quantum computing platforms mitigate these challenges by offering scalable and remote access to quantum processors and simulators, thereby supporting hybrid quantum–classical computational frameworks. This survey provides an extensive overview of QML applications deployed on cloud infrastructures. It examines core QML techniques such as Variational Quantum Circuits (VQC), Quantum Support Vector Machines (QSVM), and quantum optimization-based learning approaches, while analyzing their deployment across major cloud environments. Additionally, a comparative assessment of prominent quantum cloud platforms—including IBM Quantum, Amazon Web Service AWS Braket, Microsoft Azure Quantum, and other emerging services—is presented, emphasizing their system architectures, software development frameworks, and compatibility with classical machine learning tools. Moreover, the paper explores key technical challenges such as error sensitivity, scalability limitations, data encoding complexity, network latency, and economic factors associated with cloud-based quantum computation. Finally, it identifies prospective research directions, including the advancement of fault-tolerant quantum systems, the development of standardized evaluation benchmarks for QML models, and the integration of quantum learning techniques with large-scale data analytics ecosystems. This study supports Sustainable Development Goal 9 (Industry, Innovation and Infrastructure) by promoting technological innovation and strengthening resilient digital infrastructure through the integration of quantum computing with cloud platforms.
Advancements in quantum computing have sparked a systematic evaluation of quantum machine learning (QML) models, focusing on their behavior and performance across diverse learning tasks such as classification, regression, generative modeling, and feature compression. The primary focus is on assessing the performance and behavior of various QML architectures-including Quantum Neural Networks (QNNs), Quantum Support Vector Classifiers (QSVCs), Quantum Generative Adversarial Networks (QGANs), Quantum Convolutional Neural Networks (QCNNs), and Quantum Autoencoders (QAEs)-implemented using Qiskit's machine learning library. This evaluation is conducted using an existing benchmarking framework that ensures consistent experimental conditions and supports flexible evaluation strategies. To provide additional context and insight, these quantum models are compared against classical counterparts using widely adopted implementations. Experiments are carried out on a range of datasets, including structured and low-dimensional examples like Bars and Stripes, as well as more complex datasets like Downscaled MNIST and synthetic Hyperplanes. By providing a standardized evaluation of these models within the Qiskit framework, this study establishes a reproducible performance baseline and highlights key challenges for QML systems in the noisy intermediate-scale quantum (NISQ) era.
The integration of quantum computing and machine learning has ushered in a transformative era in data analysis, characterized by the development of Quantum Machine Learning (QML). This chapter explores the pivotal role of standardization in evaluation practices within QML research, addressing the challenges posed by inconsistent metrics, diverse benchmarking standards, and reproducibility concerns. Emphasis was placed on the establishment of unified evaluation metrics tailored to quantum algorithms, which can enhance the comparability of results across studies. Additionally, the chapter advocates for the creation of standardized benchmarking frameworks and reproducibility guidelines to bolster the reliability of QML findings. Collaborative initiatives within the research community are encouraged to promote knowledge sharing and best practices. By fostering standardized evaluation practices, this chapter aims to enhance the credibility and impact of QML research, paving the way for innovative advancements in various applications, including finance, healthcare, and artificial intelligence.
As the quantum computing community gravitates towards understanding the practical benefits of quantum computers, having a clear definition and evaluation scheme for assessing practical quantum advantage in the context of specific applications is paramount. Generative modeling, for example, is a widely accepted natural use case for quantum computers, and yet has lacked a concrete approach for quantifying success of quantum models over classical ones. In this work, we construct a simple and unambiguous approach to probe practical quantum advantage for generative modeling by measuring the algorithm's generalization performance. Using the sample-based approach proposed here, any generative model, from state-of-the-art classical generative models such as generative adversarial networks (GANs) to quantum models such as quantum circuit born machines, can be evaluated under the same conditions on a concrete well-defined framework. In contrast to other sample-based metrics for probing practical generalization, we leverage constrained optimization problems (e.g., cardinality-constrained problems) and use these discrete datasets to define specific metrics capable of unambiguously measuring the quality of the samples and the model's generalization capabilities for generating data beyond the training set, but still within the valid solution space. Additionally, our metrics can diagnose trainability issues such as mode collapse and overfitting, as we illustrate when comparing GANs to quantum-inspired models built out of tensor networks. Our simulation results show that our quantum-inspired models have up to a 68 times enhancement in generating unseen unique and valid samples compared to the GANs explored here, and a ratio of 61:2 for generating samples with better quality than those observed in the training set. We foresee these metrics as valuable tools for rigorously defining practical quantum advantage in the domain of generative modeling, where finding a winning model remains an ongoing challenge as new architectures are developed.