Distributed hybrid quantum-classical performance prediction for hyperparameter optimization
Eric Wulff, Juan Pablo García Amboage, Marcel Aach, Þorsteinn Elí Gíslason, Thorsteinn Kristinn Ingolfsson, Tomas Kristinn Ingolfsson, Edoardo Pasetto, Amer Delilbasic, Morris Riedel, Rakesh Sarma, Maria Girone, Andreas Lintermann
Abstract Hyperparameter optimization (HPO) of neural networks is a computationally expensive procedure, which requires a large number of different model configurations to be trained. To reduce such costs, this work presents a distributed, hybrid workflow, that runs the training of the neural networks on multiple graphics processing units (GPUs) on a classical supercomputer, while predicting the configurations’ performance with quantum-trained support vector regression (QT-SVR) on a quantum annealer (QA). The workflow is shown to run on up to 50 GPUs and a QA at the same time, completely automating the communication between the classical and the quantum systems. The approach is evaluated extensively on several benchmarking datasets from the computer vision (CV), high-energy physics (HEP), and natural language processing (NLP) domains. Empirical results show that resource costs for performing HPO can be reduced by up to 9% when using the hybrid workflow with performance prediction, compared to using a plain HPO algorithm without performance prediction. Additionally, the workflow obtains similar and in some cases even better accuracy of the final hyperparameter configuration, when combining multiple heuristically obtained predictions from the QA, compared to using just a single classically obtained prediction. The results highlight the potential of hybrid quantum-classical machine learning algorithms. The workflow code is made available open-source to foster adoption in the community.
Abstract Hyperparameter optimization (HPO) of neural networks is a computationally expensive procedure, which requires a large number of different model configurations to be trained. To reduce such costs, this work presents a distributed, hybrid workflow, that runs the training of the neural networks on multiple graphics processing units (GPUs) on a classical supercomputer, while predicting the configurations’ performance with quantum-trained support vector regression (QT-SVR) on a quantum annealer (QA). The workflow is shown to run on up to 50 GPUs and a QA at the same time, completely automating the communication between the classical and the quantum systems. The approach is evaluated extensively on several benchmarking datasets from the computer vision (CV), high-energy physics (HEP), and natural language processing (NLP) domains. Empirical results show that resource costs for performing HPO can be reduced by up to 9% when using the hybrid workflow with performance prediction, compared to using a plain HPO algorithm without performance prediction. Additionally, the workflow obtains similar and in some cases even better accuracy of the final hyperparameter configuration, when combining multiple heuristically obtained predictions from the QA, compared to using just a single classically obtained prediction. The results highlight the potential of hybrid quantum-classical machine learning algorithms. The workflow code is made available open-source to foster adoption in the community.
Use of AI/ML is associated with more and more computing power, which often necessitates the application to guzzle energy and take time to train. The paper presents a Quantum-Enhanced High-Performance Computing (Q-HPC) system that will combine traditional HPC units with quantum-assisted optimization to enhance the model training in addition to the predictive accuracy and energy efficiency. The framework that could be used to work with such large volumes of data is multi- GPU/CPU parallelization, and the optimization of parameters and hyperparameters can be implemented with the help of quantum-inspired algorithms. This will lead to a hybrid computation balancing dynamically the classical and quantum computations. Q-HPC was experimented with multiple AI/ML model types, such as convolutional networks, transformer models, graph neural networks and reinforcement learning agents, in which cases it was observed to traverse to the solution faster, more accurately and used less energy than the traditional HPC. It can also be said that the framework is dynamically adaptable and sustainable, i.e. it could be deployed to a massive diversifying range of tasks of AI/ML. The suggested model is a combination of the performance and scalability of the classical HPC with the optimization performance of quantum computing in order to create a new and valuable approach to the next generation AI. It deals with the issues of performance and environmental concerns of high-performance computing.
In recent years, the field of machine learning has witnessed a paradigm shift with the emergence of hybrid quantum-classical neural networks. These networks combine the power of classical neural networks with the computational advantages offered by quantum variational circuits. This study presents a comprehensive survey of hybrid quantum-classical machine learning algorithms, focusing on their architecture, performance, and hyperparameter optimization techniques. Introducing the concept of hybrid quantum-classical machine learning and its significance in contemporary image classification tasks. It explores the theoretical foundations of quantum variational circuits and their integration with classical neural network architectures. Special attention is given to prominent algorithms such as Quantum Convolutional Neural Networks, Quanvolutional Neural Networks, and Quantum ResNet, each offering unique advantages in image classification tasks. A major aspect covered in this study is the hyperparameter optimization techniques employed in hybrid quantum-classical machine learning. The chapter discusses quantum-inspired optimization methods tailored for efficiently tuning the hyperparameters of complex neural network models. Through comparative analysis, the effectiveness of these optimization techniques is demonstrated in reducing computational overhead and improving model performance. Moreover, the study also delves into experimental results obtained from benchmarking hybrid quantum-classical models against traditional classical approaches. It highlights the accuracy variations observed across different quantum architectures and explores the impact of architectural permutations on model convergence. Insights gained from these experiments provide valuable guidance for researchers and practitioners seeking to leverage hybrid quantum-classical neural networks in real-world applications.
As machine learning applications scale, optimization challenges in training deep models have intensified. Traditional gradient-based methods can struggle with high-dimensional, multimodal loss landscapes. This study proposes a quantum-inspired optimization model designed for large-scale ML workloads deployed in cloud environments. The framework incorporates simulated quantum tunneling, stochastic amplitude updates and adaptive annealing schedules to escape local minima more efficiently. It integrates seamlessly with distributed training systems such as Kubernetes and PyTorch Distributed. Experiments were conducted on vision and language benchmarks, including ImageNet and WikiText-103. Results show a 9 percent improvement in convergence speed and a 6.8 percent increase in final model accuracy compared with AdamW and Ranger optimizers. The method also demonstrated resilience to noisy gradients and unstable cloud GPU environments. Cost analysis highlights that fewer training epochs reduce compute expenses by up to 14 percent. These findings suggest that quantum-inspired optimization can enhance ML performance even without quantum hardware. Future work aims to integrate hybrid GPU-QPU cloud architectures.
Abstract Image recognition is one of the primary applications of machine learning algorithms. Nevertheless, machine learning models used in modern image recognition systems consist of millions of parameters that usually require significant computational time to be adjusted. Moreover, adjustment of model hyperparameters leads to additional overhead. Because of this, new developments in machine learning models and hyperparameter optimization techniques are required. This paper presents a quantum-inspired hyperparameter optimization technique and a hybrid quantum-classical machine learning model for supervised learning. We benchmark our hyperparameter optimization method over standard black-box objective functions and observe performance improvements in the form of reduced expected run times and fitness in response to the growth in the size of the search space. We test our approaches in a car image classification task and demonstrate a full-scale implementation of the hybrid quantum ResNet model with the tensor train hyperparameter optimization. Our tests show a qualitative and quantitative advantage over the corresponding standard classical tabular grid search approach used with a deep neural network ResNet34. A classification accuracy of 0.97 was obtained by the hybrid model after 18 iterations, whereas the classical model achieved an accuracy of 0.92 after 75 iterations.
Modern renewable power operations can be enhanced by integrating deep neural networks, particularly for forecasting solar irradiance. Recent advancements in quantum computing have shown potential improvements in classical deep neural networks. However, current challenges with quantum hardware, such as susceptibility to noise and decoherence, pose risks to its practicality. Hybrid quantum neural networks (HQNNs) are found to mitigate these issues, especially when integrated with graphics processing unit (GPU)-based pipelines. This paper presents a comparative study of different software platforms for developing HQNNs, using multi-location very short-term solar irradiance forecasting as an example. A classical benchmark model is initially designed based on statistical analysis of a 10-minute resolution solar irradiance dataset, with its parameters further optimized using Bayesian Optimization. The experimental design of this paper includes a loss comparison between classical neural networks and HQNNs across different seasons and a performance comparison between Pennylane, Torchquantum, and CUDA Quantum (CUDA-Q) as HQNN development platforms. Experimental results show that HQNNs achieve up to a 92.30% improvement in testing loss compared to classical neural networks. Regarding HQNN development platforms, Pennylane shows an 81.54% testing loss reduction from classical models, Torchquantum shows a 90.34% improvement, and CUDA-Q shows a 92.30% improvement in testing loss. Implementing hardware acceleration libraries for GPU-based state vector simulation demonstrates an approximate 275% speedup in average latency per epoch, a 218% speedup in inference time, and a 10.20% improvement in testing loss compared to CPU-based simulations. CUDA-Q achieves a training time 2.7 times shorter and an inference time 2.9 times shorter compared to Pennylane, while it is 32.3 times faster in training and 31 times faster in inference compared to Torchquantum.
The recent rise in the number of data-intensive applications has revealed some inherent limitations of traditional machine learning, such as high training costs, slow convergence rates, and the uncapable of scalability of traditional machine learning in high-dimensional spaces. Bottlenecks are caused by the iterative optimization procedure, kernel evaluations, and similarity computations that are not efficiently supported by traditional hardware. Although recent advances in quantum computing, such as the Variational Quantum Algorithm, quantum kernels, and parameterized quantum circuits, hold promise for acceleration, the application of quantum speedup is presently hindered by noisy intermediate-scale quantum computing. This research provides a hybrid quantum-classical acceleration method that integrates quantum methods with selected computationally intensive components of classical machine learning systems rather than attempting to entirely replace them. Carried out a comprehensive analysis of the performance of well-known algorithms like Support Vector Machines, k-means clustering, and gradient-based neural networks in classical and quantum-assisted hybrid scenarios. The quantum resources are greatly beneficial for particular sub-tasks such as optimization, approximation of similarity, and exploration of the feature space, with the remaining learning process performed classically. Our analysis reveals that hybrid methods can offer a tangible acceleration and improved convergence behaviour in particular scenarios without sacrificing accuracy. Moreover, the analysis investigates the trade-offs associated with the utilization of quantum resources, robustness to noise, and orchestration costs.