Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2507.04438

Quantum Algorithms for Bandits with Knapsacks with Improved Regret and Time Complexities

Su, Yuexin, Ziyi Yang, Huang, Peiyuan, Tongyang Li, Yinyu Ye

Published Jul 6, 2025• 2 citations
arXivHTMLPDF

Abstract

Bandits with knapsacks (BwK) constitute a fundamental model that combines aspects of stochastic integer programming with online learning. Classical algorithms for BwK with a time horizon $T$ achieve a problem-independent regret bound of ${O}(\sqrt{T})$ and a problem-dependent bound of ${O}(\log T)$. In this paper, we initiate the study of the BwK model in the setting of quantum computing, where both reward and resource consumption can be accessed via quantum oracles. We establish both problem-independent and problem-dependent regret bounds for quantum BwK algorithms. For the problem-independent case, we demonstrate that a quantum approach can improve the classical regret bound by a factor of $(1+\sqrt{B/\mathrm{OPT}_\mathrm{LP}})$, where $B$ is budget constraint in BwK and $\mathrm{OPT}_{\mathrm{LP}}$ denotes the optimal value of a linear programming relaxation of the BwK problem. For the problem-dependent setting, we develop a quantum algorithm using an inexact quantum linear programming solver. This algorithm achieves a quadratic improvement in terms of the problem-dependent parameters, as well as a polynomial speedup of time complexity on problem's dimensions compared to classical counterparts. Compared to previous works on quantum algorithms for multi-armed bandits, our study is the first to consider bandit models with resource constraints and hence shed light on operations research.

Related papers

Quantum Multi-Armed Bandits and Stochastic Linear Bandits Enjoy Logarithmic Regrets

Zongqi Wan, Zhijie Zhang, Tongyang Li +2 more

Multi-arm bandit (MAB) and stochastic linear bandit (SLB) are important models in reinforcement learning, and it is well-known that classical algorithms for bandits with time horizon T suffer from the regret of at least the square root of T. In this paper, we study MAB and SLB with quantum reward oracles and propose quantum algorithms for both models with the order of the polylog T regrets, exponentially improving the dependence in terms of T. To the best of our knowledge, this is the first provable quantum speedup for regrets of bandit problems and in general exploitation in reinforcement learning. Compared to previous literature on quantum exploration algorithms for MAB and reinforcement learning, our quantum input model is simpler and only assumes quantum oracles for each individual arm.

arXiv:2205.14988•Proceedings of the AAAI Conference on Artificial Intelligence•Jun 26, 2023•22 citations

Quantum Multi-Armed Bandits and Linear Bandits: Lower Bounds and Algorithms

Maoli Liu, Zhuohua Li, John C. S. Lui

We study quantum multi-armed bandits (QMAB) and quantum linear bandits (QLB), where the learner queries each arm or action through a quantum reward oracle or its inverse. Prior work gives algorithms over horizon $T$ with regret $O(K\log T)$ for QMAB with $K$ arms and $O(d^2\operatorname{polylog} T)$ for $d$-dimensional QLB. This leaves open the optimal dependence on $K$ and $T$ and whether the dependence on $d$ can be further improved. In this work, we prove the first tight minimax regret bound of $Θ(K\log(1+T/K))$ for QMAB and the first lower bound of $Ω(d\log(1+T/d))$ for finite-action QLB, ruling out regret independent of $T$. Our lower bounds rely on a high-confidence single-arm quantum testing lower bound for distinguishing a fixed reward mean from an interval of alternatives. A bandit-to-testing reduction then lifts it to the QMAB lower bound, while a linear embedding gives the finite-action QLB lower bound. The matching QMAB upper bound is obtained using a tail bound for the Quantum Monte Carlo (QMC) estimator. For finite-action QLB, we propose a phased elimination algorithm that combines a low-bias low-variance quantum mean estimator with a small-support $G$-optimal design through a query allocation matched to the design weights. When the action set has size $\operatorname{poly}(d)$, its regret is nearly linear in $d$ and matches our lower bound up to polylogarithmic factors.

cs.LGCross-listedarXiv:2608.14319•Aug 17, 2026•0 citations

Quantum Heavy-tailed Bandits

Wu, Yulian, Chaowen Guan, Vaneet Aggarwal +1 more

In this paper, we study multi-armed bandits (MAB) and stochastic linear bandits (SLB) with heavy-tailed rewards and quantum reward oracle. Unlike the previous work on quantum bandits that assumes bounded/sub-Gaussian distributions for rewards, here we investigate the quantum bandits problem under a weaker assumption that the distributions of rewards only have bounded $(1+v)$-th moment for some $v\in (0,1]$. In order to achieve regret improvements for heavy-tailed bandits, we first propose a new quantum mean estimator for heavy-tailed distributions, which is based on the Quantum Monte Carlo Mean Estimator and achieves a quadratic improvement of estimation error compared to the classical one. Based on our quantum mean estimator, we focus on quantum heavy-tailed MAB and SLB and propose quantum algorithms based on the Upper Confidence Bound (UCB) framework for both problems with $\Tilde{O}(T^{\frac{1-v}{1+v}})$ regrets, polynomially improving the dependence in terms of $T$ as compared to classical (near) optimal regrets of $\Tilde{O}(T^{\frac{1}{1+v}})$, where $T$ is the number of rounds. Finally, experiments also support our theoretical results and show the effectiveness of our proposed methods.

arXiv:2301.09680•Jan 23, 2023•15 citations

An Efficient Quantum Algorithm for Sparse Linear Bandits via Smoothed Exploration

Jiarui Hu, Wenzhi Chen, Shipra Singh +1 more

Stochastic linear bandits are an important model for sequential decision-making, for which classical algorithms typically incur a regret scaling of $\tilde {\mathcal{O}}(d\sqrt T )$. Recent quantum linear bandit algorithms use quantum mean estimation to reduce the dependence on the time horizon T, achieving O(d2poly(logT)) regret, but they do not exploit sparsity inherent in many systems and still retain polynomial dependence on the ambient dimension d. We propose Q-Lasso-OFU, a two-phase quantum algorithm for sparse linear bandits. With sparsity s, our regret bound depends logarithmically on the ambient dimension d and polynomially on the sparsity s. Suppose that the nonzero components of θ⋆are lower bounded in magnitude by γ0. Once the horizon exceeds a finite identification time, the algorithm commits to an optimal action and achieves horizon-independent problem-dependent regret $\tilde {\mathcal{O}}\left( {{s^4}\log d\log \left( {1/{\gamma _0}} \right)} \right) + \tilde {\mathcal{O}}\left( {{s^4}\log d/{\gamma _0}} \right)$. Specifically, during the exploration phase, the regret from estimation error is $\tilde {\mathcal{O}}\left( {{s^4}\log d\log T} \right)$, logarithmic in both T and d, and after that the regret is zero. Compared with prior works in quantum bandits, our bound has a logarithmic dependence on d under sparsity. Numerical experiments support our theoretical results.

IEEE International Conference on Quantum Control, Computing and Learning (qCCL)•Jun 1, 2026•0 citations

Quantum Bayesian Optimization

Zhongxiang Dai, Gregory Kang Ruey Lau, Arun Verma +3 more

Kernelized bandits, also known as Bayesian optimization (BO), has been a prevalent method for optimizing complicated black-box reward functions. Various BO algorithms have been theoretically shown to enjoy upper bounds on their cumulative regret which are sub-linear in the number T of iterations, and a regret lower bound of Omega(sqrt(T)) has been derived which represents the unavoidable regrets for any classical BO algorithm. Recent works on quantum bandits have shown that with the aid of quantum computing, it is possible to achieve tighter regret upper bounds better than their corresponding classical lower bounds. However, these works are restricted to either multi-armed or linear bandits, and are hence not able to solve sophisticated real-world problems with non-linear reward functions. To this end, we introduce the quantum-Gaussian process-upper confidence bound (Q-GP-UCB) algorithm. To the best of our knowledge, our Q-GP-UCB is the first BO algorithm able to achieve a regret upper bound of O(polylog T), which is significantly smaller than its regret lower bound of Omega(sqrt(T)) in the classical setting. Moreover, thanks to our novel analysis of the confidence ellipsoid, our Q-GP-UCB with the linear kernel achieves a smaller regret than the quantum linear UCB algorithm from the previous work. We use simulations, as well as an experiment using a real quantum computer, to verify that the theoretical quantum speedup achieved by our Q-GP-UCB is also potentially relevant in practice.

arXiv:2310.05373•neural information processing systems•Jan 1, 2023•10 citations

Multi-armed quantum bandits: Exploration versus exploitation when learning properties of quantum states

Josep Lumbreras, Erkka Haapasalo, Marco Tomamichel

We initiate the study of tradeoffs between exploration and exploitation in online learning of properties of quantum states. Given sequential oracle access to an unknown quantum state, in each round, we are tasked to choose an observable from a set of actions aiming to maximize its expectation value on the state (the reward). Information gained about the unknown state from previous rounds can be used to gradually improve the choice of action, thus reducing the gap between the reward and the maximal reward attainable with the given action set (the regret). We provide various information-theoretic lower bounds on the cumulative regret that an optimal learner must incur, and show that it scales at least as the square root of the number of rounds played. We also investigate the dependence of the cumulative regret on the number of available actions and the dimension of the underlying space. Moreover, we exhibit strategies that are optimal for bandits with a finite number of arms and general mixed states.

arXiv:2108.13050•Quantum•Jun 29, 2022•27 citations
Bandits with knapsacks (BwK) constitute a fundamental model that combines aspects of stochastic integer programming with online learning. Classical algorithms for BwK with a time horizon TTT achieve a problem-independent regret bound of O(T){O}(\sqrt{T})O(T​) and a problem-dependent bound of O(log⁡T){O}(\log T)O(logT). In this paper, we initiate the study of the BwK model in the setting of quantum computing, where both reward and resource consumption can be accessed via quantum oracles. We establish both problem-independent and problem-dependent regret bounds for quantum BwK algorithms. For the problem-independent case, we demonstrate that a quantum approach can improve the classical regret bound by a factor of (1+B/OPTLP)(1+\sqrt{B/\mathrm{OPT}_\mathrm{LP}})(1+B/OPTLP​​), where BBB is budget constraint in BwK and OPTLP\mathrm{OPT}_{\mathrm{LP}}OPTLP​ denotes the optimal value of a linear programming relaxation of the BwK problem. For the problem-dependent setting, we develop a quantum algorithm using an inexact quantum linear programming solver. This algorithm achieves a quadratic improvement in terms of the problem-dependent parameters, as well as a polynomial speedup of time complexity on problem's dimensions compared to classical counterparts. Compared to previous works on quantum algorithms for multi-armed bandits, our study is the first to consider bandit models with resource constraints and hence shed light on operations research.
Multi-arm bandit (MAB) and stochastic linear bandit (SLB) are important models in reinforcement learning, and it is well-known that classical algorithms for bandits with time horizon T suffer from the regret of at least the square root of T. In this paper, we study MAB and SLB with quantum reward oracles and propose quantum algorithms for both models with the order of the polylog T regrets, exponentially improving the dependence in terms of T. To the best of our knowledge, this is the first provable quantum speedup for regrets of bandit problems and in general exploitation in reinforcement learning. Compared to previous literature on quantum exploration algorithms for MAB and reinforcement learning, our quantum input model is simpler and only assumes quantum oracles for each individual arm.
We study quantum multi-armed bandits (QMAB) and quantum linear bandits (QLB), where the learner queries each arm or action through a quantum reward oracle or its inverse. Prior work gives algorithms over horizon TTT with regret O(Klog⁡T)O(K\log T)O(KlogT) for QMAB with KKK arms and O(d2polylog⁡T)O(d^2\operatorname{polylog} T)O(d2polylogT) for ddd-dimensional QLB. This leaves open the optimal dependence on KKK and TTT and whether the dependence on ddd can be further improved. In this work, we prove the first tight minimax regret bound of Θ(Klog⁡(1+T/K))Θ(K\log(1+T/K))Θ(Klog(1+T/K)) for QMAB and the first lower bound of Ω(dlog⁡(1+T/d))Ω(d\log(1+T/d))Ω(dlog(1+T/d)) for finite-action QLB, ruling out regret independent of TTT. Our lower bounds rely on a high-confidence single-arm quantum testing lower bound for distinguishing a fixed reward mean from an interval of alternatives. A bandit-to-testing reduction then lifts it to the QMAB lower bound, while a linear embedding gives the finite-action QLB lower bound. The matching QMAB upper bound is obtained using a tail bound for the Quantum Monte Carlo (QMC) estimator. For finite-action QLB, we propose a phased elimination algorithm that combines a low-bias low-variance quantum mean estimator with a small-support GGG-optimal design through a query allocation matched to the design weights. When the action set has size poly⁡(d)\operatorname{poly}(d)poly(d), its regret is nearly linear in ddd and matches our lower bound up to polylogarithmic factors.
In this paper, we study multi-armed bandits (MAB) and stochastic linear bandits (SLB) with heavy-tailed rewards and quantum reward oracle. Unlike the previous work on quantum bandits that assumes bounded/sub-Gaussian distributions for rewards, here we investigate the quantum bandits problem under a weaker assumption that the distributions of rewards only have bounded (1+v)(1+v)(1+v)-th moment for some v∈(0,1]v\in (0,1]v∈(0,1]. In order to achieve regret improvements for heavy-tailed bandits, we first propose a new quantum mean estimator for heavy-tailed distributions, which is based on the Quantum Monte Carlo Mean Estimator and achieves a quadratic improvement of estimation error compared to the classical one. Based on our quantum mean estimator, we focus on quantum heavy-tailed MAB and SLB and propose quantum algorithms based on the Upper Confidence Bound (UCB) framework for both problems with $\Tilde{O}(T^{\frac{1-v}{1+v}})$ regrets, polynomially improving the dependence in terms of TTT as compared to classical (near) optimal regrets of $\Tilde{O}(T^{\frac{1}{1+v}})$, where TTT is the number of rounds. Finally, experiments also support our theoretical results and show the effectiveness of our proposed methods.
Stochastic linear bandits are an important model for sequential decision-making, for which classical algorithms typically incur a regret scaling of O~(dT)\tilde {\mathcal{O}}(d\sqrt T )O~(dT​). Recent quantum linear bandit algorithms use quantum mean estimation to reduce the dependence on the time horizon T, achieving O(d2poly(logT)) regret, but they do not exploit sparsity inherent in many systems and still retain polynomial dependence on the ambient dimension d. We propose Q-Lasso-OFU, a two-phase quantum algorithm for sparse linear bandits. With sparsity s, our regret bound depends logarithmically on the ambient dimension d and polynomially on the sparsity s. Suppose that the nonzero components of θ⋆are lower bounded in magnitude by γ0. Once the horizon exceeds a finite identification time, the algorithm commits to an optimal action and achieves horizon-independent problem-dependent regret O~(s4log⁡dlog⁡(1/γ0))+O~(s4log⁡d/γ0)\tilde {\mathcal{O}}\left( {{s^4}\log d\log \left( {1/{\gamma _0}} \right)} \right) + \tilde {\mathcal{O}}\left( {{s^4}\log d/{\gamma _0}} \right)O~(s4logdlog(1/γ0​))+O~(s4logd/γ0​). Specifically, during the exploration phase, the regret from estimation error is O~(s4log⁡dlog⁡T)\tilde {\mathcal{O}}\left( {{s^4}\log d\log T} \right)O~(s4logdlogT), logarithmic in both T and d, and after that the regret is zero. Compared with prior works in quantum bandits, our bound has a logarithmic dependence on d under sparsity. Numerical experiments support our theoretical results.
Kernelized bandits, also known as Bayesian optimization (BO), has been a prevalent method for optimizing complicated black-box reward functions. Various BO algorithms have been theoretically shown to enjoy upper bounds on their cumulative regret which are sub-linear in the number T of iterations, and a regret lower bound of Omega(sqrt(T)) has been derived which represents the unavoidable regrets for any classical BO algorithm. Recent works on quantum bandits have shown that with the aid of quantum computing, it is possible to achieve tighter regret upper bounds better than their corresponding classical lower bounds. However, these works are restricted to either multi-armed or linear bandits, and are hence not able to solve sophisticated real-world problems with non-linear reward functions. To this end, we introduce the quantum-Gaussian process-upper confidence bound (Q-GP-UCB) algorithm. To the best of our knowledge, our Q-GP-UCB is the first BO algorithm able to achieve a regret upper bound of O(polylog T), which is significantly smaller than its regret lower bound of Omega(sqrt(T)) in the classical setting. Moreover, thanks to our novel analysis of the confidence ellipsoid, our Q-GP-UCB with the linear kernel achieves a smaller regret than the quantum linear UCB algorithm from the previous work. We use simulations, as well as an experiment using a real quantum computer, to verify that the theoretical quantum speedup achieved by our Q-GP-UCB is also potentially relevant in practice.
We initiate the study of tradeoffs between exploration and exploitation in online learning of properties of quantum states. Given sequential oracle access to an unknown quantum state, in each round, we are tasked to choose an observable from a set of actions aiming to maximize its expectation value on the state (the reward). Information gained about the unknown state from previous rounds can be used to gradually improve the choice of action, thus reducing the gap between the reward and the maximal reward attainable with the given action set (the regret). We provide various information-theoretic lower bounds on the cumulative regret that an optimal learner must incur, and show that it scales at least as the square root of the number of rounds played. We also investigate the dependence of the cumulative regret on the number of available actions and the dimension of the underlying space. Moreover, we exhibit strategies that are optimal for bandits with a finite number of arms and general mixed states.