Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2504.08331 · npj Quantum Information

Scalable conflict-free bandit algorithm using a quantum optical setup

Kohei Konaka, André Röhm, Takatomo Mihana, Ryoichi Horisaki

Published Mar 30, 2026• 1 citation
arXivHTMLPDF

Abstract

Abstract Quantum optics utilizes the unique properties of light for computation or communication. In this work, we explore its ability to solve certain reinforcement learning tasks, with a particular view towards the scalability of the approach. Our method utilizes the Orbital Angular Momentum (OAM) of photons to solve the Competitive Multi-Armed Bandit (CMAB) problem while maximizing rewards. In particular, we encode each player’s preferences in the OAM amplitudes, while the phases are optimized to avoid conflicts. We find that the proposed system is capable of solving the CMAB problem with a scalable number of options and demonstrates improved performance over existing techniques. Our method utilizes quantum interference to guarantee conflict avoidance using purely physical attributes of light in a way impossible for a classical setup. As an example of a system with simple rules for solving complex tasks, our OAM-based method adds to the repertoire of functionality of quantum optics.

DOI
10.1038/s41534-026-01201-6

Related papers

Conflict-free collective stochastic decision making by orbital angular momentum of photons through quantum interference

Takashi Amakasu, Nicolas Chauvet, Guillaume Bachelier +3 more

In recent cross-disciplinary studies involving both optics and computing, single-photon-based decision-making has been demonstrated by utilizing the wave-particle duality of light to solve multi-armed bandit problems. Furthermore, entangled-photon-based decision-making has managed to solve a competitive multi-armed bandit problem in such a way that conflicts of decisions among players are avoided while ensuring equality. However, as these studies are based on the polarization of light, the number of available choices is limited to two, corresponding to two orthogonal polarization states. Here we propose a scalable principle to solve competitive decision-making situations by using the orbital angular momentum of photons based on its high dimensionality, which theoretically allows an unlimited number of arms. Moreover, by extending the Hong-Ou-Mandel effect to more than two states, we theoretically establish an experimental configuration able to generate multi-photon states with orbital angular momentum and conditions that provide conflict-free selections at every turn. We numerically examine total rewards regarding three-armed bandit problems, for which the proposed strategy accomplishes almost the theoretical maximum, which is greater than a conventional mixed strategy intending to realize Nash equilibrium. This is thanks to the quantum interference effect that achieves no-conflict selections, even in the exploring phase to find the best arms.

arXiv:2107.00877•Scientific Reports•Oct 26, 2021•16 citations

Bandit Approach to Conflict-Free Parallel Q-Learning in View of Photonic Implementation

Hiroaki Shinkawa, Nicolas Chauvet, André Röhm +4 more

Recently, extensive studies on photonic reinforcement learning to accelerate the process of calculation by exploiting the physical nature of light have been conducted. Previous studies utilized quantum interference of photons to achieve collective decision-making without choice conflicts when solving the competitive multi-armed bandit problem, a fundamental example in reinforcement learning. However, the bandit problem deals with a static environment where the agent’s actions do not influence the reward probabilities. This study aims to extend the conventional approach to a more general type of parallel reinforcement learning targeting the grid world problem. Unlike the conventional approach, the proposed scheme deals with a dynamic environment where the reward changes because of the agent’s actions. A successful photonic reinforcement learning scheme requires both a photonic system that contributes to the quality of learning and a suitable algorithm. This study proposes a novel learning algorithm, a modified bandit Q-learning method, in view of a potential photonic implementation. Here, state–action pairs in the environment are regarded as slot machines in the context of the bandit problem and a change in Q -value is regarded as the reward of the bandit problem. We perform numerical simulations to validate the effectiveness of the bandit algorithm. In addition, we propose a parallel architecture in which multiple agents are indirectly connected through quantum interference of light and quantum principles ensure the conflict-free property of state–action pair selections among agents. We demonstrate that parallel reinforcement learning can be accelerated owing to conflict avoidance among multiple agents.

arXiv:2212.09926•Intelligent Computing•Jan 1, 2023•3 citations

Decision Making Photonics: Solving Bandit Problems Using Photons

Makoto Naruse, Nicolas Chauvet, Atsushi Uchida +4 more

Decision making in dynamically changing uncertain environments is one of the most important elements in information and communications technology, in applications ranging from resource assignment in data centers to wireless communications and search functions. Here, we review our research on the efficient physical realization or acceleration of decision making using photonics. The problem of interest is the multi-armed bandit (MAB) problem, where the purpose is to maximize the total rewards in unknown environments that involve difficult tradeoffs in the so-called exploration-exploitation dilemma. We describe the principle of solving MAB problems by utilizing the wave-particle duality of single photons in which the probabilistic attributes of single light quanta are employed for exploration. The principle is transformed into ultrafast laser chaos, where the chaotically oscillating irregular time series provides fast and scalable decision making abilities. The problem becomes even more difficult when multiple players are involved, in the so-called competitive MAB (CMAB) problem, where the expected value is related to maximizing the social benefit and ensuring fairness among individuals. We demonstrate that entangled photons can resolve the CMAB problem. Theoretical studies on photonic decision making are also reviewed, showing that six entities in total interact with each other according to an octahedral structure.

IEEE Journal of Selected Topics in Quantum Electronics•Jul 17, 2019•26 citations

Entangled and correlated photon mixed strategy for social decision making

Shion Maeda, Nicolas Chauvet, Hayato Saigo +4 more

Collective decision making is important for maximizing total benefits while preserving equality among individuals in the competitive multi-armed bandit (CMAB) problem, wherein multiple players try to gain higher rewards from multiple slot machines. The CMAB problem represents an essential aspect of applications such as resource management in social infrastructure. In a previous study, we theoretically and experimentally demonstrated that entangled photons can physically resolve the difficulty of the CMAB problem. This decision-making strategy completely avoids decision conflicts while ensuring equality. However, decision conflicts can sometimes be beneficial if they yield greater rewards than non-conflicting decisions, indicating that greedy actions may provide positive effects depending on the given environment. In this study, we demonstrate a mixed strategy of entangled- and correlated-photon-based decision-making so that total rewards can be enhanced when compared to the entangled-photon-only decision strategy. We show that an optimal mixture of entangled- and correlated-photon-based strategies exists depending on the dynamics of the reward environment as well as the difficulty of the given problem. This study paves the way for utilizing both quantum and classical aspects of photons in a mixed manner for decision making and provides yet another example of the supremacy of mixed strategies known in game theory, especially in evolutionary game theory.

arXiv:2010.13086•Scientific Reports•Mar 1, 2021•5 citations

Training a Quantum Neural Network to Solve the Contextual Multi-Armed Bandit Problem

Wei Hu, James Lee Hu

Artificial intelligence has permeated all aspects of our lives today. However, to make AI behave like real AI, the critical bottleneck lies in the speed of computing. Quantum computers employ the peculiar and unique properties of quantum states such as superposition, entanglement, and interference to process information in ways that classical computers cannot. As a new paradigm of computation, quantum computers are capable of performing tasks intractable for classical processors, thus providing a quantum leap in AI research and making the development of real AI a possibility. In this regard, quantum machine learning not only enhances the classical machine learning approach but more importantly it provides an avenue to explore new machine learning models that have no classical counterparts. The qubit-based quantum computers cannot naturally represent the continuous variables commonly used in machine learning, since the measurement outputs of qubit-based circuits are generally discrete. Therefore, a continuous-variable (CV) quantum architecture based on a photonic quantum computing model is selected for our study. In this work, we employ machine learning and optimization to create photonic quantum circuits that can solve the contextual multi-armed bandit problem, a problem in the domain of reinforcement learning, which demonstrates that quantum reinforcement learning algorithms can be learned by a quantum device.

Natural Science•Jan 1, 2019•9 citations

Entangled N-photon states for fair and optimal social decision making

Nicolas Chauvet, Guillaume Bachelier, S. Huant +3 more

Situations involving competition for resources among entities can be modeled by the competitive multi-armed bandit (CMAB) problem, which relates to social issues such as maximizing the total outcome and achieving the fairest resource repartition among individuals. In these respects, the intrinsic randomness and global properties of quantum states provide ideal tools for obtaining optimal solutions to this problem. Based on the previous study of the CMAB problem in the two-arm, two-player case, this paper presents the theoretical principles necessary to find polarization-entangled N-photon states that can optimize the total resource output while ensuring equality among players. These principles were applied to two-, three-, four-, and five-player cases by using numerical simulations to reproduce realistic configurations and find the best strategies to overcome potential misalignment between the polarization measurement systems of the players. Although a general formula for the N-player case is not presented here, general derivation rules and a verification algorithm are proposed. This report demonstrates the potential usability of quantum states in collective decision making with limited, probabilistic resources, which could serve as a first step toward quantum-based resource allocation systems.

arXiv:2007.09146•Scientific Reports•Nov 24, 2020•11 citations
Abstract Quantum optics utilizes the unique properties of light for computation or communication. In this work, we explore its ability to solve certain reinforcement learning tasks, with a particular view towards the scalability of the approach. Our method utilizes the Orbital Angular Momentum (OAM) of photons to solve the Competitive Multi-Armed Bandit (CMAB) problem while maximizing rewards. In particular, we encode each player’s preferences in the OAM amplitudes, while the phases are optimized to avoid conflicts. We find that the proposed system is capable of solving the CMAB problem with a scalable number of options and demonstrates improved performance over existing techniques. Our method utilizes quantum interference to guarantee conflict avoidance using purely physical attributes of light in a way impossible for a classical setup. As an example of a system with simple rules for solving complex tasks, our OAM-based method adds to the repertoire of functionality of quantum optics.
In recent cross-disciplinary studies involving both optics and computing, single-photon-based decision-making has been demonstrated by utilizing the wave-particle duality of light to solve multi-armed bandit problems. Furthermore, entangled-photon-based decision-making has managed to solve a competitive multi-armed bandit problem in such a way that conflicts of decisions among players are avoided while ensuring equality. However, as these studies are based on the polarization of light, the number of available choices is limited to two, corresponding to two orthogonal polarization states. Here we propose a scalable principle to solve competitive decision-making situations by using the orbital angular momentum of photons based on its high dimensionality, which theoretically allows an unlimited number of arms. Moreover, by extending the Hong-Ou-Mandel effect to more than two states, we theoretically establish an experimental configuration able to generate multi-photon states with orbital angular momentum and conditions that provide conflict-free selections at every turn. We numerically examine total rewards regarding three-armed bandit problems, for which the proposed strategy accomplishes almost the theoretical maximum, which is greater than a conventional mixed strategy intending to realize Nash equilibrium. This is thanks to the quantum interference effect that achieves no-conflict selections, even in the exploring phase to find the best arms.
Recently, extensive studies on photonic reinforcement learning to accelerate the process of calculation by exploiting the physical nature of light have been conducted. Previous studies utilized quantum interference of photons to achieve collective decision-making without choice conflicts when solving the competitive multi-armed bandit problem, a fundamental example in reinforcement learning. However, the bandit problem deals with a static environment where the agent’s actions do not influence the reward probabilities. This study aims to extend the conventional approach to a more general type of parallel reinforcement learning targeting the grid world problem. Unlike the conventional approach, the proposed scheme deals with a dynamic environment where the reward changes because of the agent’s actions. A successful photonic reinforcement learning scheme requires both a photonic system that contributes to the quality of learning and a suitable algorithm. This study proposes a novel learning algorithm, a modified bandit Q-learning method, in view of a potential photonic implementation. Here, state–action pairs in the environment are regarded as slot machines in the context of the bandit problem and a change in Q -value is regarded as the reward of the bandit problem. We perform numerical simulations to validate the effectiveness of the bandit algorithm. In addition, we propose a parallel architecture in which multiple agents are indirectly connected through quantum interference of light and quantum principles ensure the conflict-free property of state–action pair selections among agents. We demonstrate that parallel reinforcement learning can be accelerated owing to conflict avoidance among multiple agents.
Decision making in dynamically changing uncertain environments is one of the most important elements in information and communications technology, in applications ranging from resource assignment in data centers to wireless communications and search functions. Here, we review our research on the efficient physical realization or acceleration of decision making using photonics. The problem of interest is the multi-armed bandit (MAB) problem, where the purpose is to maximize the total rewards in unknown environments that involve difficult tradeoffs in the so-called exploration-exploitation dilemma. We describe the principle of solving MAB problems by utilizing the wave-particle duality of single photons in which the probabilistic attributes of single light quanta are employed for exploration. The principle is transformed into ultrafast laser chaos, where the chaotically oscillating irregular time series provides fast and scalable decision making abilities. The problem becomes even more difficult when multiple players are involved, in the so-called competitive MAB (CMAB) problem, where the expected value is related to maximizing the social benefit and ensuring fairness among individuals. We demonstrate that entangled photons can resolve the CMAB problem. Theoretical studies on photonic decision making are also reviewed, showing that six entities in total interact with each other according to an octahedral structure.
Collective decision making is important for maximizing total benefits while preserving equality among individuals in the competitive multi-armed bandit (CMAB) problem, wherein multiple players try to gain higher rewards from multiple slot machines. The CMAB problem represents an essential aspect of applications such as resource management in social infrastructure. In a previous study, we theoretically and experimentally demonstrated that entangled photons can physically resolve the difficulty of the CMAB problem. This decision-making strategy completely avoids decision conflicts while ensuring equality. However, decision conflicts can sometimes be beneficial if they yield greater rewards than non-conflicting decisions, indicating that greedy actions may provide positive effects depending on the given environment. In this study, we demonstrate a mixed strategy of entangled- and correlated-photon-based decision-making so that total rewards can be enhanced when compared to the entangled-photon-only decision strategy. We show that an optimal mixture of entangled- and correlated-photon-based strategies exists depending on the dynamics of the reward environment as well as the difficulty of the given problem. This study paves the way for utilizing both quantum and classical aspects of photons in a mixed manner for decision making and provides yet another example of the supremacy of mixed strategies known in game theory, especially in evolutionary game theory.
Artificial intelligence has permeated all aspects of our lives today. However, to make AI behave like real AI, the critical bottleneck lies in the speed of computing. Quantum computers employ the peculiar and unique properties of quantum states such as superposition, entanglement, and interference to process information in ways that classical computers cannot. As a new paradigm of computation, quantum computers are capable of performing tasks intractable for classical processors, thus providing a quantum leap in AI research and making the development of real AI a possibility. In this regard, quantum machine learning not only enhances the classical machine learning approach but more importantly it provides an avenue to explore new machine learning models that have no classical counterparts. The qubit-based quantum computers cannot naturally represent the continuous variables commonly used in machine learning, since the measurement outputs of qubit-based circuits are generally discrete. Therefore, a continuous-variable (CV) quantum architecture based on a photonic quantum computing model is selected for our study. In this work, we employ machine learning and optimization to create photonic quantum circuits that can solve the contextual multi-armed bandit problem, a problem in the domain of reinforcement learning, which demonstrates that quantum reinforcement learning algorithms can be learned by a quantum device.
Situations involving competition for resources among entities can be modeled by the competitive multi-armed bandit (CMAB) problem, which relates to social issues such as maximizing the total outcome and achieving the fairest resource repartition among individuals. In these respects, the intrinsic randomness and global properties of quantum states provide ideal tools for obtaining optimal solutions to this problem. Based on the previous study of the CMAB problem in the two-arm, two-player case, this paper presents the theoretical principles necessary to find polarization-entangled N-photon states that can optimize the total resource output while ensuring equality among players. These principles were applied to two-, three-, four-, and five-player cases by using numerical simulations to reproduce realistic configurations and find the best strategies to overcome potential misalignment between the polarization measurement systems of the players. Although a general formula for the N-player case is not presented here, general derivation rules and a verification algorithm are proposed. This report demonstrates the potential usability of quantum states in collective decision making with limited, probabilistic resources, which could serve as a first step toward quantum-based resource allocation systems.