Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2607.09422

Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning

Edwin De Nicolo, Rahul Marchand, Cornelius Carlsson, Pranav Vaidhyanathan, Natalia Ares

Published Jul 10, 2026• 0 citations
arXivHTMLPDF

Abstract

Cooperative multi-agent reinforcement learning is well suited to problems with large parameter spaces and exploitable local structure, such as the tuning of electrostatically-defined quantum-dot arrays. However, if parameter cross-talk is strong, a non-stationary environment from the perspective of any individual agent can destabilize learning - the same effect that plagues manual tuning of such systems. We propose using a factored representation of the action space, learned online, to decouple agents and minimize their interference. Our framework, QADAPT, uses this factorization to efficiently learn shared policies based on local measurements and rewards. With this modular strategy, we achieve zero-shot generalization to unseen quantum device sizes and maintain an approximately constant number of convergence steps to reach target regimes. This work provides a scalable route toward the rapid calibration of large-scale quantum processors.

DOI
10.48550/arxiv.2607.09422

Related papers

Cooperative multi-agent reinforcement learning based on quantum computing

Xiangping Meng

Due to the interactions among the agents in the cooperative multi-agent systems,multi-agent learning problem complexity can rise rapidly with the number of agents or their behavioral sophistication.In order to converge to desirable equilibrium,agents generally require sufficient exploration of strategy space and coordinate their policies to achieve optimal equilibrium.A novel cooperative multi-agent learning method is proposed based on quantum theory and reinforcement learning.This method not only coordinates agents’ behaviors using quantum entanglement and helps agents make action selection under quantum superposition,but also adopts Grover’s searching algorithm which can speed up learning.This method also makes a good tradeoff between exploration and exploitation using probability characteristics of quantum theory.The results of simulated experiments show that quantum theory can be effectively used to cooperative multi-agent reinforcement learning.

Computer Engineering and Applications Journal•Jan 1, 2008•0 citations

Rethinking How to Act: Action-Space Engineering for Reinforcement Learning-Based Circuit Routing in Distributed Quantum Systems

Joost Van Veen, Luise Prielinger, Sebastian Feld

As it becomes increasingly difficult to monolithically scale a quantum processor, distributed quantum computing (DQC) offers an alternative by distributing qubits across multiple smaller interconnected quantum processor modules. In such an architecture, the challenge of quantum circuit compilation shifts from placing and routing qubits within one module to placing, routing and using the qubits efficiently across modules. In order to optimize circuit execution time, the right state-dependent networking decisions must be found, such as when and where to generate shared remote quantum states to support remote operations. Reinforcement learning (RL) provides a natural framework for this problem, generating a compilation policy that can generalize across different circuits. Building on the framework of Promponas et al. (2024), we introduce an agent that combines a novel action-space formulation with effective action-masking strategies. A comprehensive numerical comparison of the two approaches under different coupling constraints shows that our agent achieves improved training and inference performance with a relative reduction in the modeled execution time of up to 35\%.

quant-pharXiv:2605.02389•May 5, 2026•0 citations

Distributed quantum architecture search using multi-agent reinforcement learning

Mikhail Sergeev, Georgii Paradezhenko, Daniil Rabinovich +1 more

Quantum architecture search (QAS) automates the design of parameterized quantum circuits for variational quantum algorithms. The framework finds a well-suited problem-specific structure of a variational ansatz. Among possible implementations of QAS the reinforcement learning (RL) stands out as one of the most promising. Current RL approaches are single-agent-based and show poor scalability with a number of qubits due to the increase of the action space dimension and the computational cost. We propose a novel multi-agent RL algorithm for QAS with each agent acting separately on its own block of a quantum circuit. This procedure allows to significantly accelerate the convergence of the RL-based QAS and reduce its computational cost. We benchmark the proposed algorithm on MaxCut problem on 3-regular graphs and on ground energy estimation for the Schwinger Hamiltonian. In addition, the proposed multi-agent approach naturally fits into the set-up of distributed quantum computing, favoring its implementation on modern intermediate scale quantum devices.

quant-pharXiv:2511.22708•Dec 1, 2025•0 citations

Q-CMAPO: A quantum-classical framework for balancing exploration and exploitation in multi-agent reinforcement learning

Mohammad Taghavi, Javad Vahidi

Abstract In this paper, we propose a novel approach, Q-CMAPO (Quantum-Classical Multi-agent Policy Optimization), for tackling complex decision-making problems in multi-agent systems. By leveraging quantum-inspired optimization techniques, Q-CMAPO efficiently addresses the exploration-exploitation tradeoff, enhancing the scalability and performance of reinforcement learning (RL) algorithms in partially observable, non-stationary environments. We introduce an innovative framework that combines centralized training with decentralized execution (CTDE), enabling seamless cooperation between agents while preserving their autonomy during execution. Through extensive empirical evaluation, including various UAV deployment scenarios, we demonstrate that Q-CMAPO consistently outperforms existing baselines in both computational efficiency and classification accuracy. Our experiments show significant improvements in performance metrics, as well as substantial gains in runtime efficiency and memory utilization. Furthermore, we provide a comprehensive theoretical analysis, proving the convergence and stability of the proposed method in non-stationary environments. We also conduct an ablation study, shedding light on the importance of different components of Q-CMAPO in optimizing agent cooperation. While promising, the proposed approach faces several challenges, including its sensitivity to hyperparameters and scalability in large-scale systems, suggesting opportunities for future refinement and expansion. The integration of Q-CMAPO into real-world applications, such as autonomous robotics, and UAV-based surveillance, opens new avenues for research, bridging the gap between quantum-inspired optimization and practical deployment in multi-agent systems.

Quantum Machine Intelligence•Feb 3, 2026•8 citations

Quantum Multi-Agent Reinforcement Learning for Autonomous Mobility Cooperation

Soohyun Park, Jae Pyoung Kim, Chanyoung Park +2 more

For Industry 4.0 Revolution, cooperative autonomous mobility systems are widely used based on multiagent reinforcement learning (MARL). However, the MARLbased algorithms suffer from huge parameter utilization and convergence difficulties with many agents. To tackle these problems, a quantum MARL (QMARL) algorithm based on the concept of actor-critic network is proposed, which is beneficial in terms of scalability, to deal with the limitations in the noisy intermediatescale quantum (NISQ) era. Additionally, our QMARL is also beneficial in terms of efficient parameter utilization and fast convergence due to quantum supremacy. Note that the reward in our QMARL is defined as task precision over computation time in multiple agents, thus, multi-agent cooperation can be realized. For further improvement, an additional technique for scalability is proposed, which is called projection value measure (PVM). Based on PVM, our proposed QMARL can achieve the highest reward, by reducing the action dimension into a logarithmicscale. Finally, we can conclude that our proposed QMARL with PVM outperforms the other algorithms in terms of efficient parameter utilization, fast convergence, and scalability.

arXiv:2308.01519•IEEE Communications Magazine•Aug 28, 2023•75 citations

Learning to Coordinate via Quantum Entanglement in Multi-Agent Reinforcement Learning

John Gardiner, Orlando Romero, Brendan Tivnan +2 more

The inability to communicate poses a major challenge to coordination in multi-agent reinforcement learning (MARL). Prior work has explored correlating local policies via shared randomness, sometimes in the form of a correlation device, as a mechanism to assist in decentralized decision-making. In contrast, this work introduces the first framework for training MARL agents to exploit shared quantum entanglement as a coordination resource, which permits a larger class of communication-free correlated policies than shared randomness alone. This is motivated by well-known results in quantum physics which posit that, for certain single-round cooperative games with no communication, shared quantum entanglement enables strategies that outperform those that only use shared randomness. In such cases, we say that there is quantum advantage. Our framework is based on a novel differentiable policy parameterization that enables optimization over quantum measurements, together with a novel policy architecture that decomposes joint policies into a quantum coordinator and decentralized local actors. To illustrate the effectiveness of our proposed method, we first show that we can learn, purely from experience, strategies that attain quantum advantage in single-round games that are treated as black box oracles. We then demonstrate how our machinery can learn policies with quantum advantage in an illustrative multi-agent sequential decision-making problem formulated as a decentralized partially observable Markov decision process (Dec-POMDP).

cs.MACross-listedarXiv:2602.08965•Feb 10, 2026•4 citations
Cooperative multi-agent reinforcement learning is well suited to problems with large parameter spaces and exploitable local structure, such as the tuning of electrostatically-defined quantum-dot arrays. However, if parameter cross-talk is strong, a non-stationary environment from the perspective of any individual agent can destabilize learning - the same effect that plagues manual tuning of such systems. We propose using a factored representation of the action space, learned online, to decouple agents and minimize their interference. Our framework, QADAPT, uses this factorization to efficiently learn shared policies based on local measurements and rewards. With this modular strategy, we achieve zero-shot generalization to unseen quantum device sizes and maintain an approximately constant number of convergence steps to reach target regimes. This work provides a scalable route toward the rapid calibration of large-scale quantum processors.
Due to the interactions among the agents in the cooperative multi-agent systems,multi-agent learning problem complexity can rise rapidly with the number of agents or their behavioral sophistication.In order to converge to desirable equilibrium,agents generally require sufficient exploration of strategy space and coordinate their policies to achieve optimal equilibrium.A novel cooperative multi-agent learning method is proposed based on quantum theory and reinforcement learning.This method not only coordinates agents’ behaviors using quantum entanglement and helps agents make action selection under quantum superposition,but also adopts Grover’s searching algorithm which can speed up learning.This method also makes a good tradeoff between exploration and exploitation using probability characteristics of quantum theory.The results of simulated experiments show that quantum theory can be effectively used to cooperative multi-agent reinforcement learning.
As it becomes increasingly difficult to monolithically scale a quantum processor, distributed quantum computing (DQC) offers an alternative by distributing qubits across multiple smaller interconnected quantum processor modules. In such an architecture, the challenge of quantum circuit compilation shifts from placing and routing qubits within one module to placing, routing and using the qubits efficiently across modules. In order to optimize circuit execution time, the right state-dependent networking decisions must be found, such as when and where to generate shared remote quantum states to support remote operations. Reinforcement learning (RL) provides a natural framework for this problem, generating a compilation policy that can generalize across different circuits. Building on the framework of Promponas et al. (2024), we introduce an agent that combines a novel action-space formulation with effective action-masking strategies. A comprehensive numerical comparison of the two approaches under different coupling constraints shows that our agent achieves improved training and inference performance with a relative reduction in the modeled execution time of up to 35\%.
Quantum architecture search (QAS) automates the design of parameterized quantum circuits for variational quantum algorithms. The framework finds a well-suited problem-specific structure of a variational ansatz. Among possible implementations of QAS the reinforcement learning (RL) stands out as one of the most promising. Current RL approaches are single-agent-based and show poor scalability with a number of qubits due to the increase of the action space dimension and the computational cost. We propose a novel multi-agent RL algorithm for QAS with each agent acting separately on its own block of a quantum circuit. This procedure allows to significantly accelerate the convergence of the RL-based QAS and reduce its computational cost. We benchmark the proposed algorithm on MaxCut problem on 3-regular graphs and on ground energy estimation for the Schwinger Hamiltonian. In addition, the proposed multi-agent approach naturally fits into the set-up of distributed quantum computing, favoring its implementation on modern intermediate scale quantum devices.
Abstract In this paper, we propose a novel approach, Q-CMAPO (Quantum-Classical Multi-agent Policy Optimization), for tackling complex decision-making problems in multi-agent systems. By leveraging quantum-inspired optimization techniques, Q-CMAPO efficiently addresses the exploration-exploitation tradeoff, enhancing the scalability and performance of reinforcement learning (RL) algorithms in partially observable, non-stationary environments. We introduce an innovative framework that combines centralized training with decentralized execution (CTDE), enabling seamless cooperation between agents while preserving their autonomy during execution. Through extensive empirical evaluation, including various UAV deployment scenarios, we demonstrate that Q-CMAPO consistently outperforms existing baselines in both computational efficiency and classification accuracy. Our experiments show significant improvements in performance metrics, as well as substantial gains in runtime efficiency and memory utilization. Furthermore, we provide a comprehensive theoretical analysis, proving the convergence and stability of the proposed method in non-stationary environments. We also conduct an ablation study, shedding light on the importance of different components of Q-CMAPO in optimizing agent cooperation. While promising, the proposed approach faces several challenges, including its sensitivity to hyperparameters and scalability in large-scale systems, suggesting opportunities for future refinement and expansion. The integration of Q-CMAPO into real-world applications, such as autonomous robotics, and UAV-based surveillance, opens new avenues for research, bridging the gap between quantum-inspired optimization and practical deployment in multi-agent systems.
For Industry 4.0 Revolution, cooperative autonomous mobility systems are widely used based on multiagent reinforcement learning (MARL). However, the MARLbased algorithms suffer from huge parameter utilization and convergence difficulties with many agents. To tackle these problems, a quantum MARL (QMARL) algorithm based on the concept of actor-critic network is proposed, which is beneficial in terms of scalability, to deal with the limitations in the noisy intermediatescale quantum (NISQ) era. Additionally, our QMARL is also beneficial in terms of efficient parameter utilization and fast convergence due to quantum supremacy. Note that the reward in our QMARL is defined as task precision over computation time in multiple agents, thus, multi-agent cooperation can be realized. For further improvement, an additional technique for scalability is proposed, which is called projection value measure (PVM). Based on PVM, our proposed QMARL can achieve the highest reward, by reducing the action dimension into a logarithmicscale. Finally, we can conclude that our proposed QMARL with PVM outperforms the other algorithms in terms of efficient parameter utilization, fast convergence, and scalability.
The inability to communicate poses a major challenge to coordination in multi-agent reinforcement learning (MARL). Prior work has explored correlating local policies via shared randomness, sometimes in the form of a correlation device, as a mechanism to assist in decentralized decision-making. In contrast, this work introduces the first framework for training MARL agents to exploit shared quantum entanglement as a coordination resource, which permits a larger class of communication-free correlated policies than shared randomness alone. This is motivated by well-known results in quantum physics which posit that, for certain single-round cooperative games with no communication, shared quantum entanglement enables strategies that outperform those that only use shared randomness. In such cases, we say that there is quantum advantage. Our framework is based on a novel differentiable policy parameterization that enables optimization over quantum measurements, together with a novel policy architecture that decomposes joint policies into a quantum coordinator and decentralized local actors. To illustrate the effectiveness of our proposed method, we first show that we can learn, purely from experience, strategies that attain quantum advantage in single-round games that are treated as black box oracles. We then demonstrate how our machinery can learn policies with quantum advantage in an illustrative multi-agent sequential decision-making problem formulated as a decentralized partially observable Markov decision process (Dec-POMDP).