Quantum machine learning (QML) as combination of quantum computing with machine learning (ML) is a promising direction to explore, in particular due to the advances in realizing quantum computers and the hoped-for quantum advantage. A field within QML that is only little approached is quantum multi-agent reinforcement learning (QMARL), despite having shown to be potentially attractive for addressing industrial applications such as factory management, cellular access and mobility cooperation. This paper presents an aerial communication use case and introduces a hybrid quantum-classical (HQC) ML algorithm to solve it. This use case intends to increase the connectivity of flying ad-hoc networks and is solved by an HQC multi-agent proximal policy optimization algorithm in which the core of the centralized critic is replaced with a data reuploading variational quantum circuit. Results show a slight increase in performance for the quantum-enhanced solution with respect to a comparable classical algorithm, earlier reaching convergence, as well as the scalability of such a solution: an increase in the size of the ansatz, and thus also in the number of trainable parameters, leading to better outcomes. These promising results show the potential of QMARL to industrially-relevant complex use cases.
Quantum machine learning (QML) as combination of quantum computing with machine learning (ML) is a promising direction to explore, in particular due to the advances in realizing quantum computers and the hoped-for quantum advantage. A field within QML that is only little approached is quantum multi-agent reinforcement learning (QMARL), despite having shown to be potentially attractive for addressing industrial applications such as factory management, cellular access and mobility cooperation. This paper presents an aerial communication use case and introduces a hybrid quantum-classical (HQC) ML algorithm to solve it. This use case intends to increase the connectivity of flying ad-hoc networks and is solved by an HQC multi-agent proximal policy optimization algorithm in which the core of the centralized critic is replaced with a data reuploading variational quantum circuit. Results show a slight increase in performance for the quantum-enhanced solution with respect to a comparable classical algorithm, earlier reaching convergence, as well as the scalability of such a solution: an increase in the size of the ansatz, and thus also in the number of trainable parameters, leading to better outcomes. These promising results show the potential of QMARL to industrially-relevant complex use cases.
Quantum machine learning (QML) has received a lot of attention according to its light training parameter numbers and speeds; and the advances of QML lead to active research on quantum multi-agent reinforcement learning (QMARL). Existing classical multi-agent reinforcement learning (MARL) features non-stationarity and uncertain properties. Therefore, this paper presents a simulation software framework for novel QMARL to control autonomous multi-drone mobility, i.e., quantum multi-drone reinforcement learning. Our proposed framework ac-complishes reasonable reward convergence and service quality performance with fewer trainable parameters. Furthermore, it shows more stable training results. Lastly, our proposed software allows us to analyze the training process and results.
This paper addresses a new quantum computing-based multi-agent reinforcement learning (QMARL) algorithm which is inspired by the quantum neural network (QNN)-based centralized critic and multiple actor networks. The benefit of the proposed QMARL-based algorithm is in the action control dimension reduction where it can reduce the size into a logarithmic-scale when project value measure (PVM) is utilized. Therefore, our proposed QMARL-based algorithm is beneficial for massive-agent MARL training convergence. Moreover, the various applications of QMARL-based algorithms are presented in massive-scale unmanned aerial vehicle (UAV) networks. Lastly, our performance evaluation results verify that the proposed QMARL-based algorithm can successfully converge when massive-action dimensions should be utilized.
Autonomous agents are being used in more complex and dynamic environments where efficient learning, adaptive decision-making, and scalable optimization are required. Although classical machine learning (ML) and reinforcement methods have greatly contributed to agent intelligence, they are usually restricted by constraints of computational complexity, scaling, and convergence to high-dimensional decision space. The current developments in quantum computing have created new possibilities to solve these problems using quantum ML (QML). In this chapter, the authors discuss the implementation of QML in collaboration with autonomous agents in the larger paradigm of agentic artificial intelligence (AI) and quantum AI agents. It offers a hybrid classical–quantum architectural model where quantum learning modules improve perception, policy learning, and decision optimization, and do not conflict with classical agent pipelines. The chapter analyzes application cases in autonomous robotics, intelligent transportation systems, multi-agent coordination, smart energy management, as well as communication networks, and the chapter shows the potential benefits of quantum-enhanced learning in these areas. The issue of quantum hardware neediness, data encoding, scaling, hybrid integration, and real-life application are critically evaluated and unanswered. In summary, the chapter is a well-organized approach to the purpose of QML in the development of the next-generation autonomous agent and an insight into creating practical, scalable, and intelligent quantum AI agents.
This paper presents a comprehensive survey of Quantum Multi-Agent Reinforcement Learning (QMARL), a nascent field at the intersection of quantum computing and multi-agent systems. The survey begins by introducing the fundamentals of quantum computing, highlighting its potential to revolutionize computational capabilities. We then delve into the principles of multi-agent reinforcement learning (MARL), examining how quantum computing can enhance learning efficiency and decision-making processes in complex environments. The core of the survey focuses on the current state of QMARL, reviewing existing literature, methodologies, and case studies that demonstrate the integration of quantum algorithms with MARL frameworks. The paper also addresses the unique challenges and opportunities presented by quantum technologies in multi-agent systems, such as quantum entanglement and superposition, and their implications for agent coordination and learning dynamics. Additionally, the survey explores the practical applications of QMARL in various domains, including cybersecurity, finance, and robotics, underscoring its transformative potential. The paper concludes by identifying key research gaps and proposing future directions for the development of QMARL. This includes the need for scalable quantum algorithms, the exploration of quantum-resistant strategies in adversarial settings, and the integration of quantum principles in agent communication and collaboration. Overall, this survey serves as a foundational guide for researchers and practitioners interested in the emerging field of QMARL, offering insights into its current achievements and future possibilities.
This paper presents a comprehensive survey of Quantum Multi-Agent Reinforcement Learning (QMARL), a nascent field at the intersection of quantum computing and multi-agent systems. The survey begins by introducing the fundamentals of quantum computing, highlighting its potential to revolutionize computational capabilities. We then delve into the principles of multi-agent reinforcement learning (MARL), examining how quantum computing can enhance learning efficiency and decision-making processes in complex environments. The core of the survey focuses on the current state of QMARL, reviewing existing literature, methodologies, and case studies that demonstrate the integration of quantum algorithms with MARL frameworks. The paper also addresses the unique challenges and opportunities presented by quantum technologies in multi-agent systems, such as quantum entanglement and superposition, and their implications for agent coordination and learning dynamics. Additionally, the survey explores the practical applications of QMARL in various domains, including cybersecurity, finance, and robotics, underscoring its transformative potential. The paper concludes by identifying key research gaps and proposing future directions for the development of QMARL. This includes the need for scalable quantum algorithms, the exploration of quantum-resistant strategies in adversarial settings, and the integration of quantum principles in agent communication and collaboration. Overall, this survey serves as a foundational guide for researchers and practitioners interested in the emerging field of QMARL, offering insights into its current achievements and future possibilities.
Quantum machine learning (QML) has received a lot of attention according to its light training parameter numbers and speeds; and the advances of QML lead to active research on quantum multi-agent reinforcement learning (QMARL). Existing classical multi-agent reinforcement learning (MARL) features non-stationarity and uncertain properties. Therefore, this paper presents a simulation software framework for novel QMARL to control autonomous multi-drones, i.e., quantum multi-drone reinforcement learning. Our proposed framework accomplishes reasonable reward convergence and service quality performance with fewer trainable parameters. Furthermore, it shows more stable training results. Lastly, our proposed software allows us to analyze the training process and results.