Finding a path with the least overall noise from quantum memories, fibers, and gate operations in a quantum network involves the challenge of acquiring knowledge of these noises, their sources, and the time of occurrences. In this paper, we propose a reinforcement learning-based route selection approach that uses a multi-arm bandit algorithm to find the least noisy path from a transmitter (Tx) to a receiver (Rx), without considering any information on qubit decoherence due to probabilistic noises inherent in quantum memories and imperfect gate operations. It only uses network deployment knowledge to find a set of feasible paths from Tx to Rx. We provide a key finding from a network design perspective which says that performing entanglement swapping on nodes within a path in a non-synchronized and parallel manner not always reduces the decoherence experienced in achieving the end-to-end entanglement on that path. Further, we design and open-source a new simulator for simulating probabilistic noises encountered during entanglement distribution between Tx-Rx on a path, which has supporting callable functions for connecting the unknown network environment required for interaction with the multi-arm bandit agent. The simulation results demonstrate that our proposed route selection approach provides a path up to ~ 33% better fidelity (less noise) compared to conventional, distance-based route selection approach for the considered quantum network.
Finding a path with the least overall noise from quantum memories, fibers, and gate operations in a quantum network involves the challenge of acquiring knowledge of these noises, their sources, and the time of occurrences. In this paper, we propose a reinforcement learning-based route selection approach that uses a multi-arm bandit algorithm to find the least noisy path from a transmitter (Tx) to a receiver (Rx), without considering any information on qubit decoherence due to probabilistic noises inherent in quantum memories and imperfect gate operations. It only uses network deployment knowledge to find a set of feasible paths from Tx to Rx. We provide a key finding from a network design perspective which says that performing entanglement swapping on nodes within a path in a non-synchronized and parallel manner not always reduces the decoherence experienced in achieving the end-to-end entanglement on that path. Further, we design and open-source a new simulator for simulating probabilistic noises encountered during entanglement distribution between Tx-Rx on a path, which has supporting callable functions for connecting the unknown network environment required for interaction with the multi-arm bandit agent. The simulation results demonstrate that our proposed route selection approach provides a path up to ~ 33% better fidelity (less noise) compared to conventional, distance-based route selection approach for the considered quantum network.
Quantum Data Networks (QDNs) have emerged as a promising framework in the field of information processing and transmission, harnessing the principles of quantum mechanics. QDNs utilize a quantum teleportation technique through long-distance entanglement connections, encoding data information in quantum bits (qubits). Despite being a cornerstone in various quantum applications, quantum entanglement encounters challenges in establishing connections over extended distances due to probabilistic processes influenced by factors like optical fiber losses. The creation of long-distance entanglement connections between quantum computers involves multiple entanglement links and entanglement swapping techniques through successive quantum nodes, including quantum computers and quantum repeaters, necessitating optimal path selection and qubit allocation. Current research predominantly assumes known success rates of entanglement links between neighboring quantum nodes and overlooks potential network attackers. This paper addresses the online challenge of optimal path selection and qubit allocation, aiming to learn the best strategy for achieving the highest success rate of entanglement connections between two chosen quantum computers without prior knowledge of the success rate and in the presence of a QDN attacker. The proposed approach is based on multi-armed bandits, specifically adversarial group neural bandits, which treat each path as a group and view qubit allocation as arm selection. Our contributions encompass formulating an online adversarial optimization problem, introducing the EXPNeuralUCB bandits algorithm with theoretical performance guarantees, and conducting comprehensive simulations to showcase its superiority over established advanced algorithms.
Entanglement generation in long-distance quantum networks is challenging because resources are limited and entanglement swapping is probabilistic. To maximize the rate of successful requests, existing quantum routing algorithms often rely on computationally expensive methods such as Integer Linear Programming (ILP) to determine which links to entangle and use for end-to-end entanglement generation. However, these approaches fail to meet the latency requirements of real-world quantum networks. In this study, we propose a Reinforcement Learning (RL)-based model that determines which links to entangle in each time slot, replacing the slow ILP-based link-selection phase used in prior algorithms. The proposed Deep Q-learning model is up to 19.2× faster than linear programming in link-selection while maintaining comparable routing performance. The RL link-selection model alone matches ILP in request success rate; combining RL link selection with entanglement caching and proactive swapping raises throughput by up to 52.55% over ILP. Overall, our approach achieves success rates exceeding state-of-the-art solutions while reducing execution time by more than an order of magnitude. Experiments on a synthetic 50-node Waxman topology and the real-world 54-node SURFnet core topology confirm that these gains generalize across network structures, providing a practical path toward scalable quantum routing.
Optimal routing is a fundamental challenge in quantum networking, with several approaches proposed to identify the most efficient path for end-to-end (e2e) entanglement generation between pairs of nodes. In this paper, we show that prior entanglements - entanglements generated in a previous network cycle but not yet utilized - are an important consideration in optimal path selection due to the dynamic nature of quantum networks. Specifically, we investigate whether a longer path with pre-existing entanglements can outperform a shorter path that starts from scratch. We account for key quantum constraints, including noisy entanglement generation and swapping, fidelity decay, probabilistic operations, and link discarding upon swap failure. Simulations reveal that longer paths with prior entanglements can establish e2e entanglement faster than shorter paths under certain conditions. We further introduce the notion of entanglement diversity, where multiple paths can be used to improve performance - either by selecting the first successful path to minimize time or using both paths to enhance fidelity through distillation. These findings highlight the importance of incorporating prior entanglements into path selection strategies for optimizing quantum communication networks.
ABSTRACT Quantum communications and quantum key distribution (QKD) occupy a unique position at the intersection of quantum information processing, stochastic noise and distributed network control. Theoretical guarantees for the security of QKD based on quantum mechanics have been developed over the last several years. In order to effectively implement QKD, however, we must understand that practical implementations are limited by factors such as decoherence, imperfection of devices, and non‐homogeneous, varying environmental conditions of our distributed networks. These challenges require the need to develop adaptive computational frameworks that will dynamically adjust to changing levels of noise and to resource availability in order to successfully operate QKD, rather than relying on fixed configurations of protocols. The current study presents a means of adaptively selecting circuits for quantum communication simulated in a noisy environment using a high level of reproducibility in terms of computational frameworks and focuses upon workloads similar to QKD. Quantum communication can be treated as a decentralized, stochastic decision‐making process in which an adapting controller selects between a set of available quantum circuits based on observable network characteristics. Using the decoherence‐free subspace (DFS) encoding method and an entanglement‐based protocol that incorporates purification, the use of modular circuit templates has allowed for the selection of templates guided by the use of learned decision policies as opposed to a fixed protocol being predetermined. Unlike previous works applying reinforcement learning to tune fixed protocol parameters or measurement strategies Yang et al. and Xu et al., we model decoherence‐mitigation primitives—DFS and entanglement purification—as dynamically selectable circuit templates . In this sense, the agent acts on protocol structure instead of tuning parameters, to allow compositional adapting, on heterogeneous and adversarial network conditions. (This amounts to a qualitatively different and under‐explored control abstraction compared to the literature). Adaptive control utilizes a quantum reinforcement learning (QRL) based six qubit, four layer parameterized quantum circuit with 48 trainable parameters and a deep Q‐network (DQN) classical baseline. The agents notice the same three‐dimensional representation of state with channel loss, node‐specific noise and eavesdropping indicators. The framework is compared with more than 50 independent trials on a simulated 10‐node quantum network under a physically enriched noise model with depolarizing noise, collective dephasing, thermal relaxation (, ), amplitude‐damping leakage, nearest‐neighbor crosstalk–yielding non‐trivial von Neumann entropy values to be up to bits that give an in realistic physical depiction of decoherence. The reward function is explicitly decomposed into weighted components that are aligned to security‐critical (QBER, key rate, fidelity, entropy) metrics that are essential security, ensuring sensitivity to the most operationally relevant performance metrics. Numerical tests indicated that with noise models and circuit expressivity, QRL and classical DQN agents consistently perform in the same manner over reported metrics, and do not show any statistically significant difference between mean QBER ( p = 0.2037). Crucially, this result is not so much negative as it suggests a regime in which quantum policy models do not do better than classical baselines, yet, the QRL policy circuit operates within a genuinely quantum hypothesis space with nonzero entanglement entropy and active gradient‐based learning. It indicates that it is circuit expressivity, incentive strategy design and environment complexity which define quantum advantage of quantum communications activities (not only quantum resources by themselves). In this work we focus on the development of a transparent and extensible simulation pipeline in order to combine quantum circuit performance, stochastic noise modeling and learning‐informed adaptive control all within the framework of a one computing platform. It's about reproducibility, controlled benchmarking, and modular design, this investigation gives a concrete proving ground for further study in adaptive workflows, concurrency‐aware decision making and hybrid quantum‐classical control strategies employed in emerging quantum communication networks.
We consider the problem of entanglement routing in heterogeneous quantum networks where each link and node may offer a different entanglement generation quality. We consider that each link offers a base rate and fidelity of entanglement generation. These base values can be altered by performing entanglement purification on each link. Furthermore, we consider that each repeater node offers a different probability of a successful entanglement swapping. With this setting, we design a reinforcement learning-based approach for flexible entanglement routing on future quantum networks. Specifically, we formulate the problem of entanglement routing in the Q-learning framework where the main objective of the agent is to deliver end-to-end entanglement while maximizing a user-specified objective function in the form of a weighted sum of achieve fidelity and rate. We demonstrate the efficacy and flexibility of developed framework by simulating random quantum networks where the random requests in fidelity maximizing, rate maximizing, or balanced mode are generated. We also numerically assess the effects of infrastructure developments and technology enhancements in future quantum networks.
Entanglement generation and swapping is a difficult process due to probabilistic nature of quantum mechanics. To overcome this issue, existing quantum routing algorithms try to create entanglement on multiple paths between source and destination. Although it is possible to save entangled qubits on unused links using quantum memories, the quantum routing algorithms discard them and try creating new entanglement in each time slot. In this work, we leverage the longevity of entanglement and introduce two enhancements to improve the performance of existing routing algorithms: (i) The generation and caching of entanglements across multiple time slots, and (ii) the proactively executing entanglement swapping for frequently utilized links, increasing the success rate of future requests. Since entanglement caching reduces available quantum memory on quantum repeaters, we apply reinforcement learning to identify the optimal set of segments that are likely to be used in the future. Through comprehensive simulations, we demonstrate that caching unused entanglements lead to 10 - 15% improvement on the performance of quantum routing algorithms REPS and SEER. Complementing the caching with reinforcement learning based proactively entanglement swapping leads to further performance improvements. Specifically, using Q-Learning to decide on which segments to proactively swap leads to 52.55% and 41.79% improvement for REPS and SEER algorithms, respectively. Deep Q-Learning outperforms Q-Learning with 61.43% and 48.27% improvements rates.