Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.
AI Summary
The paper studies scheduling simultaneous quantum-network jobs, each specified by a required entanglement graph and execution duration. A scheduler must allocate physical entangled links, generate virtual links by swapping, and place jobs while respecting memory lifetimes and, optionally, hardware-type constraints on node placement.
The problem is formulated as a Markov decision process and addressed using double deep Q-learning with a message-passing neural network, replay buffers, and curriculum training over decreasing link-generation probabilities. Rewards incorporate progress toward feasible job placements, bottleneck centrality, job complexity, and elapsed time. Evaluation uses three nine-node topologies, five parallel sublinks per physical connection, a 52-step memory lifetime, and two duration-one $K_4$ jobs, with a 200-step episode limit. Swapping is assumed deterministic.
Across 100 episodes at each tested probability, learned policies retain an observed 100% success rate at link-activation probabilities up to 71% below those of the greedy baselines. On the uncolored dumbbell at $p_l\approx0.018$, completion takes approximately 50 steps versus 125–150 for the baselines. With hardware constraints, policies maintain at least 80% success at probabilities up to 59% lower. These are simulation comparisons, not analytical guarantees.
Behavioral metrics indicate more selective link generation and longer waiting than the baselines. An LLM-generated heuristic derived from policy traces achieves similar performance on the uncolored starlink and is tested on larger instances with three fixed hubs. The paper reports that later, noisier curriculum phases can degrade low-noise performance; results therefore use selected earlier checkpoints. The colored dumbbell shows a smaller advantage over the strongest baseline.
Little sign of AI-generated textAI use disclosedSmall numerics onlyMissing comparisonStrong assumptionsOverclaims
We additionally conducted experiments with Gemini 3.1 Pro to obtain the results in Section \ref{llm_policy_distillation}.
Error found: The subgraph-edit-distance reward incorrectly defines progress by adding edges to the experiment graph, an operation that cannot repair a failed embedding into the active network and therefore leaves the stated reward undefined in relevant states.
Written by gpt-6.1-sol from the paper's full text. Automated: it can be wrong.
Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.
The paper studies scheduling simultaneous quantum-network jobs, each specified by a required entanglement graph and execution duration. A scheduler must allocate physical entangled links, generate virtual links by swapping, and place jobs while respecting memory lifetimes and, optionally, hardware-type constraints on node placement.
The problem is formulated as a Markov decision process and addressed using double deep Q-learning with a message-passing neural network, replay buffers, and curriculum training over decreasing link-generation probabilities. Rewards incorporate progress toward feasible job placements, bottleneck centrality, job complexity, and elapsed time. Evaluation uses three nine-node topologies, five parallel sublinks per physical connection, a 52-step memory lifetime, and two duration-one K4 jobs, with a 200-step episode limit. Swapping is assumed deterministic.
Across 100 episodes at each tested probability, learned policies retain an observed 100% success rate at link-activation probabilities up to 71% below those of the greedy baselines. On the uncolored dumbbell at pl≈0.018, completion takes approximately 50 steps versus 125–150 for the baselines. With hardware constraints, policies maintain at least 80% success at probabilities up to 59% lower. These are simulation comparisons, not analytical guarantees.
Behavioral metrics indicate more selective link generation and longer waiting than the baselines. An LLM-generated heuristic derived from policy traces achieves similar performance on the uncolored starlink and is tested on larger instances with three fixed hubs. The paper reports that later, noisier curriculum phases can degrade low-noise performance; results therefore use selected earlier checkpoints. The colored dumbbell shows a smaller advantage over the strongest baseline.
Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link ac Author: Andrew Stewart Caldin, Independent Researcher, UK. Part of the E8 Intelligence Research series. Platform: e8intelligence.com
Entanglement generation in long-distance quantum networks is challenging because resources are limited and entanglement swapping is probabilistic. To maximize the rate of successful requests, existing quantum routing algorithms often rely on computationally expensive methods such as Integer Linear Programming (ILP) to determine which links to entangle and use for end-to-end entanglement generation. However, these approaches fail to meet the latency requirements of real-world quantum networks. In this study, we propose a Reinforcement Learning (RL)-based model that determines which links to entangle in each time slot, replacing the slow ILP-based link-selection phase used in prior algorithms. The proposed Deep Q-learning model is up to 19.2× faster than linear programming in link-selection while maintaining comparable routing performance. The RL link-selection model alone matches ILP in request success rate; combining RL link selection with entanglement caching and proactive swapping raises throughput by up to 52.55% over ILP. Overall, our approach achieves success rates exceeding state-of-the-art solutions while reducing execution time by more than an order of magnitude. Experiments on a synthetic 50-node Waxman topology and the real-world 54-node SURFnet core topology confirm that these gains generalize across network structures, providing a practical path toward scalable quantum routing.
Protocols for distributed quantum systems commonly require the simultaneous availability of n entangled states, each with a fidelity above some fixed minimum Fapp relative to the target maximally-entangled state. However, the fidelity of entangled states degrades over time while in memory. Entangled states are therefore rendered useless when their fidelity falls below Fapp. This is problematic when entanglement generation is probabilistic and attempted in a sequential manner, because the expected completion time until n entangled states are available can be large. Motivated by existing entanglement generation schemes, we consider a system where the entanglement generation parameters (the success probability p and fidelity F of the generated entangled state) may be adjusted at each time step. We model the system as a Markov decision process, where the policy dictates which generation parameters (p,F) to use for each attempt. We use dynamic programming to derive optimal policies that minimise the expected time until n entangled states are available with fidelity greater than Fapp. We observe that the advantage of our optimal policies over the selected baselines increases significantly with n. In the parameter regimes explored, which are based closely on current experiments, we find that the optimal policy can provide a speed-up of as much as a factor of twenty over a constant-action policy. In addition, we propose a computationally inexpensive heuristic method to compute policies that perform either optimally or near-optimally in the parameter regimes explored. Our heuristic method can be used to find high-performing policies in parameter regimes where finding an optimal policy is intractable.
This paper investigates resource allocation for entanglement distribution in multi-node, multi-channel quantum networks at the metropolitan scale. A multi-slot quantum network simulation framework is developed across physical and network layers, incorporating heterogeneous link characteristics, limited quantum memories, request queuing, retry mechanisms, and concurrent entanglement distribution. Based on this framework, a centralized scheduling architecture is proposed to coordinate quantum memories, communication channels, and routing paths for multiple simultaneous entanglement requests. Three classes of resource allocation strategies are evaluated: heuristic approaches, a mixed-integer linear programming (MILP) optimization method, and a Proximal Policy Optimization (PPO)-based reinforcement learning approach. The heuristic schemes reveal fundamental trade-offs between request delay and entanglement success rate: Dynamic Efficient minimizes delay, Success Enhancement improves success probability through adaptive multi-path allocation, and Static Efficient provides a balance between these objectives. The MILP approach achieves optimized resource allocation by jointly considering request handling and multi-path assignment, while the PPO-based method learns adaptive scheduling policies to improve overall performance. Simulation results demonstrate that these approaches provide different performance advantages in terms of request delay, entanglement success rate, capacity utilization, and request handling rate, highlighting the trade-offs between efficiency and reliability in metropolitan-scale quantum network scheduling.
Efficient routing in entanglement distribution networks is a challenging task that requires the coordinated allocation of quantum memories, repeaters, and quantum operations such as entanglement swapping and distillation across dynamic network conditions. In this paper, we present a Deep Q-Learning (DQN) Two-Stage Routing strategy designed to improve both path and quantum operations selection. Our approach enables the agent to make decisions over a diverse action space, including request scheduling, multipath routing, and the strategic application of quantum operations to maximize entanglement success and resource efficiency. We further extend DQN to integrate Double DQN, Dueling DQN, and Prioritized Experience Replay (PER) architectures. Simulation results indicate that our approach significantly outperforms baseline methods regarding request success rate, especially in large-scale networks under strict fidelity requirements.
Abstract Quantum repeaters play a crucial role in the effective distribution of entanglement over long distances. The nearest-future type of quantum repeater requires two operations: entanglement generation across neighbouring repeaters and entanglement swapping to promote short-range entanglement to long-range. For many hardware setups, these actions are probabilistic, leading to longer distribution times and incurred errors. Significant efforts have been vested in finding the optimal entanglement-distribution policy, i.e. the protocol specifying when a network node needs to generate or swap entanglement, such that the expected time to distribute long-distance entanglement is minimal. This problem is even more intricate in more realistic scenarios, especially when classical communication delays are taken into account. In this work, we formulate our problem as a Markov decision problem and use reinforcement learning (RL) to optimise over centralised strategies, where one designated node instructs other nodes which actions to perform. Contrary to most RL models, ours can be readily interpreted. Additionally, we introduce and evaluate a fixed local policy, the ‘predictive swap-asap’ policy, where nodes only coordinate with nearest neighbours. Compared to the straightforward generalisation of the common swap-asap policy to the scenario with classical communication effects, the ‘wait-for-broadcast swap-asap’ policy, both of the aforementioned entanglement-delivery policies are faster at high success probabilities. Our work showcases the merit of considering policies acting with incomplete information in the realistic case when classical communication effects are significant.