Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2407.09877 · Quantum Science and Technology

Model-free distortion canceling and control of quantum devices

Ahmed Fouad, Akram Youssry, Ahmed El-Rafei, Sherif Hammad

Published Sep 27, 2024• 5 citations
arXivHTMLPDF

Abstract

Abstract Quantum devices need precise control to achieve their full capability. In this work, we address the problem of controlling closed quantum systems, tackling two main issues. First, in practice the control signals are usually subject to unknown classical distortions that could arise from the device fabrication, material properties and/or instruments generating those signals. Second, in most cases modeling the system is very difficult or not even viable due to uncertainties in the relations between some variables and inaccessibility to some measurements inside the system. In this paper, we introduce a general model-free control approach based on deep reinforcement learning (DRL), that can work for any controllable closed quantum system. We train a deep neural network (NN), using the REINFORCE policy gradient algorithm to control the state probability distribution of a controllable closed quantum system as it evolves, and drive it to different target distributions. We present a novel controller architecture that comprises multiple NNs. This enables accommodating as many different target state distributions as desired, without increasing the complexity of the NN or its training process. The used DRL algorithm works whether the control problem can be modeled as a Markov decision process (MDP) or a partially observed MDP. Our method is valid whether the control signals are discrete- or continuous-valued. We verified our method through numerical simulations based on a photonic waveguide array chip. We trained a controller to generate sequences of different target output distributions of the chip with fidelity higher than 99%, where the controller showed superior performance in canceling the classical signal distortions.

DOI
10.1088/2058-9565/ad80c1

Related papers

Model-Free Deep Recurrent Q-Network Reinforcement Learning for Quantum Circuit Architectures Design

Tomah Sogabe, Tomoaki Kimura, Chih-Chieh Chen +4 more

Artificial intelligence (AI) technology leads to new insights into the manipulation of quantum systems in the Noisy Intermediate-Scale Quantum (NISQ) era. Classical agent-based artificial intelligence algorithms provide a framework for the design or control of quantum systems. Traditional reinforcement learning methods are designed for the Markov Decision Process (MDP) and, hence, have difficulty in dealing with partially observable or quantum observable decision processes. Due to the difficulty of building or inferring a model of a specified quantum system, a model-free-based control approach is more practical and feasible than its counterpart of a model-based approach. In this work, we apply a model-free deep recurrent Q-network (DRQN) reinforcement learning method for qubit-based quantum circuit architecture design problems. This paper is the first attempt to solve the quantum circuit design problem from the recurrent reinforcement learning algorithm, while using discrete policy. Simulation results suggest that our long short-term memory (LSTM)-based DRQN method is able to learn quantum circuits for entangled Bell–Greenberger–Horne–Zeilinger (Bell–GHZ) states. However, since we also observe unstable learning curves in experiments, suggesting that the DRQN could be a promising method for AI-based quantum circuit design application, more investigation on the stability issue would be required.

Quantum Reports•Sep 21, 2022•13 citations

Control of Qubit Dynamics Using Reinforcement Learning

Dimitris Koutromanos, Dionisis Stefanatos, Emmanuel Paspalakis

The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.

Information•May 11, 2024•13 citations

Deep Reinforcement Learning for Control Design of Quantum Gates

Shouliang Hu, Chunlin Chen, Daoyi Dong

This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.

Asian Control Conference (ASCC)•May 4, 2022•4 citations

Quantum reinforcement learning in continuous action space

Shaojun Wu, Shan Jin, Dingding Wen +2 more

Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.

arXiv:2012.10711•Quantum•Mar 12, 2025•83 citations

Modeling and control of a reconfigurable photonic circuit using deep learning

Akram Youssry, Robert J. Chapman, Alberto Peruzzo +2 more

Abstract The complexity of experimental quantum information processing devices is increasing rapidly, requiring new approaches to control them. In this paper, we address the problems of practically modeling and controlling an integrated optical waveguide array chip—a technology expected to have many applications in telecommunications and optical quantum information processing. This photonic circuit can be electrically reconfigured, but only the output optical signal can be monitored. As a result, the conventional control methods cannot be naively applied. Characterizing such a chip is challenging for three reasons. First, there are uncertainties associated with the Hamiltonian model describing the chip. Second, we expect distortions of the control voltages caused by the chip’s electrical response, which cannot be directly observed. And third, there are imperfections in the measurements caused by losses from coupling the chip externally to optical fibers. We have developed a deep neural network (NN) approach to solve these problems. The architecture is designed specifically to overcome the aforementioned challenges using a gated recurrent unit -based network as the central component. The Hamiltonian is estimated as a blackbox, while the rules of quantum mechanics such as state evolution is embedded in the structure as a whitebox. The resulting overall graybox model of the chip shows good performance both quantitatively in terms of the mean square error and qualitatively in terms of the shape of the predicted waveforms We use this NN to solve a classical and a quantum control problem. In the classical application we find a control sequence to approximately realize a time-dependent output power distribution. For the quantum application we obtain the control voltages to realize a target set of quantum gates. The method we propose is generic and can be applied to other systems that can only be probed indirectly.

arXiv:1907.08023•Quantum Science and Technology•Dec 11, 2019•29 citations

Model-Free Quantum Gate Design and Calibration Using Deep Reinforcement Learning

Omar Shindi, Qi Yu, Parth Girdhar +1 more

High-fidelity quantum gate design is important for various quantum technologies, such as quantum computation and quantum communication. Numerous control policies for quantum gate design have been proposed given a dynamical model of the quantum system of interest. However, a quantum system is often highly sensitive to noise, and obtaining its accurate modeling can be difficult for many practical applications. Thus, the control policy based on a quantum system model may be unpractical for quantum gate design. Also, quantum measurements collapse quantum states, which makes it challenging to obtain information through measurements during the control process. In this article, we propose a novel training framework using deep reinforcement learning for model-free quantum control. The proposed framework relies only on the measurement at the end of the control process and offers the ability to find the optimal control policy without access to quantum systems during the learning process. The effectiveness of the proposed technique is numerically demonstrated for model-free quantum gate design and quantum gate calibration using off-policy reinforcement learning algorithms.

arXiv:2302.02371•IEEE Transactions on Artificial Intelligence•Feb 8, 2023•21 citations
Abstract Quantum devices need precise control to achieve their full capability. In this work, we address the problem of controlling closed quantum systems, tackling two main issues. First, in practice the control signals are usually subject to unknown classical distortions that could arise from the device fabrication, material properties and/or instruments generating those signals. Second, in most cases modeling the system is very difficult or not even viable due to uncertainties in the relations between some variables and inaccessibility to some measurements inside the system. In this paper, we introduce a general model-free control approach based on deep reinforcement learning (DRL), that can work for any controllable closed quantum system. We train a deep neural network (NN), using the REINFORCE policy gradient algorithm to control the state probability distribution of a controllable closed quantum system as it evolves, and drive it to different target distributions. We present a novel controller architecture that comprises multiple NNs. This enables accommodating as many different target state distributions as desired, without increasing the complexity of the NN or its training process. The used DRL algorithm works whether the control problem can be modeled as a Markov decision process (MDP) or a partially observed MDP. Our method is valid whether the control signals are discrete- or continuous-valued. We verified our method through numerical simulations based on a photonic waveguide array chip. We trained a controller to generate sequences of different target output distributions of the chip with fidelity higher than 99%, where the controller showed superior performance in canceling the classical signal distortions.
Artificial intelligence (AI) technology leads to new insights into the manipulation of quantum systems in the Noisy Intermediate-Scale Quantum (NISQ) era. Classical agent-based artificial intelligence algorithms provide a framework for the design or control of quantum systems. Traditional reinforcement learning methods are designed for the Markov Decision Process (MDP) and, hence, have difficulty in dealing with partially observable or quantum observable decision processes. Due to the difficulty of building or inferring a model of a specified quantum system, a model-free-based control approach is more practical and feasible than its counterpart of a model-based approach. In this work, we apply a model-free deep recurrent Q-network (DRQN) reinforcement learning method for qubit-based quantum circuit architecture design problems. This paper is the first attempt to solve the quantum circuit design problem from the recurrent reinforcement learning algorithm, while using discrete policy. Simulation results suggest that our long short-term memory (LSTM)-based DRQN method is able to learn quantum circuits for entangled Bell–Greenberger–Horne–Zeilinger (Bell–GHZ) states. However, since we also observe unstable learning curves in experiments, suggesting that the DRQN could be a promising method for AI-based quantum circuit design application, more investigation on the stability issue would be required.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.
This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.
Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.
Abstract The complexity of experimental quantum information processing devices is increasing rapidly, requiring new approaches to control them. In this paper, we address the problems of practically modeling and controlling an integrated optical waveguide array chip—a technology expected to have many applications in telecommunications and optical quantum information processing. This photonic circuit can be electrically reconfigured, but only the output optical signal can be monitored. As a result, the conventional control methods cannot be naively applied. Characterizing such a chip is challenging for three reasons. First, there are uncertainties associated with the Hamiltonian model describing the chip. Second, we expect distortions of the control voltages caused by the chip’s electrical response, which cannot be directly observed. And third, there are imperfections in the measurements caused by losses from coupling the chip externally to optical fibers. We have developed a deep neural network (NN) approach to solve these problems. The architecture is designed specifically to overcome the aforementioned challenges using a gated recurrent unit -based network as the central component. The Hamiltonian is estimated as a blackbox, while the rules of quantum mechanics such as state evolution is embedded in the structure as a whitebox. The resulting overall graybox model of the chip shows good performance both quantitatively in terms of the mean square error and qualitatively in terms of the shape of the predicted waveforms We use this NN to solve a classical and a quantum control problem. In the classical application we find a control sequence to approximately realize a time-dependent output power distribution. For the quantum application we obtain the control voltages to realize a target set of quantum gates. The method we propose is generic and can be applied to other systems that can only be probed indirectly.
High-fidelity quantum gate design is important for various quantum technologies, such as quantum computation and quantum communication. Numerous control policies for quantum gate design have been proposed given a dynamical model of the quantum system of interest. However, a quantum system is often highly sensitive to noise, and obtaining its accurate modeling can be difficult for many practical applications. Thus, the control policy based on a quantum system model may be unpractical for quantum gate design. Also, quantum measurements collapse quantum states, which makes it challenging to obtain information through measurements during the control process. In this article, we propose a novel training framework using deep reinforcement learning for model-free quantum control. The proposed framework relies only on the measurement at the end of the control process and offers the ability to find the optimal control policy without access to quantum systems during the learning process. The effectiveness of the proposed technique is numerically demonstrated for model-free quantum gate design and quantum gate calibration using off-policy reinforcement learning algorithms.