Accurate control of quantum states is crucial for quantum computing and other quantum technologies. In the basic scenario, the task is to steer a quantum system towards a target state through a sequence of control operations. Determining the appropriate operations, however, generally requires information about the initial state of the system. Gathering this information becomes increasingly challenging when the initial state is not a priori known and the system’s size grows large. To address this problem, we develop a machine-learning algorithm that uses a small amount of measurement data to construct its internal representation of the system’s state. The algorithm compares this data-driven representation with a representation of the target state, and uses reinforcement learning to output the appropriate control operations. We illustrate the effectiveness of the algorithm showing that it achieves accurate control of unknown many-body quantum states and non-Gaussian continuous-variable states using data from a limited set of quantum measurements.
Accurate control of quantum states is crucial for quantum computing and other quantum technologies. In the basic scenario, the task is to steer a quantum system towards a target state through a sequence of control operations. Determining the appropriate operations, however, generally requires information about the initial state of the system. Gathering this information becomes increasingly challenging when the initial state is not a priori known and the system’s size grows large. To address this problem, we develop a machine-learning algorithm that uses a small amount of measurement data to construct its internal representation of the system’s state. The algorithm compares this data-driven representation with a representation of the target state, and uses reinforcement learning to output the appropriate control operations. We illustrate the effectiveness of the algorithm showing that it achieves accurate control of unknown many-body quantum states and non-Gaussian continuous-variable states using data from a limited set of quantum measurements.
There is no doubt that quantum computing has opened up new horizons and perspectives in many fields, including scientific research. With the advancement of technology and quantum computers, we can now conduct new kinds of scientific experiments: observing quantum properties of individual systems, atoms, electrons and photons as well as influencing and controlling quantum systems. Quantum mechanics describes the properties of quantum systems by their quantum states. Unlike in classical mechanics, quantum states themselves cannot be directly observed in experiments, so many-body dynamics mathematical simulations are useful. Even though many-body dynamics problems can be solved for limited cases, but when we consider real-world pieces of matter this leads to computation-intensive calculations that are beyond classic digital computers. The memory and time needed to describe the quantum state of a many-body system scales exponentially with the size of the system. This has led physics communities to focus their attention on the algorithms underlying modern machine learning with the goal of making progress in quantum matter research. As a consequence, the successful application of machine learning requires effective and informative data representation. This paper discusses some novel aspects of applying classical machine learning algorithms to quantum data derived from measurements associated with quantum systems in many-body simulation problems. In this paper, we investigate and experimentally demonstrate one of the effective methods of the quantum data representation suited for the classical machine learning applications based on quantum classical shadows.
Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in measuring these systems. This often leads to reliance on assumed models, introducing model bias, a problem that is exacerbated in open quantum dynamics where Markovian approximations are not valid. To address these challenges, we propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. By initially employing a series of measurements as a 'dataset', we utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. Our methodology aims to demonstrates that by integrating reinforcement learning with model learning, it is possible to devise control policies and models that can counteract decoherence in a spin-boson system. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics, paving the way for more effective quantum control strategies.
Neural networks are a promising tool for characterizing intermediate-scale quantum devices from limited amounts of measurement data. A challenging problem in this area is to learn the action of an unknown quantum process on an ensemble of physically relevant input states. To tackle this problem, we introduce a neural network that emulates the unknown process by constructing an internal representation of the input ensemble and by mimicking the action of the process at the state representation level. After being trained with measurement data from a few pairs of input/output quantum states, the network becomes able to predict the measurement statistics for all inputs in the ensemble of interest. We show that our model exhibits high accuracy in applications to quantum computing, quantum photonics, and quantum many-body physics.
Characterization of quantum systems from experimental data is a central problem in quantum science and technology. But which measurements should be used to gather data in the first place? While optimal measurement choices can be worked out for small quantum systems, the optimization becomes intractable as the system size grows large. To address this problem, we introduce a deep neural network with a sequence model architecture that searches for efficient measurement choices in a data-driven, adaptive manner. The model can be applied to a variety of tasks, including the prediction of linear and nonlinear properties of quantum states, as well as state clustering and state tomography tasks. In all these tasks, we find that the measurement choices identified by our neural network consistently outperform the uniformly random choice. Intriguingly, for topological quantum systems, our model tends to recommend measurements at the system's boundaries, even when the task is to predict bulk properties. This behavior suggests that the neural network may have independently discovered a connection between boundaries and bulk, without having been provided any built-in knowledge of quantum physics.
With advances in digital technology in recent years, parallel computation utilizing GPUs has achieved remarkable efficiency, which has made it possible to use large-scale machine learning algorithms with large datasets, and deep learning, which is machine learning utilizing deep neural networks trained with millions to billions of data and parameters, has become increasingly popular and found its use for various tasks including facial recognition, language translation and combinatorial problems, etc., achieving considerably many record-breaking results.As a powerful generic tool to solve problems, deep learning has been applied to physical problems as well, and has become an important alternative tool for scientists to solve a number of important problems.With the development of experimental quantum technology over the past several decades, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems, which have opened up a new regime of complex quantum systems that can be engineered.Quantum control has found its use in controlled quantum-chemical processes and artificial quantum systems including quantum dots, superconducting qubits, trapped ions, and cavity optomechanical systems, etc., which are of considerable importance for future technology as candidates for sensors and quantum-computational devices.However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and generic numerical algorithms which search for proper control protocols are adopted.Therefore, deep learning, especially deep reinforcement learning, is a promising generic candidate solution for the control problems.Although there have already been a few successful examples of applications of deep reinforcement learning to quantum control problems, most of the existing reinforcement learning algorithms intrinsically suffer from instabilities and unsatisfactory reproducibility, and as a consequence, they typically require a large amount of fine-tuning and a large computational budget, both of which limit their applicability to quantum control problems and require expertise in machine learning.To resolve the issue of instabilities of the reinforcement learning algorithms, in this thesis, we first investigate the non-convergence issue of Q-learning, which is one of the most efficient reinforcement learning strategies.Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm.We prove the convergence of the C-DQN algorithm, and since the algorithm is scalable and computationally efficient, we apply it to a standard reinforcement learning benchmark, the Atari 2600 benchmark, to demonstrate its effectiveness.We show that when the DQN algorithm fail, the C-DQN algorithm still learns successfully.Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum-mechanical quartic oscillator i
Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.