In this work, we present quantum reinforcement learning algorithms for process flowsheet synthesis. Particularly, we discuss the implementation of encoding strategies to improve the algorithmic scalability. Reinforcement learning (RL)-driven flowsheet synthesis techniques provide a promising approach for conceptual process design, in addition to traditional optimization-based methods. These RL-based strategies identify the optimal flowsheet configurations from a maximum set of available processing units, without requiring to pre-postulate an interconnected superstructure. However, the resulting combinatorial design space for RL can scale extensively with the increased number of available processing units, which can render the algorithms to be computationally intensive or even intractable. To address this challenge, our prior work has introduced a quantum-enhanced approach to RL-driven process synthesis. However, this algorithm was limited in its capacity to solve larger flowsheeting problems as a large number of qubits were needed. To reduce qubit requirements and improve scalability, this work presents two different encoding strategies to embed information into the gates of quantum circuits instead of embedding it into qubits alone. We demonstrate the efficacy of these algorithms on three different scenarios of a flowsheet synthesis problem and interpret the results obtained from the IonQ quantum computing platform.
In this work, we present quantum reinforcement learning algorithms for process flowsheet synthesis. Particularly, we discuss the implementation of encoding strategies to improve the algorithmic scalability. Reinforcement learning (RL)-driven flowsheet synthesis techniques provide a promising approach for conceptual process design, in addition to traditional optimization-based methods. These RL-based strategies identify the optimal flowsheet configurations from a maximum set of available processing units, without requiring to pre-postulate an interconnected superstructure. However, the resulting combinatorial design space for RL can scale extensively with the increased number of available processing units, which can render the algorithms to be computationally intensive or even intractable. To address this challenge, our prior work has introduced a quantum-enhanced approach to RL-driven process synthesis. However, this algorithm was limited in its capacity to solve larger flowsheeting problems as a large number of qubits were needed. To reduce qubit requirements and improve scalability, this work presents two different encoding strategies to embed information into the gates of quantum circuits instead of embedding it into qubits alone. We demonstrate the efficacy of these algorithms on three different scenarios of a flowsheet synthesis problem and interpret the results obtained from the IonQ quantum computing platform.
In this work, we present quantum reinforcement learning (RL) as a solution strategy for process synthesis problems. Building on our prior work, we develop a generalized framework that formally poses process synthesis as a Markov decision process and introduces quantum-enhanced RL algorithms to solve it with improved scalability. Earlier implementations of quantum-based RL for process synthesis were limited by qubit requirements, which scaled poorly with problem complexity. This work overcomes this challenge by introducing state encoding algorithms to decouple qubit requirements from problem size. A classical RL-based solution strategy is used as a baseline to benchmark the quantum algorithms under identical training conditions. All algorithms are evaluated across a flowsheet synthesis problem of increasing unit counts to analyze their performance and scalability. Results show that all approaches are capable of identifying the optimal flowsheet designs in small design spaces. For moderate-scale unit counts, quantum approaches demonstrate competitive performance on a per-episode basis and improved efficiency on a per-parameter basis versus the classical RL benchmark. This work provides a foundation for future quantum computing applications within process systems engineering, establishes a controlled benchmark for comparing classical and quantum algorithms, and shows that the proposed quantum variants remain competitive for the process synthesis problem examined in this work.
In this work, we introduce a quantum-enhanced reinforcement learning (RL) framework for process design synthesis. RL-driven methods for generating process designs have gained momentum due to their ability to intelligently identify optimal configurations without requiring pre-defined superstructures or flowsheet configurations. This eliminates reliance on prior expert knowledge, offering a comprehensive and robust design strategy. However, navigating the vast combinatorial design space poses computational challenges. To address this, a novel approach integrating RL with quantum machine learning (QML) is proposed. QML leverages theoretical advantages over classical methods to accelerate searches in large spaces. Built upon our prior work, the approach begins with a maximum set of available unit operations, represented in a flowsheet structure using an input-output stream matrix as RL observations. A Deep Q-Network (DQN) algorithm trains a parameterized quantum circuit (PQC) in place of a classical neural network (NN). The design structures generated by the RL agent are optimized using the IDAES Process Systems Engineering Framework, with the optimization objectives used as rewards to RL (e.g., cost, productivity). This quantum-enhanced algorithm is performed on a hydrodealkylation process case study, showcasing its efficiency and improved performance in navigating complex design spaces.
Quantum unitary synthesis addresses the problem of translating abstract quantum algorithms into sequences of hardware-executable quantum gates. Solving this task exactly is infeasible in general due to the exponential growth of the underlying combinatorial search space. Existing approaches suffer from misaligned optimization objectives, substantial training costs and limited generalization across different qubit counts. We mitigate these limitations by using supervised learning to approximate the minimum description length of residual unitaries and combining this estimate with stochastic beam search to identify near optimal gate sequences. Our method relies on a lightweight model with zero-shot generalization, substantially reducing training overhead compared to prior baselines. Across multiple benchmarks, we achieve faster wall-clock synthesis times while exceeding state-of-the-art methods in terms of success rate for complex circuits.
The design of autonomous quantum systems constitutes an entirely new paradigm for next-generation computing by having machine intelligence do the engineering work to build and optimise successful quantum architectures. We address the problem of self-optimizing quantum systems that perform adaptive control, circuit synthesis and decision making in both accurate and uncertain controllable environments via a two-pronged approach: Reinforcement Learning (RL) together with Evolutionary Strategies (ES). In this work, we find that reinforcement learning provides a general framework for sequential decision-making, where agents learn optimal policies from reward signals by interacting with quantum environments. Evolutionary strategies, which are based on ideas from biological evolution, provide an alternative to RL by performing global optimization of quantum circuit parameters and architectures using the standard canonical processes of mutation, selection and recombination. Recent developments in quantum technologies show, again, that machine learning tools (ML), especially RL can be very useful to optimize unavailable closed blow-ups related with quantity functions. Moreover, evolutionary optimization based hybrids with RL have demonstrated better performance on the complex multi-objective quantum problems due to balanced exploitation and exploration. In this paper, we provide an overview of a versatile framework for autonomous quantum system design that showcases example algorithmic architectures, hybrid machine learning models and real world applications in areas such as quantum control, quantum circuit synthesis and energy optimization. In addition, we discuss well-known obstacles such as the scalability problem, the noise sensitivity issue, barren plateaus of quantum optimization and computational complexity. Comparisons to traditional methods show that reinforcement learning (RL) when combined with evolutionary strategies enables a significant improvement in efficiency, adaptability, and robustness for quantum system design as supported by experimental studies and case analyses. Future research directions are then discussed, including quantum-native learning algorithms, multi-agent quantum reinforcement learning and hardware-aware optimization techniques
Quantum computing offers exciting opportunities for simulating complex quantum systems and optimizing large-scale combinatorial problems, but its practical use is limited by device noise and constrained connectivity. Designing quantum circuits, which are fundamental to quantum algorithms, is therefore a central challenge in current quantum hardware. Existing reinforcement learning-based methods for circuit design lose accuracy when restricted to hardware-native gates and device-level compilation. Here, we introduce gadget reinforcement learning (GRL) that combines learning with program synthesis to automatically construct composite gates that expand the action space while respecting hardware constraints. We show that this approach improves accuracy, hardware compatibility, and scalability for transverse-field Ising and quantum chemistry problems, reaching systems of up to ten qubits within realistic computational budgets. This framework demonstrates how learned, reusable circuit building blocks can guide the co-design of algorithms and hardware for quantum processors. Designing quantum circuits is challenging due to the exponential growth of the state space. The authors introduce gadget reinforcement learning, which combines reinforcement learning with program synthesis to enhance quantum circuit exploration, demonstrating improved accuracy and scalability in complex tasks, thus bridging algorithmic design and practical implementation for hardware-specific optimizations.
The optimization of the limited resources of current quantum devices hinges on the implementation of quantum-compiling techniques. The process of transforming high-level quantum algorithms into executable quantum circuits has a significant impact on the performance and fidelity of computations. One of the most critical aspects of the compiling stack is the synthesis of quantum circuits, which aims to minimize the depth and number of gates, as well as the error rate. Nevertheless, the optimal synthesis algorithms are extremely computationally expensive. This is why synthesis algorithms based on reinforcement learning (RL) have recently been introduced. However, such algorithms are constrained by the topology of quantum processors, necessitating retraining of the RL agent for different topologies. To achieve a balance between adaptability, optimality, and computational cost in RL-based quantum circuit synthesis techniques, this work proposes an RL-based quantum circuit synthesis algorithm of Clifford operators that addresses different topologies simultaneously. Experimental tests demonstrate that our synthesizer outperforms state-of-the-art greedy techniques while being significantly faster than boolean satisfiability solvers (SAT).