Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2008.07454 · Quantum Science and Technology

Higher order derivatives of quantum neural networks with barren plateaus

Marco Cerezo, Patrick J. Coles

Published Apr 6, 2021• 144 citations
arXivHTMLPDF

Abstract

Abstract Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speed up applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speed-up is the barren plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size n for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with n . Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training QNNs (outside of the context of BPs).

DOI
10.1088/2058-9565/abf51a

Related papers

Impact of Barren Plateaus on the Hessian and Higher Order Derivatives

Marco Cerezo, Patrick J. Coles

The Barren Plateau (BP) phenomenon is an issue for certain quantum neural networks and variational quantum algorithms, whereby the gradient vanishes exponentially in the system size $n$. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with $n$. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent.

•Aug 17, 2020•23 citations

Absence of Barren Plateaus in Quantum Convolutional Neural Networks

Arthur Chalom Pesah, Marco Cerezo, Samson Wang +3 more

Quantum neural networks (QNNs) have generated excitement around the possibility of efficiently analyzing quantum data.But this excitement has been tempered by the existence of exponentially vanishing gradients, known as barren plateau landscapes, for many QNN architectures.Recently, quantum convolutional neural networks (QCNNs) have been proposed, involving a sequence of convolutional and pooling layers that reduce the number of qubits while preserving information about relevant data features.In this work, we rigorously analyze the gradient scaling for the parameters in the QCNN architecture.We find that the variance of the gradient vanishes no faster than polynomially, implying that QCNNs do not exhibit barren plateaus.This result provides an analytical guarantee for the trainability of randomly initialized QCNNs, which highlights QCNNs as being trainable under random initialization unlike many other QNN architectures.To derive our results, we introduce a novel graph-based method to analyze expectation values over Haar-distributed unitaries, which will likely be useful in other contexts.Finally, we perform numerical simulations to verify our analytical results.

arXiv:2011.02966•Physical Review X•Oct 15, 2021•412 citations

Effect of barren plateaus on gradient-free optimization

Andrew Arrasmith, M. Cerezo, Piotr Czarnik +2 more

Barren plateau landscapes correspond to gradients that vanish exponentially in the number of qubits. Such landscapes have been demonstrated for variational quantum algorithms and quantum neural networks with either deep circuits or global cost functions. For obvious reasons, it is expected that gradient-based optimizers will be significantly affected by barren plateaus. However, whether or not gradient-free optimizers are impacted is a topic of debate, with some arguing that gradient-free approaches are unaffected by barren plateaus. Here we show that, indeed, gradient-free optimizers do not solve the barren plateau problem. Our main result proves that cost function differences, which are the basis for making decisions in a gradient-free optimization, are exponentially suppressed in a barren plateau. Hence, without exponential precision, gradient-free optimizers will not make progress in the optimization. We numerically confirm this by training in a barren plateau with several gradient-free optimizers (Nelder-Mead, Powell, and COBYLA algorithms), and show that the numbers of shots required in the optimization grows exponentially with the number of qubits.

arXiv:2011.12245•Quantum•Oct 5, 2021•308 citations

Quantum Generative Training Using Rényi Divergences

Mária Kieferová, Ortiz Marrero Carlos, Nathan Wiebe

Quantum neural networks (QNNs) are a framework for creating quantum algorithms that promises to combine the speedups of quantum computation with the widespread successes of machine learning. A major challenge in QNN development is a concentration of measure phenomenon known as a barren plateau that leads to exponentially small gradients for a range of QNNs models. In this work, we examine the assumptions that give rise to barren plateaus and show that an unbounded loss function can circumvent the existing no-go results. We propose a training algorithm that minimizes the maximal Rényi divergence of order two and present techniques for gradient computation. We compute the closed form of the gradients for Unitary QNNs and Quantum Boltzmann Machines and provide sufficient conditions for the absence of barren plateaus in these models. We demonstrate our approach in two use cases: thermal state learning and Hamiltonian learning. In our numerical experiments, we observed rapid convergence of our training loss function and frequently archived a $99\%$ average fidelity in fewer than $100$ epochs.

arXiv:2106.09567•Jun 17, 2021•51 citations

The Goldilocks Regime: Quantum-Aware Activation Functions for Barren Plateau Mitigation in Hybrid QNNs

R. Gökhan Türeci

Abstract The barren plateau phenomenon --- the exponential vanishing of gradient norms with increasing qubit number --- remains a fundamental obstacle in training hybrid classical-quantum neural networks. We propose that the choice of activation function at the classical-quantum interface constitutes an underexplored yet effective lever for barren plateau mitigation. Specifically, we introduce the concept of the Goldilocks Regime : the range of maximum activation derivative magnitudes \((\max|f'| \in (1.0,\, \pi/2])\) within which quantum-aware activation functions achieve optimal gradient preservation in parameterized quantum circuits.Through systematic experiments on hardware-efficient parameterized quantum circuits with up to 18 qubits using Qiskit 2.3.1 and the parameter shift rule, we evaluate three quantum-aware activation functions --- PiTanh (\((\pi\cdot\tanh(x))\)), SmoothAngle (\((\pi\cdot(2\cdot\text{sigmoid}(x)-1))\)), and LearnableAngle (\((\pi\cdot\tanh(\alpha x))\)) --- alongside random uniform initialization as a baseline.LearnableAngle with \((\alpha = 0.4)\) (\((\max|f'| \approx 1.257)\)), lying within the Goldilocks Regime, consistently achieves the highest gradient norm across all tested qubit counts, maintaining \((0.404)\) at \((n = 18)\) compared to \((0.013)\) for random initialization --- a 31-fold improvement. An ablation study incorporating the Physics-Constrained Surrogate Modelling (PCSM) framework reveals that physical derivative constraints provide the greatest benefit when the activation function operates outside the optimal Goldilocks Regime, establishing activation design and PCSM regularization as complementary strategies for gradient preservation.These results identify the Goldilocks Regime as a principled design criterion for quantum-aware activation functions and demonstrate that structured classical pre-networks can substantially delay the onset of barren plateaus without modifying the quantum circuit architecture.

•Apr 20, 2026•0 citations

Benchmarking Barren Plateau Mitigation Strategies in Quantum Neural Networks on Standard and Medical Image Datasets

Maqsudur Rahman, Rui Liu, Anup Majumder +8 more

Barren plateaus (BPs) pose a major trainability challenge for quantum neural networks (QNNs) by causing gradients to concentrate near zero as circuit size, depth, or expressibility increases. This study presents a comparative benchmark of 10 BP mitigation strategies across six qubit settings (2, 4, 8, 12, 16, and 20) and three datasets of increasing complexity: Iris, MNIST, and MedMNIST. The evaluated methods include eight initialization-based strategies (Beta, Gaussian, Uniform Norm, CNN-based initialization, He-normal, He-uniform, Xavier-normal, and Xavier-uniform), one model-based variational encoder, and one optimization-based time-nonlocal Fourier parameterization. Experiments were implemented using PennyLane 3.10 and PyTorch 2.5 with simulator backends. We evaluate trainability using gradient variance and training loss, and we clarify that the benchmark analyzes simulated QNN optimization behavior rather than hardware-noise-resilient or noisy-label learning. Across the tested two-layer circuit configurations, the mitigation strategies maintained measurable gradient variance and stable loss reduction, suggesting that severe barren plateau behavior was not observed under the benchmark conditions. CNN-based and Beta initialization showed strong empirical behavior in variance retention and convergence speed, while Gaussian initialization was comparatively weaker in higher-dimensional settings. The study provides a reproducible benchmark structure for comparing BP mitigation behavior and identifies important limitations related to circuit depth, hardware noise, feature encoding, and classification performance that should be addressed in future QNN benchmarking.

Journal of Imaging•Jun 23, 2026•0 citations
Abstract Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speed up applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speed-up is the barren plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size n for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with n . Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training QNNs (outside of the context of BPs).
The Barren Plateau (BP) phenomenon is an issue for certain quantum neural networks and variational quantum algorithms, whereby the gradient vanishes exponentially in the system size nnn. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with nnn. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent.
Quantum neural networks (QNNs) have generated excitement around the possibility of efficiently analyzing quantum data.But this excitement has been tempered by the existence of exponentially vanishing gradients, known as barren plateau landscapes, for many QNN architectures.Recently, quantum convolutional neural networks (QCNNs) have been proposed, involving a sequence of convolutional and pooling layers that reduce the number of qubits while preserving information about relevant data features.In this work, we rigorously analyze the gradient scaling for the parameters in the QCNN architecture.We find that the variance of the gradient vanishes no faster than polynomially, implying that QCNNs do not exhibit barren plateaus.This result provides an analytical guarantee for the trainability of randomly initialized QCNNs, which highlights QCNNs as being trainable under random initialization unlike many other QNN architectures.To derive our results, we introduce a novel graph-based method to analyze expectation values over Haar-distributed unitaries, which will likely be useful in other contexts.Finally, we perform numerical simulations to verify our analytical results.
Barren plateau landscapes correspond to gradients that vanish exponentially in the number of qubits. Such landscapes have been demonstrated for variational quantum algorithms and quantum neural networks with either deep circuits or global cost functions. For obvious reasons, it is expected that gradient-based optimizers will be significantly affected by barren plateaus. However, whether or not gradient-free optimizers are impacted is a topic of debate, with some arguing that gradient-free approaches are unaffected by barren plateaus. Here we show that, indeed, gradient-free optimizers do not solve the barren plateau problem. Our main result proves that cost function differences, which are the basis for making decisions in a gradient-free optimization, are exponentially suppressed in a barren plateau. Hence, without exponential precision, gradient-free optimizers will not make progress in the optimization. We numerically confirm this by training in a barren plateau with several gradient-free optimizers (Nelder-Mead, Powell, and COBYLA algorithms), and show that the numbers of shots required in the optimization grows exponentially with the number of qubits.
Quantum neural networks (QNNs) are a framework for creating quantum algorithms that promises to combine the speedups of quantum computation with the widespread successes of machine learning. A major challenge in QNN development is a concentration of measure phenomenon known as a barren plateau that leads to exponentially small gradients for a range of QNNs models. In this work, we examine the assumptions that give rise to barren plateaus and show that an unbounded loss function can circumvent the existing no-go results. We propose a training algorithm that minimizes the maximal Rényi divergence of order two and present techniques for gradient computation. We compute the closed form of the gradients for Unitary QNNs and Quantum Boltzmann Machines and provide sufficient conditions for the absence of barren plateaus in these models. We demonstrate our approach in two use cases: thermal state learning and Hamiltonian learning. In our numerical experiments, we observed rapid convergence of our training loss function and frequently archived a 99%99\%99% average fidelity in fewer than 100100100 epochs.
Abstract The barren plateau phenomenon --- the exponential vanishing of gradient norms with increasing qubit number --- remains a fundamental obstacle in training hybrid classical-quantum neural networks. We propose that the choice of activation function at the classical-quantum interface constitutes an underexplored yet effective lever for barren plateau mitigation. Specifically, we introduce the concept of the Goldilocks Regime : the range of maximum activation derivative magnitudes (max⁡∣f′∣∈(1.0, π/2])(\max|f'| \in (1.0,\, \pi/2])(max∣f′∣∈(1.0,π/2]) within which quantum-aware activation functions achieve optimal gradient preservation in parameterized quantum circuits.Through systematic experiments on hardware-efficient parameterized quantum circuits with up to 18 qubits using Qiskit 2.3.1 and the parameter shift rule, we evaluate three quantum-aware activation functions --- PiTanh ((π⋅tanh⁡(x))(\pi\cdot\tanh(x))(π⋅tanh(x))), SmoothAngle ((π⋅(2⋅sigmoid(x)−1))(\pi\cdot(2\cdot\text{sigmoid}(x)-1))(π⋅(2⋅sigmoid(x)−1))), and LearnableAngle ((π⋅tanh⁡(αx))(\pi\cdot\tanh(\alpha x))(π⋅tanh(αx))) --- alongside random uniform initialization as a baseline.LearnableAngle with (α=0.4)(\alpha = 0.4)(α=0.4) ((max⁡∣f′∣≈1.257)(\max|f'| \approx 1.257)(max∣f′∣≈1.257)), lying within the Goldilocks Regime, consistently achieves the highest gradient norm across all tested qubit counts, maintaining (0.404)(0.404)(0.404) at (n=18)(n = 18)(n=18) compared to (0.013)(0.013)(0.013) for random initialization --- a 31-fold improvement. An ablation study incorporating the Physics-Constrained Surrogate Modelling (PCSM) framework reveals that physical derivative constraints provide the greatest benefit when the activation function operates outside the optimal Goldilocks Regime, establishing activation design and PCSM regularization as complementary strategies for gradient preservation.These results identify the Goldilocks Regime as a principled design criterion for quantum-aware activation functions and demonstrate that structured classical pre-networks can substantially delay the onset of barren plateaus without modifying the quantum circuit architecture.
Barren plateaus (BPs) pose a major trainability challenge for quantum neural networks (QNNs) by causing gradients to concentrate near zero as circuit size, depth, or expressibility increases. This study presents a comparative benchmark of 10 BP mitigation strategies across six qubit settings (2, 4, 8, 12, 16, and 20) and three datasets of increasing complexity: Iris, MNIST, and MedMNIST. The evaluated methods include eight initialization-based strategies (Beta, Gaussian, Uniform Norm, CNN-based initialization, He-normal, He-uniform, Xavier-normal, and Xavier-uniform), one model-based variational encoder, and one optimization-based time-nonlocal Fourier parameterization. Experiments were implemented using PennyLane 3.10 and PyTorch 2.5 with simulator backends. We evaluate trainability using gradient variance and training loss, and we clarify that the benchmark analyzes simulated QNN optimization behavior rather than hardware-noise-resilient or noisy-label learning. Across the tested two-layer circuit configurations, the mitigation strategies maintained measurable gradient variance and stable loss reduction, suggesting that severe barren plateau behavior was not observed under the benchmark conditions. CNN-based and Beta initialization showed strong empirical behavior in variance retention and convergence speed, while Gaussian initialization was comparatively weaker in higher-dimensional settings. The study provides a reproducible benchmark structure for comparing BP mitigation behavior and identifies important limitations related to circuit depth, hardware noise, feature encoding, and classification performance that should be addressed in future QNN benchmarking.