Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1707.00810 · IEEE Transactions on Information Theory

Rényi Resolvability and Its Applications to the Wiretap Channel

Lei Yu, Vincent Y. F. Tan

Published Dec 7, 2018• 45 citations
arXivHTMLPDF

Abstract

The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy. In contrast to previous works, in this paper, we use the (normalized or unnormalized) Rényi divergence (with the Rényi parameter in[0, 2] U {∞}) to measure the level of approximation. We also provide asymptotic expressions for normalized Rényi divergence when the Rényi parameter is larger than or equal to 1 as well as (lower and upper) bounds for the case when the same parameter is smaller than 1. We characterize the Rényi resolvability, which is defined as the minimum rate required to ensure that the Rényi divergence vanishes asymptotically. The Rényi resolvabilities are the same for both the normalized and unnormalized divergence cases. In addition, when the Rényi parameter smaller than 1, consistent with the traditional case where the Rényi parameter is equal to 1, the Rényi resolvability equals the minimum mutual information over all input distributions that induce the target output distribution. When the Rényi parameter is larger than 1 the Rényi resolvability is, in general, larger than the mutual information. The optimal Rényi divergence is proven to vanish at least exponentially fast for both of these two cases, as long as the code rate is larger than the Rényi resolvability. The optimal exponential rate of decay for i.i.d. random codes is also characterized exactly. We apply these results to the wiretap channel, and completely characterize the optimal tradeoff between the rates of the secret and non-secret messages when the leakage measure is given by the (unnormalized) Rényi divergence. This tradeoff differs from the conventional setting when the leakage is measured by the traditional mutual information.

DOI
10.1109/tit.2018.2885318

Related papers

$E_{\gamma}$-Resolvability

Jingbo Liu, Paul Cuff, Sergio Verdú

The conventional channel resolvability refers to the minimum rate needed for an input process to approximate the channel output distribution in total variation distance. In this paper, we study Eγ-resolvability, in which total variation is replaced by the more general Eγdistance. A general one-shot achievability bound for the precision of such an approximation is developed. Let QX|Ube a random transformation, n be an integer, and E ∈ (0, +∞). We show that in the asymptotic setting where γ = exp(nE), a (nonnegative) randomness rate above inf QU:D(QXIIπX)≤E{D(QXIIπX) + I(QU, QX|U) - E} is sufficient to approximate the output distribution πX⊗nusing the channel QX|U⊗n, where QU→ QX|U→ QX, and is also necessary in the case of finite U and X . In particular, a randomness rate of inf QUI(QU, QX|U) - E is always sufficient. We also study the convergence of the approximation error under the high-probability criteria in the case of random codebooks. Moreover, by developing simple bounds relating Eγand other distance measures, we are able to determine the exact linear growth rate of the approximation errors measured in relative entropy and smooth Rényi divergences for a fixed-input randomness rate. The new resolvability result is then used to derive: 1) a one-shot upper bound on the probability of excess distortion in lossy compression, which is exponentially tight in the i.i.d. setting; 2) a one-shot version of the mutual covering lemma; and 3) a lower bound on the size of the eavesdropper list to include the actual message and a lower bound on the eavesdropper false-alarm probability in the wiretap channel problem, which is (asymptotically) ensemble-tight.

arXiv:1511.07829•IEEE Transactions on Information Theory•Jan 1, 2016•32 citations

Large Deviation Analysis for the Reverse Shannon Theorem

Shi-Bing Li, Ke Li, Lei Yu

We consider the problem of simulating a noisy channel using noiseless channels with unlimited shared randomness. This can be interpreted as the reverse problem of Shannon’s noisy channel coding theorem. In contrast to previous works, we employ Rényi divergence (with the parameter α ∈ [0,∞]) to measure the level of approximation, and we obtain the reverse Shannon theorem under this measure, which characterizes the Rényi simulation rate, the minimum communication rate required for the Rényi divergence vanishing asymptotically. Our derivation is done by a precise large-deviation analysis. When the communication rate is above the Rényi simulation rate, we provide a complete characterization of the convergence exponent for the Rényi divergence, called the reliability function. When the communication rate is below the Rényi simulation rate, we determine the linear increasing rate for the Rényi divergence, which implies the strong converse exponent for the order-α fidelity.

arXiv:2410.07984•IEEE Transactions on Information Theory•May 25, 2026•4 citations

Resolvability in Eγ with applications to lossy compression and wiretap channels

Jingbo Liu, Paul Cuff, Sergio Verdu

We study the amount of randomness needed for an input process to approximate a given output distribution of a channel in the Eγ distance. A general one-shot achievability bound for the precision of such an approximation is developed. In the i.i.d. setting where γ = exp(nE), a (nonnegative) randomness rate above infQU:D(QX||πX)≤E{D(QX||πX) + I(QU, QX|U) - E} is necessary and sufficient to asymptotically approximate the output distribution πX⊗nusing the channel QX|U⊗n, where QU→ QX|U→ QX. The new resolvability result is then used to derive a oneshot upper bound on the error probability in the rate distortion problem; and a lower bound on the size of the eavesdropper list to include the actual message in the wiretap channel problem. Both bounds are asymptotically tight in i.i.d. settings.

arXiv:1506.00154•IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2015•9 citations

On channel resolvability in presence of Feedback

Mani Bastani Parizi, Emre Telatar

We study the problem of generating an approximately i.i.d. string at the output of a discrete memoryless channel using a limited amount of randomness at its input in presence of causal noiseless feedback. Feedback does not decrease the channel resolution, the minimum entropy rate required to achieve an accurate approximation of an i.i.d. output string. However, we show that, at least over a binary symmetric channel, a significantly larger resolvability exponent (the exponential decay rate of the divergence between the output distribution and product measure), compared to the best known achievable resolvability exponent in a system without feedback, is possible. We show that by employing a variable-length resolvability scheme and using an average number of R coin-flips per channel use, the average divergence between the distribution of the output sequence and product measure decays exponentially fast in the average length of output sequence with an exponent equal to [R - I(U; V)]+where I(U; V) is the mutual information developed across the channel.

Allerton Conference on Communication, Control, and Computing•Sep 1, 2016•0 citations

Optimum Source Resolvability Rate with Respect to f-Divergences Using the Smooth Rényi Entropy

Ryo Nomura, Hideki Yagi

The source resolvability problem (or resolvability problem for short) is one of random number generation problems in information theory. In the literature, the optimum achievable rates in the resolvability problem have been characterized in different two ways. One is based on the information spectrum quantity and the other is based on the smooth Rényi entropy. Recently, Nomura has revealed the optimum achievable rate with respect to the f-divergence, which includes the variational distance, the Kullback-Leibler (KL) divergence and so on. On the other hand, the optimum achievable rates with respect to the variational distance has been characterized by using the smooth Rényi entropy. In this paper, we try to extend this result to the case of other distances. To do so, we consider the resolvability problem with respect to the subclass of f-divergences and determine the optimum achievable rate in terms of the smooth Rényi entropy. The subclass of f-divergences considered in this paper includes typical distance measures such as the total variational distance, the KL divergence, the Hellinger distance and so on.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2020•1 citation

On Channel Resolvability in Presence of Feedback

Mani Bastani Parizi, Emre Telatar

We study the problem of generating an approximately i.i.d. string at the output of a discrete memoryless channel using a limited amount of randomness at its input in presence of causal noiseless feedback. Feedback does not decrease the channel resolution, the minimum entropy rate required to achieve an accurate approximation of an i.i.d. output string. However, we show that, at least over a binary symmetric channel, a significantly larger resolvability exponent (the exponential decay rate of the divergence between the output distribution and product measure), compared to the best known achievable resolvability exponent in a system without feedback, is possible. We show that by employing a variable-length resolvability scheme and using an average number of coin-flips per channel use, the average divergence between the distribution of the output sequence and product measure decays exponentially fast in the average length of output sequence with an exponent equal to $[R-I(U;V)]^+$ where $I(U;V)$ is the mutual information developed across the channel.

arXiv:1607.02298•Jul 8, 2016•0 citations
The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy. In contrast to previous works, in this paper, we use the (normalized or unnormalized) Rényi divergence (with the Rényi parameter in[0, 2] U {∞}) to measure the level of approximation. We also provide asymptotic expressions for normalized Rényi divergence when the Rényi parameter is larger than or equal to 1 as well as (lower and upper) bounds for the case when the same parameter is smaller than 1. We characterize the Rényi resolvability, which is defined as the minimum rate required to ensure that the Rényi divergence vanishes asymptotically. The Rényi resolvabilities are the same for both the normalized and unnormalized divergence cases. In addition, when the Rényi parameter smaller than 1, consistent with the traditional case where the Rényi parameter is equal to 1, the Rényi resolvability equals the minimum mutual information over all input distributions that induce the target output distribution. When the Rényi parameter is larger than 1 the Rényi resolvability is, in general, larger than the mutual information. The optimal Rényi divergence is proven to vanish at least exponentially fast for both of these two cases, as long as the code rate is larger than the Rényi resolvability. The optimal exponential rate of decay for i.i.d. random codes is also characterized exactly. We apply these results to the wiretap channel, and completely characterize the optimal tradeoff between the rates of the secret and non-secret messages when the leakage measure is given by the (unnormalized) Rényi divergence. This tradeoff differs from the conventional setting when the leakage is measured by the traditional mutual information.
EγE_{\gamma}Eγ​-Resolvability
The conventional channel resolvability refers to the minimum rate needed for an input process to approximate the channel output distribution in total variation distance. In this paper, we study Eγ-resolvability, in which total variation is replaced by the more general Eγdistance. A general one-shot achievability bound for the precision of such an approximation is developed. Let QX|Ube a random transformation, n be an integer, and E ∈ (0, +∞). We show that in the asymptotic setting where γ = exp(nE), a (nonnegative) randomness rate above inf QU:D(QXIIπX)≤E{D(QXIIπX) + I(QU, QX|U) - E} is sufficient to approximate the output distribution πX⊗nusing the channel QX|U⊗n, where QU→ QX|U→ QX, and is also necessary in the case of finite U and X . In particular, a randomness rate of inf QUI(QU, QX|U) - E is always sufficient. We also study the convergence of the approximation error under the high-probability criteria in the case of random codebooks. Moreover, by developing simple bounds relating Eγand other distance measures, we are able to determine the exact linear growth rate of the approximation errors measured in relative entropy and smooth Rényi divergences for a fixed-input randomness rate. The new resolvability result is then used to derive: 1) a one-shot upper bound on the probability of excess distortion in lossy compression, which is exponentially tight in the i.i.d. setting; 2) a one-shot version of the mutual covering lemma; and 3) a lower bound on the size of the eavesdropper list to include the actual message and a lower bound on the eavesdropper false-alarm probability in the wiretap channel problem, which is (asymptotically) ensemble-tight.
We consider the problem of simulating a noisy channel using noiseless channels with unlimited shared randomness. This can be interpreted as the reverse problem of Shannon’s noisy channel coding theorem. In contrast to previous works, we employ Rényi divergence (with the parameter α ∈ [0,∞]) to measure the level of approximation, and we obtain the reverse Shannon theorem under this measure, which characterizes the Rényi simulation rate, the minimum communication rate required for the Rényi divergence vanishing asymptotically. Our derivation is done by a precise large-deviation analysis. When the communication rate is above the Rényi simulation rate, we provide a complete characterization of the convergence exponent for the Rényi divergence, called the reliability function. When the communication rate is below the Rényi simulation rate, we determine the linear increasing rate for the Rényi divergence, which implies the strong converse exponent for the order-α fidelity.
We study the amount of randomness needed for an input process to approximate a given output distribution of a channel in the Eγ distance. A general one-shot achievability bound for the precision of such an approximation is developed. In the i.i.d. setting where γ = exp(nE), a (nonnegative) randomness rate above infQU:D(QX||πX)≤E{D(QX||πX) + I(QU, QX|U) - E} is necessary and sufficient to asymptotically approximate the output distribution πX⊗nusing the channel QX|U⊗n, where QU→ QX|U→ QX. The new resolvability result is then used to derive a oneshot upper bound on the error probability in the rate distortion problem; and a lower bound on the size of the eavesdropper list to include the actual message in the wiretap channel problem. Both bounds are asymptotically tight in i.i.d. settings.
We study the problem of generating an approximately i.i.d. string at the output of a discrete memoryless channel using a limited amount of randomness at its input in presence of causal noiseless feedback. Feedback does not decrease the channel resolution, the minimum entropy rate required to achieve an accurate approximation of an i.i.d. output string. However, we show that, at least over a binary symmetric channel, a significantly larger resolvability exponent (the exponential decay rate of the divergence between the output distribution and product measure), compared to the best known achievable resolvability exponent in a system without feedback, is possible. We show that by employing a variable-length resolvability scheme and using an average number of R coin-flips per channel use, the average divergence between the distribution of the output sequence and product measure decays exponentially fast in the average length of output sequence with an exponent equal to [R - I(U; V)]+where I(U; V) is the mutual information developed across the channel.
The source resolvability problem (or resolvability problem for short) is one of random number generation problems in information theory. In the literature, the optimum achievable rates in the resolvability problem have been characterized in different two ways. One is based on the information spectrum quantity and the other is based on the smooth Rényi entropy. Recently, Nomura has revealed the optimum achievable rate with respect to the f-divergence, which includes the variational distance, the Kullback-Leibler (KL) divergence and so on. On the other hand, the optimum achievable rates with respect to the variational distance has been characterized by using the smooth Rényi entropy. In this paper, we try to extend this result to the case of other distances. To do so, we consider the resolvability problem with respect to the subclass of f-divergences and determine the optimum achievable rate in terms of the smooth Rényi entropy. The subclass of f-divergences considered in this paper includes typical distance measures such as the total variational distance, the KL divergence, the Hellinger distance and so on.
We study the problem of generating an approximately i.i.d. string at the output of a discrete memoryless channel using a limited amount of randomness at its input in presence of causal noiseless feedback. Feedback does not decrease the channel resolution, the minimum entropy rate required to achieve an accurate approximation of an i.i.d. output string. However, we show that, at least over a binary symmetric channel, a significantly larger resolvability exponent (the exponential decay rate of the divergence between the output distribution and product measure), compared to the best known achievable resolvability exponent in a system without feedback, is possible. We show that by employing a variable-length resolvability scheme and using an average number of coin-flips per channel use, the average divergence between the distribution of the output sequence and product measure decays exponentially fast in the average length of output sequence with an exponent equal to [R−I(U;V)]+[R-I(U;V)]^+[R−I(U;V)]+ where I(U;V)I(U;V)I(U;V) is the mutual information developed across the channel.