Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Curriculum learning for polar and PAC decoders

Viraj Nadkarni

Published Jul 19, 2022• 0 citations
OpenAlex

Abstract

Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5 th generation wireless standards (5G).Polar-Adjusted Convolutional (PAC) codes are a recent modification to Polar codes that provide better reliability even at shorter block lengths.Training efficient neural decoders for both kinds of codes proves challenging at longer block lengths or smaller model sizes.We show that the technique of Curriculum learning is useful in training such models.We also show that particular kinds of curricula work better than others in training the network and also explain reasons why this is the case using the structure of the encoding procedure in Polar codes.Various neural architectures, such as transformers, recurrent neural networks and convolutional neural networks, are tried.For a few codes, anomalous behaviour is observed in terms of which curriculum works the best based on what architecture is used.The work explicated in this thesis would not have been possible without the guidance of my adviser, Prof. Pramod Viswanath and close collaborators

Related papers

CRISP: Curriculum based Sequential Neural Decoders for Polar Code Family

S. Ashwin Hebbar, Viraj Nadkarni, Ashok Vardhan Makkuva +3 more

Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5th generation wireless standards (5G). However, there remains room for the design of polar decoders that are both efficient and reliable in the short blocklength regime. Motivated by recent successes of data-driven channel decoders, we introduce a novel $\textbf{C}$ur$\textbf{RI}$culum based $\textbf{S}$equential neural decoder for $\textbf{P}$olar codes (CRISP). We design a principled curriculum, guided by information-theoretic insights, to train CRISP and show that it outperforms the successive-cancellation (SC) decoder and attains near-optimal reliability performance on the Polar(32,16) and Polar(64,22) codes. The choice of the proposed curriculum is critical in achieving the accuracy gains of CRISP, as we show by comparing against other curricula. More notably, CRISP can be readily extended to Polarization-Adjusted-Convolutional (PAC) codes, where existing SC decoders are significantly less reliable. To the best of our knowledge, CRISP constructs the first data-driven decoder for PAC codes and attains near-optimal performance on the PAC(32,16) code.

arXiv:2210.00313•Oct 1, 2022•6 citations

A RNN Based Decoder for Polar Codes

Yuan Chai, Zuting Chen, Shuai Han +1 more

Polar code is the first type of constructive coding with achievable capacity, which has become the encoding scheme for 5G mobile communication systems and is a cutting-edge direction in channel coding research. However, traditional decoding algorithms for Polar codes still need to be improved in terms of errorcorrection performance and decoding delay. For Polar code short codes, due to the phenomenon of incomplete Polar, the traditional SC decoding algorithm has a problem of poor decoding performance. Algorithms that utilize deep learning for direct or auxiliary operations often yield more accurate computational results with lower computational complexity. Therefore, considering that applying deep learning to the communication field is a mainstream trend in the 6G era, a (16,8) short Polar code decoder based on recurrent neural network (RNN) is proposed. At the same time, a suitable dataset construction method was proposed, which achieved better performance than SC decoding algorithm under low signal-to-noise ratio by adjusting network parameters and training under the appropriate dataset.

International Conference on Information Processing and Network Provisioning (ICIPNP)•Oct 26, 2023•2 citations

Deep code: representation and learning algorithms for neural networks & their applications to communication codes

Ashok Vardhan Makkuva

Codes are the backbone of modern information age.Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication.Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough.Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades.On the other hand, designing codes is a challenging task, mostly driven by human ingenuity.Befittingly, historically, the progress in discovery of codes has been sporadic.In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning.Our major result is the invention of KO codes, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime.The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the Kronecker Operation (KO) central to RM and polar codes.These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes.This design technique can be viewed as an instantiation of the classical neural augmentation principle.In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle.We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades.B.1 Comparison of SGD on our losses (L 4 , L log ) vs. ℓ 2 and the EM algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .106 C.2 KO code achieves a significant gain over the Polar(64, 7) code in BLER when trained on AWGN channel.KO decoder also matches the optimal MAP decoder. . . . . . . . . . . . .108 xii C.6 KO(6, 1) code.Left: KO(6, 1) code achieves significant gain over RM(6, 1) code (with Dumer) when trained on AWGN channel.Right: Under the optimal MAP decoding, KO(6, 1) and RM(6, 1) codes achieve the same performance.Error rates for a random Gaussian codebook are also plotted as a baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . .120 C.1 Plotkin trees for the Polar(64, 7) encoder and our neural KO encoder counterpart.Both codes have dimension k = 7 and blocklength n = 64. . . . . . . . . . . . . . . . . . . . . .121 C.3 The same KO(8,2) encoder and decoder as those used in Figure 4.4 achieve a significant gain (without any retraining or fine-tuning) when list decoding is used together with the KO decoder.The magnitude of the gain is comparable to the gain achieved by the same list decoding technique on the successive cancellation decoder of the RM(8,2) code.We used the list decoding from [1] but without the permutation technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122 C.4 Plotkin trees for RM(8,2) and KO(8,2) encoders.Leaves are highlighted in green.Both codes have dimension k = 37 and blocklength n = 256. . . . . . . . . . . . . . . . . . . . . .123 C.5 Plotkin trees for the RM(8, 2) and KO(8, 2) decoders.Red arrows indicate the bit decoding order. . . . . . . .

•Jul 15, 2022•0 citations

Enhanced Learning For Recurrent Neural Network-Based Polar Decoder

Ziad Ibrahim, Yasmine A. H. Fahmy

Several researchers are interested in polar codes for the sake of their capacity which nearly catches memoryless channels capacity. They have been used as part of the 5G technology. Polar decoders like belief propagation decoders showed unfavorable performance during limited iterations. Nowadays, Machine-learning (ML) techniques are used to enhance the efficiency of polar codes decoders. Recurrent neural network belief propagation (RNN-BP) decoders proved better efficiency in a lower number of cycles. In this paper, a performance scheduling learning technique is proposed. This technique achieves a coding gain of 0.2 dB for the RNN-BP decoder at BER of 2*10−4. A new approach is also introduced in the quantization step. In this approach, the quantization is done after the learning process is finished instead of after each iteration. This approach fixes the issue of model convergence. These combined approaches show up to 0.45 dB improvement at BER of 2*10−4.

•Mar 29, 2022•11 citations

Improved polar decoder based on deep learning

Weihong Xu, Zhizhen Wu, Yeong-Luh Ueng +2 more

Deep learning recently shows strong competitiveness to improve polar code decoding. However, suffering from prohibitive training and computation complexity, the conventional deep neural network (DNN) is only possible for very short code length. In this paper, the main problems of deep learning in decoding are well solved. We first present the multiple scaled belief propagation (BP) algorithm, aiming at obtaining faster convergence and better performance. Based on this, deep neural network decoder (NND) with low complexity and latency, is proposed for any code length. The training only requires a small set of zero codewords. Besides, its computation complexity is close to the original BP. Experiment results show that the proposed (64,32) NND with 5 iterations achieves even lower bit error rate (BER) than the 30-iteration conventional BP and (512, 256) NND also outperforms conventional BP decoder with same iterations. The hardware architecture of basic computation block is given and folding technique is also considered, saving about 50% hardware cost.

SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems•Oct 1, 2017•157 citations

Scaling Deep Learning-Based Decoding of Polar Codes via Partitioning

Sebastian Cammerer, Tobias Gruber, Jakob Hoydis +1 more

The training complexity of deep learning-based channel decoders scales exponentially with the codebook size and therefore with the number of information bits. Thus, neural network decoding (NND) is currently only feasible for very short block lengths. In this work, we show that the conventional iterative decoding algorithm for polar codes can be enhanced when sub-blocks of the decoder are replaced by neural network (NN) based components. Thus, we partition the encoding graph into smaller sub-blocks and train them individually, closely approaching maximum a posteriori (MAP) performance per sub-block. These blocks are then connected via the remaining conventional belief propagation decoding stage(s). The resulting decoding algorithm is non-iterative and inherently enables a highlevel of parallelization, while showing a competitive bit error rate (BER) performance. We examine the degradation through partitioning and compare the resulting decoder to state-of-the art polar decoders such as successive cancellation list and belief propagation decoding.

Globecom•Dec 1, 2017•208 citations
Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5 th generation wireless standards (5G).Polar-Adjusted Convolutional (PAC) codes are a recent modification to Polar codes that provide better reliability even at shorter block lengths.Training efficient neural decoders for both kinds of codes proves challenging at longer block lengths or smaller model sizes.We show that the technique of Curriculum learning is useful in training such models.We also show that particular kinds of curricula work better than others in training the network and also explain reasons why this is the case using the structure of the encoding procedure in Polar codes.Various neural architectures, such as transformers, recurrent neural networks and convolutional neural networks, are tried.For a few codes, anomalous behaviour is observed in terms of which curriculum works the best based on what architecture is used.The work explicated in this thesis would not have been possible without the guidance of my adviser, Prof. Pramod Viswanath and close collaborators
Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5th generation wireless standards (5G). However, there remains room for the design of polar decoders that are both efficient and reliable in the short blocklength regime. Motivated by recent successes of data-driven channel decoders, we introduce a novel C\textbf{C}CurRI\textbf{RI}RIculum based S\textbf{S}Sequential neural decoder for P\textbf{P}Polar codes (CRISP). We design a principled curriculum, guided by information-theoretic insights, to train CRISP and show that it outperforms the successive-cancellation (SC) decoder and attains near-optimal reliability performance on the Polar(32,16) and Polar(64,22) codes. The choice of the proposed curriculum is critical in achieving the accuracy gains of CRISP, as we show by comparing against other curricula. More notably, CRISP can be readily extended to Polarization-Adjusted-Convolutional (PAC) codes, where existing SC decoders are significantly less reliable. To the best of our knowledge, CRISP constructs the first data-driven decoder for PAC codes and attains near-optimal performance on the PAC(32,16) code.
Polar code is the first type of constructive coding with achievable capacity, which has become the encoding scheme for 5G mobile communication systems and is a cutting-edge direction in channel coding research. However, traditional decoding algorithms for Polar codes still need to be improved in terms of errorcorrection performance and decoding delay. For Polar code short codes, due to the phenomenon of incomplete Polar, the traditional SC decoding algorithm has a problem of poor decoding performance. Algorithms that utilize deep learning for direct or auxiliary operations often yield more accurate computational results with lower computational complexity. Therefore, considering that applying deep learning to the communication field is a mainstream trend in the 6G era, a (16,8) short Polar code decoder based on recurrent neural network (RNN) is proposed. At the same time, a suitable dataset construction method was proposed, which achieved better performance than SC decoding algorithm under low signal-to-noise ratio by adjusting network parameters and training under the appropriate dataset.
Codes are the backbone of modern information age.Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication.Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough.Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades.On the other hand, designing codes is a challenging task, mostly driven by human ingenuity.Befittingly, historically, the progress in discovery of codes has been sporadic.In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning.Our major result is the invention of KO codes, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime.The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the Kronecker Operation (KO) central to RM and polar codes.These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes.This design technique can be viewed as an instantiation of the classical neural augmentation principle.In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle.We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades.B.1 Comparison of SGD on our losses (L 4 , L log ) vs. ℓ 2 and the EM algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .106 C.2 KO code achieves a significant gain over the Polar(64, 7) code in BLER when trained on AWGN channel.KO decoder also matches the optimal MAP decoder. . . . . . . . . . . . .108 xii C.6 KO(6, 1) code.Left: KO(6, 1) code achieves significant gain over RM(6, 1) code (with Dumer) when trained on AWGN channel.Right: Under the optimal MAP decoding, KO(6, 1) and RM(6, 1) codes achieve the same performance.Error rates for a random Gaussian codebook are also plotted as a baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . .120 C.1 Plotkin trees for the Polar(64, 7) encoder and our neural KO encoder counterpart.Both codes have dimension k = 7 and blocklength n = 64. . . . . . . . . . . . . . . . . . . . . .121 C.3 The same KO(8,2) encoder and decoder as those used in Figure 4.4 achieve a significant gain (without any retraining or fine-tuning) when list decoding is used together with the KO decoder.The magnitude of the gain is comparable to the gain achieved by the same list decoding technique on the successive cancellation decoder of the RM(8,2) code.We used the list decoding from [1] but without the permutation technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122 C.4 Plotkin trees for RM(8,2) and KO(8,2) encoders.Leaves are highlighted in green.Both codes have dimension k = 37 and blocklength n = 256. . . . . . . . . . . . . . . . . . . . . .123 C.5 Plotkin trees for the RM(8, 2) and KO(8, 2) decoders.Red arrows indicate the bit decoding order. . . . . . . .
Several researchers are interested in polar codes for the sake of their capacity which nearly catches memoryless channels capacity. They have been used as part of the 5G technology. Polar decoders like belief propagation decoders showed unfavorable performance during limited iterations. Nowadays, Machine-learning (ML) techniques are used to enhance the efficiency of polar codes decoders. Recurrent neural network belief propagation (RNN-BP) decoders proved better efficiency in a lower number of cycles. In this paper, a performance scheduling learning technique is proposed. This technique achieves a coding gain of 0.2 dB for the RNN-BP decoder at BER of 2*10−4. A new approach is also introduced in the quantization step. In this approach, the quantization is done after the learning process is finished instead of after each iteration. This approach fixes the issue of model convergence. These combined approaches show up to 0.45 dB improvement at BER of 2*10−4.
Deep learning recently shows strong competitiveness to improve polar code decoding. However, suffering from prohibitive training and computation complexity, the conventional deep neural network (DNN) is only possible for very short code length. In this paper, the main problems of deep learning in decoding are well solved. We first present the multiple scaled belief propagation (BP) algorithm, aiming at obtaining faster convergence and better performance. Based on this, deep neural network decoder (NND) with low complexity and latency, is proposed for any code length. The training only requires a small set of zero codewords. Besides, its computation complexity is close to the original BP. Experiment results show that the proposed (64,32) NND with 5 iterations achieves even lower bit error rate (BER) than the 30-iteration conventional BP and (512, 256) NND also outperforms conventional BP decoder with same iterations. The hardware architecture of basic computation block is given and folding technique is also considered, saving about 50% hardware cost.
The training complexity of deep learning-based channel decoders scales exponentially with the codebook size and therefore with the number of information bits. Thus, neural network decoding (NND) is currently only feasible for very short block lengths. In this work, we show that the conventional iterative decoding algorithm for polar codes can be enhanced when sub-blocks of the decoder are replaced by neural network (NN) based components. Thus, we partition the encoding graph into smaller sub-blocks and train them individually, closely approaching maximum a posteriori (MAP) performance per sub-block. These blocks are then connected via the remaining conventional belief propagation decoding stage(s). The resulting decoding algorithm is non-iterative and inherently enables a highlevel of parallelization, while showing a competitive bit error rate (BER) performance. We examine the degradation through partitioning and compare the resulting decoder to state-of-the art polar decoders such as successive cancellation list and belief propagation decoding.