Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Deep code: representation and learning algorithms for neural networks & their applications to communication codes

Ashok Vardhan Makkuva

Published Jul 15, 2022• 0 citations
OpenAlex

Abstract

Codes are the backbone of modern information age.Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication.Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough.Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades.On the other hand, designing codes is a challenging task, mostly driven by human ingenuity.Befittingly, historically, the progress in discovery of codes has been sporadic.In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning.Our major result is the invention of KO codes, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime.The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the Kronecker Operation (KO) central to RM and polar codes.These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes.This design technique can be viewed as an instantiation of the classical neural augmentation principle.In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle.We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades.B.1 Comparison of SGD on our losses (L 4 , L log ) vs. ℓ 2 and the EM algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .106 C.2 KO code achieves a significant gain over the Polar(64, 7) code in BLER when trained on AWGN channel.KO decoder also matches the optimal MAP decoder. . . . . . . . . . . . .108 xii C.6 KO(6, 1) code.Left: KO(6, 1) code achieves significant gain over RM(6, 1) code (with Dumer) when trained on AWGN channel.Right: Under the optimal MAP decoding, KO(6, 1) and RM(6, 1) codes achieve the same performance.Error rates for a random Gaussian codebook are also plotted as a baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . .120 C.1 Plotkin trees for the Polar(64, 7) encoder and our neural KO encoder counterpart.Both codes have dimension k = 7 and blocklength n = 64. . . . . . . . . . . . . . . . . . . . . .121 C.3 The same KO(8,2) encoder and decoder as those used in Figure 4.4 achieve a significant gain (without any retraining or fine-tuning) when list decoding is used together with the KO decoder.The magnitude of the gain is comparable to the gain achieved by the same list decoding technique on the successive cancellation decoder of the RM(8,2) code.We used the list decoding from [1] but without the permutation technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122 C.4 Plotkin trees for RM(8,2) and KO(8,2) encoders.Leaves are highlighted in green.Both codes have dimension k = 37 and blocklength n = 256. . . . . . . . . . . . . . . . . . . . . .123 C.5 Plotkin trees for the RM(8, 2) and KO(8, 2) decoders.Red arrows indicate the bit decoding order. . . . . . . .

Related papers

DeepPolar: Inventing Nonlinear Large-Kernel Polar Codes via Deep Learning

S. Ashwin Hebbar, Sravan Kumar Ankireddy, Hyeji Kim +2 more

Progress in designing channel codes has been driven by human ingenuity and, fittingly, has been sporadic. Polar codes, developed on the foundation of Arikan's polarization kernel, represent the latest breakthrough in coding theory and have emerged as the state-of-the-art error-correction code for short-to-medium block length regimes. In an effort to automate the invention of good channel codes, especially in this regime, we explore a novel, non-linear generalization of Polar codes, which we call DeepPolar codes. DeepPolar codes extend the conventional Polar coding framework by utilizing a larger kernel size and parameterizing these kernels and matched decoders through neural networks. Our results demonstrate that these data-driven codes effectively leverage the benefits of a larger kernel size, resulting in enhanced reliability when compared to both existing neural codes and conventional Polar codes.

arXiv:2402.08864•Feb 14, 2024•11 citations

Interpreting KO Codes

R. Shekhar, Natasha Devroye, Gy. Turán +1 more

The KO (Kronecker Operation) code is a recent deep-learned error-correcting code using a neural network architecture to generalize a Reed-Muller code with Dumer decoding. Analyzing the encoder modules and using ablation techniques, we give interpretations of the KO encoder which significantly reduce the number of parameters. We also discuss interpretability aspects of the KO decoder. The interpretation opens up possibilities to give an explicit representation of KO codes, which could be useful for more efficient learning of KO codes and explaining the learning mechanism underlying the empirical observations made about its performance in previous work.

IEEE International Symposium on Information Theory (ISIT)•Jun 22, 2025•0 citations

Deep Learning for Channel Coding

Kou Tian

Next-generation communication systems face challenges in meeting the growing demand of ultra-reliable low-latency communications (URLLC), which require advanced physical-layer designs, especially in channel coding. To satisfy the strict reliability and latency constraints of URLLC in 5G and emerging 6G networks, it is essential to use short codes with robust error-correction capabilities. This thesis explores the application of deep learning methodologies to develop advanced channel coding frameworks for short linear block codes. First, I propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, algorithmically aligning with conventional belief propagation (BP) decoding. The decoder is characterized by scalability, with trainable parameters independent of codeword length, and outperforms BP and existing deep-learning-based methods. I present a novel auto-encoder integrating deep reinforcement learning (DRL) and graph neural networks (GNN) for end-to-end channel coding. Code generation is modeled as a Markov decision process to optimize error rates and algebraic properties. An iterative joint training of the DRL-based code designer and the EW-GNN decoder is performed to optimise the end-to-end encoding and decoding process. Simulation results show significant improvements over traditional coding schemes including LDPC and BCH codes. Finally, I design a supervised learning framework using an integrated auto-encoder architecture with a systematic code generator and matrix-based adaptive double-weighted (MA-DW) decoder. This approach achieves superior performance compared to conventional systems while reducing training complexity through joint optimization of code design and decoding parameters.

•Jan 1, 2025•0 citations

Scaling Deep Learning-Based Decoding of Polar Codes via Partitioning

Sebastian Cammerer, Tobias Gruber, Jakob Hoydis +1 more

The training complexity of deep learning-based channel decoders scales exponentially with the codebook size and therefore with the number of information bits. Thus, neural network decoding (NND) is currently only feasible for very short block lengths. In this work, we show that the conventional iterative decoding algorithm for polar codes can be enhanced when sub-blocks of the decoder are replaced by neural network (NN) based components. Thus, we partition the encoding graph into smaller sub-blocks and train them individually, closely approaching maximum a posteriori (MAP) performance per sub-block. These blocks are then connected via the remaining conventional belief propagation decoding stage(s). The resulting decoding algorithm is non-iterative and inherently enables a highlevel of parallelization, while showing a competitive bit error rate (BER) performance. We examine the degradation through partitioning and compare the resulting decoder to state-of-the art polar decoders such as successive cancellation list and belief propagation decoding.

Globecom•Dec 1, 2017•208 citations

ProductAE: Toward Deep Learning Driven Error-Correction Codes of Large Dimensions

Mohammad Vahid Jamali, Hamid Saber, Homayoon Hatami +1 more

While decades of theoretical research have led to the invention of several classes of error-correction codes, the design of such codes is an extremely challenging task, mostly driven by human ingenuity. Recent studies demonstrate that such designs can be effectively automated and accelerated via tools from machine learning (ML), thus enabling ML-driven classes of error-correction codes with promising performance gains compared to classical designs. A fundamental challenge, however, is that it is prohibitively complex, if not impossible, to design and train fully ML-driven encoder and decoder pairs for large code dimensions. In this paper, we propose Product Autoencoder (ProductAE) -- a computationally-efficient family of deep learning driven (encoder, decoder) pairs -- aimed at enabling the training of relatively large codes (both encoder and decoder) with a manageable training complexity. We build upon ideas from classical product codes and propose constructing large neural codes using smaller code components. ProductAE boils down the complex problem of training the encoder and decoder for a large code dimension $k$ and blocklength $n$ to less-complex sub-problems of training encoders and decoders for smaller dimensions and blocklengths. Our training results show successful training of ProductAEs of dimensions as large as $k = 300$ bits with meaningful performance gains compared to state-of-the-art classical and neural designs. Moreover, we demonstrate excellent robustness and adaptivity of ProductAEs to channel models different than the ones used for training.

arXiv:2303.16424•Mar 29, 2023•5 citations

Curriculum learning for polar and PAC decoders

Viraj Nadkarni

Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5 th generation wireless standards (5G).Polar-Adjusted Convolutional (PAC) codes are a recent modification to Polar codes that provide better reliability even at shorter block lengths.Training efficient neural decoders for both kinds of codes proves challenging at longer block lengths or smaller model sizes.We show that the technique of Curriculum learning is useful in training such models.We also show that particular kinds of curricula work better than others in training the network and also explain reasons why this is the case using the structure of the encoding procedure in Polar codes.Various neural architectures, such as transformers, recurrent neural networks and convolutional neural networks, are tried.For a few codes, anomalous behaviour is observed in terms of which curriculum works the best based on what architecture is used.The work explicated in this thesis would not have been possible without the guidance of my adviser, Prof. Pramod Viswanath and close collaborators

•Jul 19, 2022•0 citations
Codes are the backbone of modern information age.Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication.Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough.Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades.On the other hand, designing codes is a challenging task, mostly driven by human ingenuity.Befittingly, historically, the progress in discovery of codes has been sporadic.In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning.Our major result is the invention of KO codes, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime.The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the Kronecker Operation (KO) central to RM and polar codes.These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes.This design technique can be viewed as an instantiation of the classical neural augmentation principle.In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle.We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades.B.1 Comparison of SGD on our losses (L 4 , L log ) vs. ℓ 2 and the EM algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .106 C.2 KO code achieves a significant gain over the Polar(64, 7) code in BLER when trained on AWGN channel.KO decoder also matches the optimal MAP decoder. . . . . . . . . . . . .108 xii C.6 KO(6, 1) code.Left: KO(6, 1) code achieves significant gain over RM(6, 1) code (with Dumer) when trained on AWGN channel.Right: Under the optimal MAP decoding, KO(6, 1) and RM(6, 1) codes achieve the same performance.Error rates for a random Gaussian codebook are also plotted as a baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . .120 C.1 Plotkin trees for the Polar(64, 7) encoder and our neural KO encoder counterpart.Both codes have dimension k = 7 and blocklength n = 64. . . . . . . . . . . . . . . . . . . . . .121 C.3 The same KO(8,2) encoder and decoder as those used in Figure 4.4 achieve a significant gain (without any retraining or fine-tuning) when list decoding is used together with the KO decoder.The magnitude of the gain is comparable to the gain achieved by the same list decoding technique on the successive cancellation decoder of the RM(8,2) code.We used the list decoding from [1] but without the permutation technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122 C.4 Plotkin trees for RM(8,2) and KO(8,2) encoders.Leaves are highlighted in green.Both codes have dimension k = 37 and blocklength n = 256. . . . . . . . . . . . . . . . . . . . . .123 C.5 Plotkin trees for the RM(8, 2) and KO(8, 2) decoders.Red arrows indicate the bit decoding order. . . . . . . .
Progress in designing channel codes has been driven by human ingenuity and, fittingly, has been sporadic. Polar codes, developed on the foundation of Arikan's polarization kernel, represent the latest breakthrough in coding theory and have emerged as the state-of-the-art error-correction code for short-to-medium block length regimes. In an effort to automate the invention of good channel codes, especially in this regime, we explore a novel, non-linear generalization of Polar codes, which we call DeepPolar codes. DeepPolar codes extend the conventional Polar coding framework by utilizing a larger kernel size and parameterizing these kernels and matched decoders through neural networks. Our results demonstrate that these data-driven codes effectively leverage the benefits of a larger kernel size, resulting in enhanced reliability when compared to both existing neural codes and conventional Polar codes.
The KO (Kronecker Operation) code is a recent deep-learned error-correcting code using a neural network architecture to generalize a Reed-Muller code with Dumer decoding. Analyzing the encoder modules and using ablation techniques, we give interpretations of the KO encoder which significantly reduce the number of parameters. We also discuss interpretability aspects of the KO decoder. The interpretation opens up possibilities to give an explicit representation of KO codes, which could be useful for more efficient learning of KO codes and explaining the learning mechanism underlying the empirical observations made about its performance in previous work.
Next-generation communication systems face challenges in meeting the growing demand of ultra-reliable low-latency communications (URLLC), which require advanced physical-layer designs, especially in channel coding. To satisfy the strict reliability and latency constraints of URLLC in 5G and emerging 6G networks, it is essential to use short codes with robust error-correction capabilities. This thesis explores the application of deep learning methodologies to develop advanced channel coding frameworks for short linear block codes. First, I propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, algorithmically aligning with conventional belief propagation (BP) decoding. The decoder is characterized by scalability, with trainable parameters independent of codeword length, and outperforms BP and existing deep-learning-based methods. I present a novel auto-encoder integrating deep reinforcement learning (DRL) and graph neural networks (GNN) for end-to-end channel coding. Code generation is modeled as a Markov decision process to optimize error rates and algebraic properties. An iterative joint training of the DRL-based code designer and the EW-GNN decoder is performed to optimise the end-to-end encoding and decoding process. Simulation results show significant improvements over traditional coding schemes including LDPC and BCH codes. Finally, I design a supervised learning framework using an integrated auto-encoder architecture with a systematic code generator and matrix-based adaptive double-weighted (MA-DW) decoder. This approach achieves superior performance compared to conventional systems while reducing training complexity through joint optimization of code design and decoding parameters.
The training complexity of deep learning-based channel decoders scales exponentially with the codebook size and therefore with the number of information bits. Thus, neural network decoding (NND) is currently only feasible for very short block lengths. In this work, we show that the conventional iterative decoding algorithm for polar codes can be enhanced when sub-blocks of the decoder are replaced by neural network (NN) based components. Thus, we partition the encoding graph into smaller sub-blocks and train them individually, closely approaching maximum a posteriori (MAP) performance per sub-block. These blocks are then connected via the remaining conventional belief propagation decoding stage(s). The resulting decoding algorithm is non-iterative and inherently enables a highlevel of parallelization, while showing a competitive bit error rate (BER) performance. We examine the degradation through partitioning and compare the resulting decoder to state-of-the art polar decoders such as successive cancellation list and belief propagation decoding.
While decades of theoretical research have led to the invention of several classes of error-correction codes, the design of such codes is an extremely challenging task, mostly driven by human ingenuity. Recent studies demonstrate that such designs can be effectively automated and accelerated via tools from machine learning (ML), thus enabling ML-driven classes of error-correction codes with promising performance gains compared to classical designs. A fundamental challenge, however, is that it is prohibitively complex, if not impossible, to design and train fully ML-driven encoder and decoder pairs for large code dimensions. In this paper, we propose Product Autoencoder (ProductAE) -- a computationally-efficient family of deep learning driven (encoder, decoder) pairs -- aimed at enabling the training of relatively large codes (both encoder and decoder) with a manageable training complexity. We build upon ideas from classical product codes and propose constructing large neural codes using smaller code components. ProductAE boils down the complex problem of training the encoder and decoder for a large code dimension kkk and blocklength nnn to less-complex sub-problems of training encoders and decoders for smaller dimensions and blocklengths. Our training results show successful training of ProductAEs of dimensions as large as k=300k = 300k=300 bits with meaningful performance gains compared to state-of-the-art classical and neural designs. Moreover, we demonstrate excellent robustness and adaptivity of ProductAEs to channel models different than the ones used for training.
Polar codes are widely used state-of-the-art codes for reliable communication that have recently been included in the 5 th generation wireless standards (5G).Polar-Adjusted Convolutional (PAC) codes are a recent modification to Polar codes that provide better reliability even at shorter block lengths.Training efficient neural decoders for both kinds of codes proves challenging at longer block lengths or smaller model sizes.We show that the technique of Curriculum learning is useful in training such models.We also show that particular kinds of curricula work better than others in training the network and also explain reasons why this is the case using the structure of the encoding procedure in Polar codes.Various neural architectures, such as transformers, recurrent neural networks and convolutional neural networks, are tried.For a few codes, anomalous behaviour is observed in terms of which curriculum works the best based on what architecture is used.The work explicated in this thesis would not have been possible without the guidance of my adviser, Prof. Pramod Viswanath and close collaborators