Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2506.10121

HiKO: A Hierarchical Framework for Beyond-Second-Order KO Codes

Shubham Srivastava, Adrish Banerjee

Published Jun 11, 2025• 0 citations
arXivHTMLPDF

Abstract

This paper introduces HiKO (Hierarchical Kronecker Operation), a novel framework for training high-rate neural error-correcting codes that enables KO codes to outperform Reed-Muller codes beyond second order. To our knowledge, this is the first attempt to extend KO codes beyond second order. While conventional KO codes show promising results for low-rate regimes ($r < 2$), they degrade at higher rates -- a critical limitation for practical deployment. Our framework incorporates three key innovations: (1) a hierarchical training methodology that decomposes complex high-rate codes into simpler constituent codes for efficient knowledge transfer, (2) enhanced neural architectures with dropout regularization and learnable skip connections tailored for the Plotkin structure, and (3) a progressive unfreezing strategy that systematically transitions from pre-trained components to fully optimized integrated codes. Our experiments show that HiKO codes consistently outperform traditional Reed-Muller codes across various configurations, achieving notable performance improvements for third-order ($r = 3$) and fourth-order ($r = 4$) codes. Analysis reveals that HiKO codes successfully approximate Shannon-optimal Gaussian codebooks while preserving efficient decoding properties. This represents the first successful extension of KO codes beyond second order, opening new possibilities for neural code deployment in high-throughput communication systems.

DOI
10.48550/arxiv.2506.10121

Related papers

Interpreting KO Codes

R. Shekhar, Natasha Devroye, Gy. Turán +1 more

The KO (Kronecker Operation) code is a recent deep-learned error-correcting code using a neural network architecture to generalize a Reed-Muller code with Dumer decoding. Analyzing the encoder modules and using ablation techniques, we give interpretations of the KO encoder which significantly reduce the number of parameters. We also discuss interpretability aspects of the KO decoder. The interpretation opens up possibilities to give an explicit representation of KO codes, which could be useful for more efficient learning of KO codes and explaining the learning mechanism underlying the empirical observations made about its performance in previous work.

IEEE International Symposium on Information Theory (ISIT)•Jun 22, 2025•0 citations

Deep code: representation and learning algorithms for neural networks & their applications to communication codes

Ashok Vardhan Makkuva

Codes are the backbone of modern information age.Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication.Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough.Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades.On the other hand, designing codes is a challenging task, mostly driven by human ingenuity.Befittingly, historically, the progress in discovery of codes has been sporadic.In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning.Our major result is the invention of KO codes, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime.The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the Kronecker Operation (KO) central to RM and polar codes.These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes.This design technique can be viewed as an instantiation of the classical neural augmentation principle.In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle.We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades.B.1 Comparison of SGD on our losses (L 4 , L log ) vs. ℓ 2 and the EM algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .106 C.2 KO code achieves a significant gain over the Polar(64, 7) code in BLER when trained on AWGN channel.KO decoder also matches the optimal MAP decoder. . . . . . . . . . . . .108 xii C.6 KO(6, 1) code.Left: KO(6, 1) code achieves significant gain over RM(6, 1) code (with Dumer) when trained on AWGN channel.Right: Under the optimal MAP decoding, KO(6, 1) and RM(6, 1) codes achieve the same performance.Error rates for a random Gaussian codebook are also plotted as a baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . .120 C.1 Plotkin trees for the Polar(64, 7) encoder and our neural KO encoder counterpart.Both codes have dimension k = 7 and blocklength n = 64. . . . . . . . . . . . . . . . . . . . . .121 C.3 The same KO(8,2) encoder and decoder as those used in Figure 4.4 achieve a significant gain (without any retraining or fine-tuning) when list decoding is used together with the KO decoder.The magnitude of the gain is comparable to the gain achieved by the same list decoding technique on the successive cancellation decoder of the RM(8,2) code.We used the list decoding from [1] but without the permutation technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122 C.4 Plotkin trees for RM(8,2) and KO(8,2) encoders.Leaves are highlighted in green.Both codes have dimension k = 37 and blocklength n = 256. . . . . . . . . . . . . . . . . . . . . .123 C.5 Plotkin trees for the RM(8, 2) and KO(8, 2) decoders.Red arrows indicate the bit decoding order. . . . . . . .

•Jul 15, 2022•0 citations

Recursive decoding of Reed-Muller codes

Ilya I. Dumer

New soft- and hard decision decoding algorithms are presented for general Reed-Muller codes $\left\{\genfrac{}{}{0pt}{}{m}{r}\right\} $ of length $2^{m}$ and distance $2^{m-r}$. We use Plotkin $(u,u+v)$ construction and decompose code $\left\{\genfrac{}{}{0pt}{}{m}{r}\right\} $ onto subblocks $u\in\left\{\genfrac{}{}{0pt}{}{m-1}{r}\right\} $ and $v\in\left\{\genfrac{}{}{0pt}{}{m-1}{r-1}\right\} .$ In decoding, we first try to find a subblock $v$ from the better protected code and then proceed with the block $u$. The likelihoods of the received symbols are recalculated in a way similar to belief propagation. Thus, decoding is relegated to the two constituent codes. We repeat this recursion and execute decoding only at the end nodes $\left\{\genfrac{}{}{0pt}{}{j}{1}\right\} $ and $\left\{\genfrac{}{}{0pt}{}{j}{j-1}\right\} $. The overall complexity has low order of $n\log n.$ It is shown that this decoding substantially outperforms other algorithms of polynomial complexity known for RM codes. In particular, for medium and high code rates, the algorithm corrects most error patterns of weight $d\ln d/2.$

arXiv:1703.05303•Mar 14, 2017•7 citations

Optimizing LDPC Decoding with Machine Learning Using Learnable Weights: A Hybrid Framework Integrating Neural Networks with Iterative Message-Passing for Enhanced Error-Correction in 5G Systems

Niklas Savijoki

This thesis investigates the optimization of Low-Density Parity-Check (LDPC) decoding through machine learning techniques, with a specific focus on 5G communication systems. Traditional LDPC decoding algorithms face a critical trade-off: Belief Propagation (BP) offers excellent error-correction performance but with high computational complexity, while Min-Sum (MS) reduces complexity at the cost of degraded performance. To address this limitation, a hybrid neural network architecture is proposed that incorporates learnable normalization and offset weights into the message-passing framework of LDPC decoders. This approach integrates multiplicative (normalization) and additive (offset) weights into the Tanner graph structure, allowing the decoder to adapt to specific channel conditions. A novel weighted binary cross-entropy loss function is developed that prioritizes entire block recovery by assigning different importance to information and parity bits during training. Experimental results demonstrate that ML-assisted decoder consistently outperforms conventional Min-Sum decoding, approaching the error-correction capability of Belief Propagation while maintaining comparable computational efficiency to Min-Sum. Comprehensive evaluations across different weight configurations (scalar and vector-based) reveal that increased parametrization generally improves decoding performance. Analysis of the impact of iteration counts shows that the optimal number of decoding iterations varies between different code configurations with studied Basegraph 1 case requiring 15 iterations for optimal performance while studied Basegraph 2 case benefits significantly from 20 iterations. The proposed ML-assisted LDPC decoder represents a significant advancement for 5G systems, offering improved error correction at lower signal strengths while balancing latency and computational requirements. This research contributes to the ongoing evolution of wireless technology by demonstrating how machine learning can effectively enhance traditional decoding algorithms, providing a practical pathway toward more efficient and adaptive error correction in modern communication systems.

•Jan 1, 2025•0 citations

Multi-Step Structure of Reed-Muller Codes

Junyu Ren, Guanghui Wang, Guiying Yan

Currently, many low-complexity decoding algorithms for Reed-Muller (RM) codes utilize the Plotkin structure of RM codes to decompose them into shorter RM subcodes for decoding. This study reveals a key relationship among RM subcodes, demonstrating that specific subcodes can generate all subcodes. Furthermore, we identify conditions where subcodes are inherently free of 4-cycles and 6-cycles structures. Building upon these insights, we modified existing well-known algorithms for RM codes, achieving similar error-correction performance with significantly lower complexity by requiring fewer subcode decodings.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2026•0 citations

Boosting Learning for LDPC Codes to Improve the Error-Floor Performance

Hee-Youl Kwak, Dae-Young Yun, Yongjune Kim +2 more

Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor .

arXiv:2310.07194•neural information processing systems•Jan 1, 2023•13 citations
This paper introduces HiKO (Hierarchical Kronecker Operation), a novel framework for training high-rate neural error-correcting codes that enables KO codes to outperform Reed-Muller codes beyond second order. To our knowledge, this is the first attempt to extend KO codes beyond second order. While conventional KO codes show promising results for low-rate regimes (r<2r < 2r<2), they degrade at higher rates -- a critical limitation for practical deployment. Our framework incorporates three key innovations: (1) a hierarchical training methodology that decomposes complex high-rate codes into simpler constituent codes for efficient knowledge transfer, (2) enhanced neural architectures with dropout regularization and learnable skip connections tailored for the Plotkin structure, and (3) a progressive unfreezing strategy that systematically transitions from pre-trained components to fully optimized integrated codes. Our experiments show that HiKO codes consistently outperform traditional Reed-Muller codes across various configurations, achieving notable performance improvements for third-order (r=3r = 3r=3) and fourth-order (r=4r = 4r=4) codes. Analysis reveals that HiKO codes successfully approximate Shannon-optimal Gaussian codebooks while preserving efficient decoding properties. This represents the first successful extension of KO codes beyond second order, opening new possibilities for neural code deployment in high-throughput communication systems.
The KO (Kronecker Operation) code is a recent deep-learned error-correcting code using a neural network architecture to generalize a Reed-Muller code with Dumer decoding. Analyzing the encoder modules and using ablation techniques, we give interpretations of the KO encoder which significantly reduce the number of parameters. We also discuss interpretability aspects of the KO decoder. The interpretation opens up possibilities to give an explicit representation of KO codes, which could be useful for more efficient learning of KO codes and explaining the learning mechanism underlying the empirical observations made about its performance in previous work.
Codes are the backbone of modern information age.Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication.Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough.Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades.On the other hand, designing codes is a challenging task, mostly driven by human ingenuity.Befittingly, historically, the progress in discovery of codes has been sporadic.In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning.Our major result is the invention of KO codes, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime.The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the Kronecker Operation (KO) central to RM and polar codes.These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes.This design technique can be viewed as an instantiation of the classical neural augmentation principle.In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle.We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades.B.1 Comparison of SGD on our losses (L 4 , L log ) vs. ℓ 2 and the EM algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .106 C.2 KO code achieves a significant gain over the Polar(64, 7) code in BLER when trained on AWGN channel.KO decoder also matches the optimal MAP decoder. . . . . . . . . . . . .108 xii C.6 KO(6, 1) code.Left: KO(6, 1) code achieves significant gain over RM(6, 1) code (with Dumer) when trained on AWGN channel.Right: Under the optimal MAP decoding, KO(6, 1) and RM(6, 1) codes achieve the same performance.Error rates for a random Gaussian codebook are also plotted as a baseline. . . . . . . . . . . . . . . . . . . . . . . . . . . .120 C.1 Plotkin trees for the Polar(64, 7) encoder and our neural KO encoder counterpart.Both codes have dimension k = 7 and blocklength n = 64. . . . . . . . . . . . . . . . . . . . . .121 C.3 The same KO(8,2) encoder and decoder as those used in Figure 4.4 achieve a significant gain (without any retraining or fine-tuning) when list decoding is used together with the KO decoder.The magnitude of the gain is comparable to the gain achieved by the same list decoding technique on the successive cancellation decoder of the RM(8,2) code.We used the list decoding from [1] but without the permutation technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . .122 C.4 Plotkin trees for RM(8,2) and KO(8,2) encoders.Leaves are highlighted in green.Both codes have dimension k = 37 and blocklength n = 256. . . . . . . . . . . . . . . . . . . . . .123 C.5 Plotkin trees for the RM(8, 2) and KO(8, 2) decoders.Red arrows indicate the bit decoding order. . . . . . . .
New soft- and hard decision decoding algorithms are presented for general Reed-Muller codes $\left\{\genfrac{}{}{0pt}{}{m}{r}\right\} $ of length 2m2^{m}2m and distance 2m−r2^{m-r}2m−r. We use Plotkin (u,u+v)(u,u+v)(u,u+v) construction and decompose code $\left\{\genfrac{}{}{0pt}{}{m}{r}\right\} $ onto subblocks $u\in\left\{\genfrac{}{}{0pt}{}{m-1}{r}\right\} $ and v∈{m−1r−1}.v\in\left\{\genfrac{}{}{0pt}{}{m-1}{r-1}\right\} .v∈{r−1m−1​}. In decoding, we first try to find a subblock vvv from the better protected code and then proceed with the block uuu. The likelihoods of the received symbols are recalculated in a way similar to belief propagation. Thus, decoding is relegated to the two constituent codes. We repeat this recursion and execute decoding only at the end nodes $\left\{\genfrac{}{}{0pt}{}{j}{1}\right\} $ and $\left\{\genfrac{}{}{0pt}{}{j}{j-1}\right\} $. The overall complexity has low order of nlog⁡n.n\log n.nlogn. It is shown that this decoding substantially outperforms other algorithms of polynomial complexity known for RM codes. In particular, for medium and high code rates, the algorithm corrects most error patterns of weight dln⁡d/2.d\ln d/2.dlnd/2.
This thesis investigates the optimization of Low-Density Parity-Check (LDPC) decoding through machine learning techniques, with a specific focus on 5G communication systems. Traditional LDPC decoding algorithms face a critical trade-off: Belief Propagation (BP) offers excellent error-correction performance but with high computational complexity, while Min-Sum (MS) reduces complexity at the cost of degraded performance. To address this limitation, a hybrid neural network architecture is proposed that incorporates learnable normalization and offset weights into the message-passing framework of LDPC decoders. This approach integrates multiplicative (normalization) and additive (offset) weights into the Tanner graph structure, allowing the decoder to adapt to specific channel conditions. A novel weighted binary cross-entropy loss function is developed that prioritizes entire block recovery by assigning different importance to information and parity bits during training. Experimental results demonstrate that ML-assisted decoder consistently outperforms conventional Min-Sum decoding, approaching the error-correction capability of Belief Propagation while maintaining comparable computational efficiency to Min-Sum. Comprehensive evaluations across different weight configurations (scalar and vector-based) reveal that increased parametrization generally improves decoding performance. Analysis of the impact of iteration counts shows that the optimal number of decoding iterations varies between different code configurations with studied Basegraph 1 case requiring 15 iterations for optimal performance while studied Basegraph 2 case benefits significantly from 20 iterations. The proposed ML-assisted LDPC decoder represents a significant advancement for 5G systems, offering improved error correction at lower signal strengths while balancing latency and computational requirements. This research contributes to the ongoing evolution of wireless technology by demonstrating how machine learning can effectively enhance traditional decoding algorithms, providing a practical pathway toward more efficient and adaptive error correction in modern communication systems.
Currently, many low-complexity decoding algorithms for Reed-Muller (RM) codes utilize the Plotkin structure of RM codes to decompose them into shorter RM subcodes for decoding. This study reveals a key relationship among RM subcodes, demonstrating that specific subcodes can generate all subcodes. Furthermore, we identify conditions where subcodes are inherently free of 4-cycles and 6-cycles structures. Building upon these insights, we modified existing well-known algorithms for RM codes, achieving similar error-correction performance with significantly lower complexity by requiring fewer subcode decodings.
Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor .