Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE International Symposium on Information Theory (ISIT)

ProductMPT: Message-Passing Transformer for Product Codes

Jinyuan Li, Shuai Xiao, Weijun Fang

Published Jun 1, 2026• 0 citations
Publisher

Abstract

Deep learning has shown significant promise in neural decoding, and Transformer-based decoders have achieved competitive performance across various code families. However, applying these decoders to product codes remains challenging, as they often treat the code as a generic linear block code, failing to exploit the intrinsic row-column algebraic constraints. This paper proposes the Product Message-Passing Transformer (ProductMPT), a novel architecture that transforms the inherent row-column parity-check structure of product codes into a structured neural message-passing mechanism. To facilitate this mechanism, we introduce a unified syndrome that explicitly couples row-wise and column-wise parity constraints. Extensive experiments on various product codes demonstrate that ProductMPT achieves substantial bit error rate (BER) improvements, typically by about an order of magnitude in the moderate-to-high signal-to-noise ratio (SNR) regime, while significantly reducing computational complexity, inference latency, and memory usage compared with state-of-the-art Transformer-based decoders.

DOI
10.1109/isit62367.2026.11653848

Related papers

CrossMPT: Cross-attention Message-Passing Transformer for Error Correcting Codes

Seong‐Joon Park, Kwak, Hee-Youl, Sang‐Hyo Kim +2 more

Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ self-attention mechanism without distinguishing between the types of input vectors (i.e., magnitude and syndrome vectors), CrossMPT updates the two types of input vectors separately and iteratively using two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures the irrelevant relationship between two input vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.

arXiv:2405.01033•May 2, 2024•14 citations

E2E DC-CrossMPT: Cross-Attention Message-Passing Transformer for Joint Design and Decoding of Linear Block Codes

Yeji Cho, Junghyun Kim

In this paper, we propose a novel deep learning-based framework for the joint design and decoding of linear block codes, the end-to-end deep coding cross-attention message-passing Transformer (E2E DC-CrossMPT). To improve linear block code design and decoding, we redesign the conventional error correction code (ECC) decoder, CrossMPT, to fit within an end-to-end framework. The redesigned decoder separately utilizes magnitude and syndrome vectors obtained from the received signals as inputs. It further employs one-hot encoding based syndrome embedding and incorporates a parity-check matrix into the output layers. Experimental results demonstrate that, across various code lengths and code rates, E2E DC-CrossMPT consistently outperforms both traditional decoding algorithms and a conventional end-to-end deep coding model in terms of decoding performance. Moreover, the codes designed by E2E DC-CrossMPT achieve superior error-correction capability compared with both traditional linear block codes and those designed by the conventional end-to-end deep coding model, while requiring lower computational complexity.

Electronics•Jun 25, 2026•1 citation

Unified Product GNN Decoder for Short Codes: Systematic Binary and Non-Binary Approaches

Jingjing Guo, Chentao Yue, Xuewan Zhang +2 more

Ultra-reliable and low-latency communications (URLLC) in 6 G networks pose significant challenges for channel coding, particularly where both high reliability and low latency are crucial. Deep learning–based decoders have shown promise for short codes, yet their complexity grows with code length, and their error-correction performance remains limited. This paper introduces a product graph neural network (ProductGNN) decoder that employs graph neural networks (GNN) as component decoders to enhance iterative decoding within the product code framework. The proposed unified framework supports both systematic binary product codes and non-binary product analog fountain codes (AFC), with tailored soft information exchange mechanisms between row and column decoders. The architecture maintains a parameter count independent of code length. Extrinsic information transfer (EXIT) chart analysis is conducted for ProductGNN to characterize the mutual information transfer between row and column decoders, revealing faster convergence compared to belief propagation (BP)-based iterative decoding. Numerical results demonstrate that ProductGNN consistently outperforms BP and standalone GNN decoders across multiple code families, achieving up to 2 dB gain for high-density codes affected by short cycles and up to 0.8 dB gain for non-binary AFC codes at a bit error rate (BER) of$10^{-4}$.

IEEE Transactions on Vehicular Technology•Jan 1, 2026•0 citations

Denoising Diffusion Decoding with U-Shaped Error Correction Code Transformers

Elaheh Bassak, Elvino Silveira Sousa

Recent neural decoders substantially improve error correction for short block codes but remain computationally intensive and lack adaptive refinement. This work introduces enhanced U-shaped transformer architectures with two key innovations: (i) double-masked attention from complementary parity-check matrices for improved structural diversity, and (ii) cross-attention message-passing that bridges neural and classical decoding principles. To enable adaptive iterative refinement, we extend these single-shot U-shaped decoders into the denoising diffusion paradigm, where channel corruption is modeled as a forward diffusion process and iteratively reversed based on continuous soft syndrome conditioning. Our key finding demonstrates that U-shaped backbones provide disproportionately larger performance gains under diffusion compared to shallow or deep alternatives, nearly matching deep architectures while utilizing half the parameters. Extensive experiments on LDPC(49,24) and Polar(64,32) codes show that the proposed diffusion-enhanced U-shaped transformers achieve state-of-the-art bit-error-rate performance while achieving superior computational efficiency.

IEEE International Conference on Communications workshops•May 24, 2026•0 citations

Cross-Attention Message-Passing Transformers for Code-Agnostic Decoding in 6G Networks

Seong‐Joon Park, Kwak, Hee-Youl, Sang‐Hyo Kim +2 more

Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.

arXiv:2507.01038•Jun 22, 2025•1 citation

Section-Wise Revolving NBP-Like Decoders for QC-LDPC Codes

Qinshan Zhang, Bin Chen, Tianqu Zhuang +2 more

Recently, deep learning has demonstrated improvements over classical decoding algorithms in various families of error-correcting codes with short to moderate block lengths. Neural decoders like neural belief propagation (NBP), constructed based on the Tanner graphs of linear codes, have been extensively studied, but their application to longer codes is limited due to increased network complexity with longer block lengths. This complexity leads to higher computational costs and deployment challenges, such as GPU memory usage. To address this, we propose a novel revolving framework for NBP-like decoders tailored to QC-LDPC codes, a commonly used class of error correction codes. Our approach leverages the section-wise cyclic structure inherent in QC-LDPC codes, significantly simplifying the complexity of the network. Experimental results demonstrate the effectiveness of our method across various QC-LDPC codes. Especially, compared to the traditional decoding scheme based on the section-wise cyclic structure, our proposed decoder shows substantial improvements on 5G LDPC codes, exhibits better performance than the traditional min-sum decoder, and approaches the sum-product decoder without a noticeable error floor within the investigated noise levels.

IEEE International Symposium on Information Theory (ISIT)•Jul 7, 2024•4 citations
Deep learning has shown significant promise in neural decoding, and Transformer-based decoders have achieved competitive performance across various code families. However, applying these decoders to product codes remains challenging, as they often treat the code as a generic linear block code, failing to exploit the intrinsic row-column algebraic constraints. This paper proposes the Product Message-Passing Transformer (ProductMPT), a novel architecture that transforms the inherent row-column parity-check structure of product codes into a structured neural message-passing mechanism. To facilitate this mechanism, we introduce a unified syndrome that explicitly couples row-wise and column-wise parity constraints. Extensive experiments on various product codes demonstrate that ProductMPT achieves substantial bit error rate (BER) improvements, typically by about an order of magnitude in the moderate-to-high signal-to-noise ratio (SNR) regime, while significantly reducing computational complexity, inference latency, and memory usage compared with state-of-the-art Transformer-based decoders.
Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ self-attention mechanism without distinguishing between the types of input vectors (i.e., magnitude and syndrome vectors), CrossMPT updates the two types of input vectors separately and iteratively using two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures the irrelevant relationship between two input vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.
In this paper, we propose a novel deep learning-based framework for the joint design and decoding of linear block codes, the end-to-end deep coding cross-attention message-passing Transformer (E2E DC-CrossMPT). To improve linear block code design and decoding, we redesign the conventional error correction code (ECC) decoder, CrossMPT, to fit within an end-to-end framework. The redesigned decoder separately utilizes magnitude and syndrome vectors obtained from the received signals as inputs. It further employs one-hot encoding based syndrome embedding and incorporates a parity-check matrix into the output layers. Experimental results demonstrate that, across various code lengths and code rates, E2E DC-CrossMPT consistently outperforms both traditional decoding algorithms and a conventional end-to-end deep coding model in terms of decoding performance. Moreover, the codes designed by E2E DC-CrossMPT achieve superior error-correction capability compared with both traditional linear block codes and those designed by the conventional end-to-end deep coding model, while requiring lower computational complexity.
Ultra-reliable and low-latency communications (URLLC) in 6 G networks pose significant challenges for channel coding, particularly where both high reliability and low latency are crucial. Deep learning–based decoders have shown promise for short codes, yet their complexity grows with code length, and their error-correction performance remains limited. This paper introduces a product graph neural network (ProductGNN) decoder that employs graph neural networks (GNN) as component decoders to enhance iterative decoding within the product code framework. The proposed unified framework supports both systematic binary product codes and non-binary product analog fountain codes (AFC), with tailored soft information exchange mechanisms between row and column decoders. The architecture maintains a parameter count independent of code length. Extrinsic information transfer (EXIT) chart analysis is conducted for ProductGNN to characterize the mutual information transfer between row and column decoders, revealing faster convergence compared to belief propagation (BP)-based iterative decoding. Numerical results demonstrate that ProductGNN consistently outperforms BP and standalone GNN decoders across multiple code families, achieving up to 2 dB gain for high-density codes affected by short cycles and up to 0.8 dB gain for non-binary AFC codes at a bit error rate (BER) of10−410^{-4}10−4.
Recent neural decoders substantially improve error correction for short block codes but remain computationally intensive and lack adaptive refinement. This work introduces enhanced U-shaped transformer architectures with two key innovations: (i) double-masked attention from complementary parity-check matrices for improved structural diversity, and (ii) cross-attention message-passing that bridges neural and classical decoding principles. To enable adaptive iterative refinement, we extend these single-shot U-shaped decoders into the denoising diffusion paradigm, where channel corruption is modeled as a forward diffusion process and iteratively reversed based on continuous soft syndrome conditioning. Our key finding demonstrates that U-shaped backbones provide disproportionately larger performance gains under diffusion compared to shallow or deep alternatives, nearly matching deep architectures while utilizing half the parameters. Extensive experiments on LDPC(49,24) and Polar(64,32) codes show that the proposed diffusion-enhanced U-shaped transformers achieve state-of-the-art bit-error-rate performance while achieving superior computational efficiency.
Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.
Recently, deep learning has demonstrated improvements over classical decoding algorithms in various families of error-correcting codes with short to moderate block lengths. Neural decoders like neural belief propagation (NBP), constructed based on the Tanner graphs of linear codes, have been extensively studied, but their application to longer codes is limited due to increased network complexity with longer block lengths. This complexity leads to higher computational costs and deployment challenges, such as GPU memory usage. To address this, we propose a novel revolving framework for NBP-like decoders tailored to QC-LDPC codes, a commonly used class of error correction codes. Our approach leverages the section-wise cyclic structure inherent in QC-LDPC codes, significantly simplifying the complexity of the network. Experimental results demonstrate the effectiveness of our method across various QC-LDPC codes. Especially, compared to the traditional decoding scheme based on the section-wise cyclic structure, our proposed decoder shows substantial improvements on 5G LDPC codes, exhibits better performance than the traditional min-sum decoder, and approaches the sum-product decoder without a noticeable error floor within the investigated noise levels.