HiKO: A Hierarchical Framework for Beyond-Second-Order KO Codes

TL;DR

HiKO introduces a hierarchical training framework for neural error-correcting codes, enabling KO codes to surpass Reed-Muller codes at higher orders, thus advancing high-rate neural coding techniques.

Contribution

This work is the first to extend KO codes beyond second order using a hierarchical training approach with novel neural architectures and training strategies.

Findings

HiKO codes outperform Reed-Muller codes at third and fourth order.

HiKO codes approximate Shannon-optimal Gaussian codebooks.

The framework enables effective high-rate neural error-correcting codes.

Abstract

This paper introduces HiKO (Hierarchical Kronecker Operation), a novel framework for training high-rate neural error-correcting codes that enables KO codes to outperform Reed-Muller codes beyond second order. To our knowledge, this is the first attempt to extend KO codes beyond second order. While conventional KO codes show promising results for low-rate regimes ($r < 2$), they degrade at higher rates -- a critical limitation for practical deployment. Our framework incorporates three key innovations: (1) a hierarchical training methodology that decomposes complex high-rate codes into simpler constituent codes for efficient knowledge transfer, (2) enhanced neural architectures with dropout regularization and learnable skip connections tailored for the Plotkin structure, and (3) a progressive unfreezing strategy that systematically transitions from pre-trained components to fully optimized integrated codes. Our experiments show that HiKO codes consistently outperform traditional Reed-Muller codes across various configurations, achieving notable performance improvements for third-order ($r = 3$) and fourth-order ($r = 4$) codes. Analysis reveals that HiKO codes successfully approximate Shannon-optimal Gaussian codebooks while preserving efficient decoding properties. This represents the first successful extension of KO codes beyond second order, opening new possibilities for neural code deployment in high-throughput communication systems.