Learning Binary Autoencoder-Based Codes with Progressive Training

TL;DR

This paper introduces a two-stage training method for autoencoders to learn binary error-correcting codes, achieving optimal code properties and error rates without complex gradient approximation techniques.

Contribution

A simple two-stage training process enables autoencoders to learn structured binary codes with optimal properties, improving stability and performance over previous methods.

Findings

Learned codes match the performance of optimal Hamming codes.

The approach naturally recovers linear and distance properties of codes.

Achieves the same block error rate as maximum likelihood decoding.

Abstract

Error correcting codes play a central role in digital communication, ensuring that transmitted information can be accurately reconstructed despite channel impairments. Recently, autoencoder (AE) based approaches have gained attention for the end-to-end design of communication systems, offering a data driven alternative to conventional coding schemes. However, enforcing binary codewords within differentiable AE architectures remains difficult, as discretization breaks gradient flow and often leads to unstable convergence. To overcome this limitation, a simplified two stage training procedure is proposed, consisting of a continuous pretraining phase followed by direct binarization and fine tuning without gradient approximation techniques. For the (7,4) block configuration over a binary symmetric channel (BSC), the learned encoder-decoder pair learns a rotated version (coset code) of the optimal Hamming code, naturally recovering its linear and distance properties and thereby achieving the same block error rate (BLER) with maximum likelihood (ML) decoding. These results indicate that compact AE architectures can effectively learn structured, algebraically optimal binary codes through stable and straightforward training.