Memory effects can make the transmission capability of a communication channel uncomputable

TL;DR

This paper demonstrates that the capacity of channels with memory, even simple finite state channels, can be uncomputable to within a certain precision, highlighting fundamental limits in information theory.

Contribution

It proves that the capacity of channels with memory can be uncomputable to within a specified precision, even for simple finite state channels with limited input, output, and memory.

Findings

Capacity cannot be computed to within 1/5 precision for certain channels with memory.

Uncomputability holds even for simple finite state machine channels with limited input/output.

Memory effects can fundamentally limit the computability of channel capacity.

Abstract

Most communication channels are subjected to noise. One of the goals of Information Theory is to add redundancy in the transmission of information so that the information is transmitted reliably and the amount of information transmitted through the channel is as large as possible. The maximum rate at which reliable transmission is possible is called the capacity. If the channel does not keep memory of its past, the capacity is given by a simple optimization problem and can be efficiently computed. The situation of channels with memory is less clear. Here we show that for channels with memory the capacity cannot be computed to within precision 1/5. Our result holds even if we consider one of the simplest families of such channels -information-stable finite state machine channels-, restrict the input and output of the channel to 4 and 1 bit respectively and allow 6 bits of memory.