Capacity of Finite State Channels with Feedback: Algorithmic and Optimization Theoretic Properties

TL;DR

This paper investigates the fundamental algorithmic limits of computing the feedback capacity of finite state channels, showing it is not computable and cannot be expressed as a finite-letter formula, highlighting intrinsic complexity.

Contribution

It proves that the feedback capacity of finite state channels is not Banach-Mazur computable and cannot be characterized by a finite-letter entropic formula.

Findings

Feedback capacity is not Banach-Mazur computable.

No finite-letter formula can characterize the feedback capacity.

Achievability or converse bounds are not always computable.

Abstract

The capacity of finite state channels (FSCs) with feedback has been shown to be a limit of a sequence of multi-letter expressions. Despite many efforts, a closed-form single-letter capacity characterization is unknown to date. In this paper, the feedback capacity is studied from a fundamental algorithmic point of view by addressing the question of whether or not the capacity can be algorithmically computed. To this aim, the concept of Turing machines is used, which provides fundamental performance limits of digital computers. It is shown that the feedback capacity of FSCs is not Banach-Mazur computable and therefore not Borel-Turing computable. As a consequence, it is shown that either achievability or converse is not Banach-Mazur computable, which means that there are computable FSCs for which it is impossible to find computable tight upper and lower bounds. Furthermore, it is shown that the feedback capacity cannot be characterized as the maximization of a finite-letter formula of entropic quantities.