A Non-Probabilistic Game-Theoretic Information Theory Which Subsumes Probabilistic Channel Coding

TL;DR

This paper introduces a unifying non-probabilistic, game-theoretic framework for information theory that encompasses probabilistic, zero-error, and adversarial channel coding scenarios.

Contribution

It develops a deterministic game model for coding that generalizes classical theorems and relaxes probabilistic assumptions using a nonconvex cone approach.

Findings

Framework subsumes probabilistic and adversarial channel coding theorems.

Models coding as a game between encoder and adversary with insurance payments.

Provides a canonical form for various channel types using a nonconvex cone.

Abstract

Probabilistic settings (e.g., vanishing-error channel coding) and non-probabilistic settings (e.g., zero-error channel coding and adversarial channels) were considered two related but different branches of information theory which do not subsume each other. We propose a unifying non-probabilistic information theory based on game theory and dynamic hedging which subsumes the conventional probabilistic channel coding theorem (vanishing error, with or without feedback) and lossless source coding theorem, as well as adversarial settings. Coding is modelled as a deterministic game between an encoder and an adversary, where the encoder may purchase insurance with a payoff that depends on the channel outputs. Our framework is based on a generalization of the works by Ville, Dawid, Shafer and Vovk on the game-theoretic formulation of probabilistic concepts, by relaxing the convex pricing cone to a nonconvex downward closed cone, which is precisely the relaxation needed to model information transmission. Pricing downward closed cone is a versatile tool for non-probabilistic coding results that can subsume their probabilistic counterparts, and provides a canonical form for probabilistic channels, adversarial channels and arbitrarily varying channels.