Kelly Bets and Single-Letter Codes: Optimal Information Processing in Natural Systems

TL;DR

This paper demonstrates that Kelly betting strategies are mathematically equivalent to optimal single-letter codes, enabling natural systems with limited resources to achieve optimal information processing and growth in changing environments.

Contribution

It establishes a theoretical link between Kelly bets and optimal coding, showing how biological systems can attain information-theoretic optimality with limited computational capacity.

Findings

Kelly bets achieve the rate-distortion bound with equality.

Increasing strategy diversity can enhance performance across channels.

More phenotypes can lead to higher growth rates under certain conditions.

Abstract

In an information-processing investment game, such as the growth of a population of organisms in a changing environment, Kelly betting maximizes the expected log rate of growth. In this paper, we show that Kelly bets are closely related to optimal single-letter codes (i.e., they can achieve the rate-distortion bound with equality). Thus, natural information processing systems with limited computational resources can achieve information-theoretically optimal performance. We show that the rate-distortion tradeoff for an investment game has a simple linear bound, and that the bound is achievable at the point where the corresponding single-letter code is optimal. This interpretation has two interesting consequences. First, we show that increasing the organism's portfolio of potential strategies can lead to optimal performance over a continuous range of channels, even if the strategy portfolio is fixed. Second, we show that increasing an organism's number of phenotypes (i.e., its number of possible behaviours in response to the environment) can lead to higher growth rate, and we give conditions under which this occurs. Examples illustrating the results in simplified biological scenarios are presented.