Information-related complexity: a problem-oriented approach

TL;DR

This paper introduces a general, task-oriented measure of information-related complexity that quantifies the minimal information needed for an agent to achieve a certain success level in various systems, extending existing concepts.

Contribution

It proposes a unified complexity measure applicable to both natural and artificial systems, generalizing statistical complexity and aiding decision-making under uncertainty.

Findings

The complexity measure generalizes statistical complexity for stochastic processes.

It applies to optimization and decision problems, assessing their susceptibility to additional information.

The measure provides a way to estimate the value of extra information in complex tasks.

Abstract

A general notion of information-related complexity applicable to both natural and man-made systems is proposed. The overall approach is to explicitly consider a rational agent performing a certain task with a quantifiable degree of success. The complexity is defined as the minimum (quasi-)quantity of information that's necessary to complete the task to the given extent -- measured by the corresponding loss. The complexity so defined is shown to generalize the existing notion of statistical complexity when the system in question can be described by a discrete-time stochastic process. The proposed definition also applies, in particular, to optimization and decision making problems under uncertainty in which case it gives the agent a useful measure of the problem's "susceptibility" to additional information and allows for an estimation of the potential value of the latter.