The information-theoretic complexity of differentiable functions

TL;DR

This paper introduces the V-complexity, an information-theoretic measure for differentiable functions based on piecewise constant approximations, linking it to compression and applying it to diffusion models.

Contribution

It formalizes a new complexity measure for differentiable functions and connects it to compression algorithms and effective complexity in complex systems.

Findings

V-complexity formalizes intuitions about function simplicity.

V-complexity correlates with compression measures like RLE and LZ77.

Application to diffusion shows V-complexity dynamics in physical models.

Abstract

A measure for the complexity of a differentiable function f(x) on an interval is introduced. It is based on approximations of the function by piecewise constant functions. The measure takes into account the quality of the approximation and the number of intervals in the approximating function. This measure, called the V-complexity of f(x), is shown to formalize some intuitions about the simplicity or complexity of f(x). The V-complexity is then compared to another measure of complexity, namely how compressible an approximation of f(x) is. It is hypothesized that V-complexity is equivalent to the compression measure, in the case of the Run Length Encoding and the Lempel Ziv 77 algorithms. V-complexity can be used as an ingredient in the definition of the Effective Complexity (EC) of a Complex System. When the perceived regularities of such a system are described by a differentiable function on an interval, the EC can be defined as the V-complexity of that function. EC is applied to the model of diffusion of cream in a cup of coffee. The perceived regularity of this model is given by the diffusion equation. The V-complexity of the solution of the equation starts at zero, quickly increases to a maximum and then decreases back to zero as the liquid reaches its equilibrium state. It is shown that this is also the result when a cellular automaton approach and the concept of Apparent Complexity is used.