Comment on Is Complexity an Illusion?

TL;DR

This paper challenges the notion of complexity by formally analyzing policies, demonstrating that correct policies cannot exist for a simple classification task, and discusses implications for the theory of complexity.

Contribution

It introduces a formal framework for policies and proves the non-existence of correct policies in a basic classification scenario, questioning assumptions about complexity.

Findings

Correct policies do not exist for the simple classification task

Mathematical proof and exhaustive search support the non-existence

Implications challenge existing views on complexity

Abstract

The paper "Is Complexity an Illusion?" (Bennett, 2024) provides a formalism for complexity, learning, inference, and generalization, and introduces a formal definition for a "policy". This reply shows that correct policies do not exist for a simple task of supervised multi-class classification, via mathematical proof and exhaustive search. Implications of this result are discussed, as well as possible responses and amendments to the theory.