Computability of Optimizers

TL;DR

This paper investigates the fundamental limitations of digital computers in solving optimization problems, showing that in many cases, the optimizer itself is non-computable or cannot be approximated, highlighting intrinsic computational barriers.

Contribution

The paper demonstrates that the optimizer in various optimization problems is often non-computable on Turing machines, revealing fundamental limits of digital computation.

Findings

Optimizer is often unattainable on Turing machines

No Turing machine can approximate the optimizer within a constant error

Many problems are not Banach-Mazur computable, even approximately

Abstract

Optimization problems are a staple of today's scientific and technical landscape. However, at present, solvers of such problems are almost exclusively run on digital hardware. Using Turing machines as a mathematical model for any type of digital hardware, in this paper, we analyze fundamental limitations of this conceptual approach of solving optimization problems. Since in most applications, the optimizer itself is of significantly more interest than the optimal value of the corresponding function, we will focus on computability of the optimizer. In fact, we will show that in various situations the optimizer is unattainable on Turing machines and consequently on digital computers. Moreover, even worse, there does not exist a Turing machine, which approximates the optimizer itself up to a certain constant error. We prove such results for a variety of well-known problems from very different areas, including artificial intelligence, financial mathematics, and information theory, often deriving the even stronger result that such problems are not Banach-Mazur computable, also not even in an approximate sense.