G\"odel Incompleteness Theorem for PAC Learnable Theory from the view of complexity measurement

TL;DR

This paper explores the relationship between G"odel's incompleteness theorem and PAC learnability, proposing a complexity measure under an interpreter that reveals limits on AI interpretability and generalization.

Contribution

It introduces a new complexity framework based on interpreters and generalizes G"odel's theorem to show limitations in finding optimal interpreters within PAC learning.

Findings

Some objects are PAC-learnable but lack a suitable interpreter.

Strong algorithms cannot interpret all objects, indicating limits on AI generalization.

A fundamental upper bound exists on the interpretability of strong machine learning models.

Abstract

Different from the view that information is objective reality, this paper adopts the idea that all information needs to be compiled by the interpreter before it can be observed. From the traditional complexity definition, this paper defines the complexity under "the interpreter", which means that heuristically finding the best interpreter is equivalent to using PAC to find the most suitable interpreter. Then we generalize the observation process to the formal system with functors, in which we give concrete proof of the generalized G\"odel incompleteness theorem which indicates that there are some objects that are PAC-learnable, but the best interpreter is not found among the alternative interpreters. A strong enough machine algorithm cannot be interpretable in the face of any object. There are always objects that make a strong enough machine learning algorithm uninterpretable, which puts an upper bound on the generalization ability of strong AI.