Measuring the intelligence of an idealized mechanical knowing agent

TL;DR

This paper introduces a simplified measure of intelligence for idealized mechanical agents based on the supremum of known computable ordinals, providing insights into the nature of machine intelligence and its limits.

Contribution

It defines a novel, simplified measure of intelligence for idealized agents and demonstrates its implications for understanding intelligence hierarchies and the impossibility of an intelligence explosion.

Findings

Higher intelligence levels imply knowledge of more complex ordinals.

Results suggest limits on the potential for an intelligence explosion.

The measure offers a new perspective on formalizing machine intelligence.

Abstract

We define a notion of the intelligence level of an idealized mechanical knowing agent. This is motivated by efforts within artificial intelligence research to define real-number intelligence levels of complicated intelligent systems. Our agents are more idealized, which allows us to define a much simpler measure of intelligence level for them. In short, we define the intelligence level of a mechanical knowing agent to be the supremum of the computable ordinals that have codes the agent knows to be codes of computable ordinals. We prove that if one agent knows certain things about another agent, then the former necessarily has a higher intelligence level than the latter. This allows our intelligence notion to serve as a stepping stone to obtain results which, by themselves, are not stated in terms of our intelligence notion (results of potential interest even to readers totally skeptical that our notion correctly captures intelligence). As an application, we argue that these results comprise evidence against the possibility of intelligence explosion (that is, the notion that sufficiently intelligent machines will eventually be capable of designing even more intelligent machines, which can then design even more intelligent machines, and so on).