P: A Universal Measure of Predictive Intelligence

TL;DR

This paper proposes a universal measure of intelligence based on prediction accuracy and complexity, aiming to compare humans, animals, and AI systems on a single scale.

Contribution

It introduces a new theoretical framework for measuring intelligence through prediction, using Kolmogorov complexity, and demonstrates its practical feasibility.

Findings

Successfully measured intelligence of agents in a virtual maze

Evaluated prediction of time-series data

Proposes a unified scale for comparing different intelligent systems

Abstract

Over the last thirty years, considerable progress has been made with the development of systems that can drive cars, play games, predict protein folding and generate natural language. These systems are described as intelligent and there has been a great deal of talk about the rapid increase in artificial intelligence and its potential dangers. However, our theoretical understanding of intelligence and ability to measure it lag far behind our capacity for building systems that mimic intelligent human behaviour. There is no commonly agreed definition of the intelligence that AI systems are said to possess. No-one has developed a practical measure that would enable us to compare the intelligence of humans, animals and AIs on a single ratio scale. This paper sets out a new universal measure of intelligence that is based on the hypothesis that prediction is the most important component of intelligence. As an agent interacts with its normal environment, the accuracy of its predictions is summed up and the complexity of its predictions and perceived environment is accounted for using Kolmogorov complexity. Two experiments were carried out to evaluate the practical feasibility of the algorithm. These demonstrated that it could measure the intelligence of an agent embodied in a virtual maze and an agent that makes predictions about time-series data. This universal measure could be the starting point for a new comparative science of intelligence that ranks humans, animals and AIs on a single ratio scale.