Measuring Intelligence through Games

TL;DR

This paper explores how games can serve as practical benchmarks for measuring artificial general intelligence by extending the concept of universal intelligence to finite time and sampling game spaces.

Contribution

It proposes a method to evaluate general intelligence through game sampling and extends the universal intelligence framework for practical, finite-time testing.

Findings

Games share properties useful for testing AI

Proposes a sampling-based approach for practical testing

Extends universal intelligence to finite time

Abstract

Artificial general intelligence (AGI) refers to research aimed at tackling the full problem of artificial intelligence, that is, create truly intelligent agents. This sets it apart from most AI research which aims at solving relatively narrow domains, such as character recognition, motion planning, or increasing player satisfaction in games. But how do we know when an agent is truly intelligent? A common point of reference in the AGI community is Legg and Hutter's formal definition of universal intelligence, which has the appeal of simplicity and generality but is unfortunately incomputable. Games of various kinds are commonly used as benchmarks for "narrow" AI research, as they are considered to have many important properties. We argue that many of these properties carry over to the testing of general intelligence as well. We then sketch how such testing could practically be carried out. The central part of this sketch is an extension of universal intelligence to deal with finite time, and the use of sampling of the space of games expressed in a suitably biased game description language.