Using Thought-Provoking Children's Questions to Drive Artificial Intelligence Research

TL;DR

This paper introduces a new method for advancing AI by using children's thought-provoking questions to evaluate reasoning and learning capabilities, highlighting their broad coverage of intelligence aspects.

Contribution

It proposes the TPCQ task utilizing BrainPlay questions to assess AI reasoning and learning, and provides an analysis of question types and their implications.

Findings

BrainPlay questions cover many intelligence aspects.

Answers and generalizations are often open-ended, requiring human judgment.

The TPCQ task effectively stimulates AI reasoning and learning.

Abstract

We propose to use thought-provoking children's questions (TPCQs), namely Highlights BrainPlay questions, as a new method to drive artificial intelligence research and to evaluate the capabilities of general-purpose AI systems. These questions are designed to stimulate thought and learning in children, and they can be used to do the same thing in AI systems, while demonstrating the system's reasoning capabilities to the evaluator. We introduce the TPCQ task, which which takes a TPCQ question as input and produces as output (1) answers to the question and (2) learned generalizations. We discuss how BrainPlay questions stimulate learning. We analyze 244 BrainPlay questions, and we report statistics on question type, question class, answer cardinality, answer class, types of knowledge needed, and types of reasoning needed. We find that BrainPlay questions span many aspects of intelligence. Because the answers to BrainPlay questions and the generalizations learned from them are often highly open-ended, we suggest using human judges for evaluation.