Solving Verbal Comprehension Questions in IQ Test by Knowledge-Powered Word Embedding

TL;DR

This paper presents a novel AI framework that uses knowledge-powered word embeddings to automatically solve verbal comprehension questions in IQ tests, addressing challenges like multiple word senses and complex relations.

Contribution

The work introduces a new multi-sense aware word embedding method and a question-type classifier, improving AI performance on verbal IQ questions beyond existing methods.

Findings

Outperforms existing methods in solving verbal IQ questions.

Exceeds average human performance on the test set.

Demonstrates potential for AI to approach human-level verbal reasoning.

Abstract

Intelligence Quotient (IQ) Test is a set of standardized questions designed to evaluate human intelligence. Verbal comprehension questions appear very frequently in IQ tests, which measure human's verbal ability including the understanding of the words with multiple senses, the synonyms and antonyms, and the analogies among words. In this work, we explore whether such tests can be solved automatically by artificial intelligence technologies, especially the deep learning technologies that are recently developed and successfully applied in a number of fields. However, we found that the task was quite challenging, and simply applying existing technologies (e.g., word embedding) could not achieve a good performance, mainly due to the multiple senses of words and the complex relations among words. To tackle these challenges, we propose a novel framework consisting of three components. First, we build a classifier to recognize the specific type of a verbal question (e.g., analogy, classification, synonym, or antonym). Second, we obtain distributed representations of words and relations by leveraging a novel word embedding method that considers the multi-sense nature of words and the relational knowledge among words (or their senses) contained in dictionaries. Third, for each type of questions, we propose a specific solver based on the obtained distributed word representations and relation representations. Experimental results have shown that the proposed framework can not only outperform existing methods for solving verbal comprehension questions but also exceed the average performance of the Amazon Mechanical Turk workers involved in the study. The results indicate that with appropriate uses of the deep learning technologies we might be a further step closer to the human intelligence.