Raven's Progressive Matrices Completion with Latent Gaussian Process Priors

TL;DR

This paper introduces a deep latent variable model using Gaussian process priors to interpret and generate answers for Raven's Progressive Matrices, demonstrating effective extrapolation and interpretability with minimal training data.

Contribution

It proposes a novel latent Gaussian process-based model for RPM that enables answer generation, interpretability, and extrapolation beyond standard classification approaches.

Findings

Requires few training samples for high-quality answer painting

Can generate novel RPM panels

Provides interpretability through concept-specific latent variables

Abstract

Abstract reasoning ability is fundamental to human intelligence. It enables humans to uncover relations among abstract concepts and further deduce implicit rules from the relations. As a well-known abstract visual reasoning task, Raven's Progressive Matrices (RPM) are widely used in human IQ tests. Although extensive research has been conducted on RPM solvers with machine intelligence, few studies have considered further advancing the standard answer-selection (classification) problem to a more challenging answer-painting (generating) problem, which can verify whether the model has indeed understood the implicit rules. In this paper we aim to solve the latter one by proposing a deep latent variable model, in which multiple Gaussian processes are employed as priors of latent variables to separately learn underlying abstract concepts from RPMs; thus the proposed model is interpretable in terms of concept-specific latent variables. The latent Gaussian process also provides an effective way of extrapolation for answer painting based on the learned concept-changing rules. We evaluate the proposed model on RPM-like datasets with multiple continuously-changing visual concepts. Experimental results demonstrate that our model requires only few training samples to paint high-quality answers, generate novel RPM panels, and achieve interpretability through concept-specific latent variables.