Johnny: Structuring Representation Space to Enhance Machine Abstract Reasoning Ability

TL;DR

This paper introduces Johnny, a novel representation space framework, and Spin-Transformer, a new architecture, significantly improving AI's ability to perform complex abstract reasoning tasks like Raven's Progressive Matrices.

Contribution

The paper proposes Johnny, a representation space-based model, and Spin-Transformer, an architecture enhancing positional relationship modeling for better reasoning performance.

Findings

Johnny outperforms traditional RPM models.

Spin-Transformer improves reasoning accuracy with reduced computation.

The combined approach advances AI abstract reasoning capabilities.

Abstract

This paper thoroughly investigates the challenges of enhancing AI's abstract reasoning capabilities, with a particular focus on Raven's Progressive Matrices (RPM) tasks involving complex human-like concepts. Firstly, it dissects the empirical reality that traditional end-to-end RPM-solving models heavily rely on option pool configurations, highlighting that this dependency constrains the model's reasoning capabilities. To address this limitation, the paper proposes the Johnny architecture - a novel representation space-based framework for RPM-solving. Through the synergistic operation of its Representation Extraction Module and Reasoning Module, Johnny significantly enhances reasoning performance by supplementing primitive negative option configurations with a learned representation space. Furthermore, to strengthen the model's capacity for capturing positional relationships among local features, the paper introduces the Spin-Transformer network architecture, accompanied by a lightweight Straw Spin-Transformer variant that reduces computational overhead through parameter sharing and attention mechanism optimization. Experimental evaluations demonstrate that both Johnny and Spin-Transformer achieve superior performance on RPM tasks, offering innovative methodologies for advancing AI's abstract reasoning capabilities.