ARC Is a Vision Problem!

TL;DR

This paper introduces Vision ARC (VARC), a vision-based approach to the ARC problem, framing it as image-to-image translation using vision transformers, achieving competitive accuracy without relying on language models.

Contribution

The paper presents a novel vision-centric formulation of ARC and demonstrates that standard vision architectures can effectively solve abstract reasoning tasks.

Findings

Achieves 60.4% accuracy on ARC-1 benchmark

Outperforms existing scratch-trained methods

Close to human performance

Abstract

The Abstraction and Reasoning Corpus (ARC) is designed to promote research on abstract reasoning, a fundamental aspect of human intelligence. Common approaches to ARC treat it as a language-oriented problem, addressed by large language models (LLMs) or recurrent reasoning models. However, although the puzzle-like tasks in ARC are inherently visual, existing research has rarely approached the problem from a vision-centric perspective. In this work, we formulate ARC within a vision paradigm, framing it as an image-to-image translation problem. To incorporate visual priors, we represent the inputs on a "canvas" that can be processed like natural images. It is then natural for us to apply standard vision architectures, such as a vanilla Vision Transformer (ViT), to perform image-to-image mapping. Our model is trained from scratch solely on ARC data and generalizes to unseen tasks through test-time training. Our framework, termed Vision ARC (VARC), achieves 60.4% accuracy on the ARC-1 benchmark, substantially outperforming existing methods that are also trained from scratch. Our results are competitive with those of leading LLMs and close the gap to average human performance.