Multi-Perspective Transformers in ARC-AGI-2 Challenge

TL;DR

This paper presents a method using TinyLM with test-time fine-tuning techniques to solve ARC-AGI-2 visual puzzles, achieving high training accuracy but moderate evaluation performance.

Contribution

It introduces a multi-perspective transformer approach with test-time training and products of experts for solving visual puzzles in ARC-AGI-2.

Findings

96.1% training accuracy

21.7% evaluation accuracy

demonstrates potential of test-time fine-tuning techniques

Abstract

ARC-AGI-2 is a benchmark of human-intuitive visual puzzles that measures a machine's ability to generalize from limited examples, interpret symbolic meaning, and flexibly apply rules in varying contexts. In this paper, we discuss our approach to solving the ARC-AGI-2 puzzles with TinyLM, with additional fine-tuning at test time, including Test-Time-Training (TTT) and Products of Experts (POE). Our model achieves 96.1% accuracy on the training set and 21.7% accuracy on the evaluation set.