ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning

TL;DR

ARCLE is a reinforcement learning environment designed to tackle the complex Abstraction and Reasoning Corpus, demonstrating that advanced RL techniques can learn tasks within this challenging benchmark and opening new research avenues.

Contribution

The paper introduces ARCLE, a novel environment for RL research on ARC, and shows that proximal policy optimization with auxiliary losses improves task learning performance.

Findings

Proximal policy optimization can learn ARC tasks effectively.

Non-factorial policies and auxiliary losses enhance performance.

ARCLE enables exploration of advanced RL methods like MAML and GFlowNets.

Abstract

This paper introduces ARCLE, an environment designed to facilitate reinforcement learning research on the Abstraction and Reasoning Corpus (ARC). Addressing this inductive reasoning benchmark with reinforcement learning presents these challenges: a vast action space, a hard-to-reach goal, and a variety of tasks. We demonstrate that an agent with proximal policy optimization can learn individual tasks through ARCLE. The adoption of non-factorial policies and auxiliary losses led to performance enhancements, effectively mitigating issues associated with action spaces and goal attainment. Based on these insights, we propose several research directions and motivations for using ARCLE, including MAML, GFlowNets, and World Models.