Boosting Reinforcement Learning in 3D Visuospatial Tasks Through Human-Informed Curriculum Design

TL;DR

This paper explores enhancing reinforcement learning in complex 3D visuospatial tasks by integrating human-informed curriculum design, demonstrating improved learning outcomes over traditional methods.

Contribution

It introduces a curriculum learning approach guided by human experiment insights to improve RL performance in complex 3D visuospatial tasks.

Findings

Curriculum learning significantly improved RL success rates.

Traditional RL methods struggled with the task without curriculum guidance.

Human insights effectively informed curriculum design.

Abstract

Reinforcement Learning is a mature technology, often suggested as a potential route towards Artificial General Intelligence, with the ambitious goal of replicating the wide range of abilities found in natural and artificial intelligence, including the complexities of human cognition. While RL had shown successes in relatively constrained environments, such as the classic Atari games and specific continuous control problems, recent years have seen efforts to expand its applicability. This work investigates the potential of RL in demonstrating intelligent behaviour and its progress in addressing more complex and less structured problem domains. We present an investigation into the capacity of modern RL frameworks in addressing a seemingly straightforward 3D Same-Different visuospatial task. While initial applications of state-of-the-art methods, including PPO, behavioural cloning and imitation learning, revealed challenges in directly learning optimal strategies, the successful implementation of curriculum learning offers a promising avenue. Effective learning was achieved by strategically designing the lesson plan based on the findings of a real-world human experiment.