On the Importance of Critical Period in Multi-stage Reinforcement Learning

TL;DR

This paper explores the significance of the critical period in multi-stage reinforcement learning, emphasizing the importance of appropriate stimuli during early learning stages to improve AI performance and stability.

Contribution

It introduces a multi-stage reinforcement learning framework that focuses on identifying and utilizing appropriate stimuli during the critical period for enhanced learning outcomes.

Findings

Improved AI performance during critical periods

Enhanced learning efficiency and stability

Demonstrated the importance of stage-specific guidance

Abstract

The initial years of an infant's life are known as the critical period, during which the overall development of learning performance is significantly impacted due to neural plasticity. In recent studies, an AI agent, with a deep neural network mimicking mechanisms of actual neurons, exhibited a learning period similar to human's critical period. Especially during this initial period, the appropriate stimuli play a vital role in developing learning ability. However, transforming human cognitive bias into an appropriate shaping reward is quite challenging, and prior works on critical period do not focus on finding the appropriate stimulus. To take a step further, we propose multi-stage reinforcement learning to emphasize finding ``appropriate stimulus" around the critical period. Inspired by humans' early cognitive-developmental stage, we use multi-stage guidance near the critical period, and demonstrate the appropriate shaping reward (stage-2 guidance) in terms of the AI agent's performance, efficiency, and stability.