Incremental procedural and sensorimotor learning in cognitive humanoid robots

TL;DR

This paper presents a cognitive humanoid robot that incrementally learns complex movements and behaviors by adding new functions and reward terms, inspired by Piaget's sensorimotor stages, demonstrating effective task solving in simulation.

Contribution

It introduces a novel incremental learning framework based on CONAIM and reinforcement learning, inspired by Piaget's theory, for complex behavior acquisition in humanoid robots.

Findings

Successfully learned complex object tracking tasks incrementally.

Adding new reward terms improves learning of complex behaviors.

The approach demonstrates scalable cognitive complexity management.

Abstract

The ability to automatically learn movements and behaviors of increasing complexity is a long-term goal in autonomous systems. Indeed, this is a very complex problem that involves understanding how knowledge is acquired and reused by humans as well as proposing mechanisms that allow artificial agents to reuse previous knowledge. Inspired by Jean Piaget's theory's first three sensorimotor substages, this work presents a cognitive agent based on CONAIM (Conscious Attention-Based Integrated Model) that can learn procedures incrementally. Throughout the paper, we show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Experiments were conducted with a humanoid robot in a simulated environment modeled with the Cognitive Systems Toolkit (CST) performing an object tracking task. The system is modeled using a single procedural learning mechanism based on Reinforcement Learning. The increasing agent's cognitive complexity is managed by adding new terms to the reward function for each learning phase. Results show that this approach is capable of solving complex tasks incrementally.